GRU Networks

On Chapter 7's task, naming a handwritten digit 20 steps after reading it, a 24-unit gated recurrent unit (GRU) trained with the recipe that worked for the LSTM reaches 33.6% against the LSTM's 77.3%. Give the same GRU a learning rate that suits it and it reaches 92.7%. Averaged over three seeds, with each cell at its best learning rate, the GRU beats the LSTM, 93.0% against 85.8%, with a fifth fewer parameters. A GRU is an LSTM with two decisions tied together and one state removed, and which of the two wins depends as much on how you compare them as on the cells.