LSTM Networks

> Chapter 6's plain RNN gradient collapsed to 0.00000000000000000002 after 20 steps. An LSTM's cell-state gradient, over the identical 20 steps, is 0.378 -- eighteen orders of magnitude larger, from one architectural change: a separate pathway that multiplies by a gate instead of an activation derivative.