How did giving neural networks a 'memory loop' allow them to finally understand sequences?

The answer

Recurrent Neural Networks (RNNs) introduced the concept of feeding a network's output back into itself as input, giving it a form of memory that enables processing sequential data like text, speech, and time series. The foundational ideas trace back to John Hopfield's 1982 network and were formalized by Jeffrey Elman's 'simple recurrent network' in 1990 and Michael Jordan's architecture in 1986. RNNs maintain a hidden state that acts as a dynamic memory, updated at each time step, theoretically allowing them to capture dependencies of arbitrary length. In practice, early RNNs suffered from the vanishing gradient problem - gradients shrink exponentially during backpropagation through time - which limited their ability to learn long-range dependencies until the invention of LSTM networks by Hochreiter and Schmidhuber in 1997.