How did neural networks stage their comeback in the 1990s after being twice declared dead?

The answer

The neural network renaissance of the 1990s was driven by several converging breakthroughs. Yann LeCun's 1989 work at Bell Labs demonstrated convolutional neural networks (CNNs) successfully reading handwritten zip codes, leading to the LeNet architecture that was deployed by the U.S. Postal Service to process millions of checks. Vladimir Vapnik's support vector machines (SVMs), introduced in their modern form in 1995, provided a theoretically grounded alternative to neural networks that dominated pattern recognition benchmarks. Meanwhile, Sepp Hochreiter and Jurgen Schmidhuber's 1997 Long Short-Term Memory (LSTM) networks solved the vanishing gradient problem that plagued recurrent networks. By the late 1990s, neural networks had proven themselves in practical applications from speech recognition to financial forecasting, quietly laying the foundation for the deep learning revolution that would explode in the 2010s.