What did the phrase 'Attention Is All You Need' actually change about AI?

One 2017 paper killed the old guard of neural networks and launched the modern AI era.

The answer

Published by eight Google researchers in June 2017, 'Attention Is All You Need' introduced the Transformer architecture, which replaced recurrent and convolutional layers entirely with self-attention mechanisms. This allowed models to process all parts of an input sequence in parallel rather than one step at a time, dramatically speeding up training. The original Transformer achieved state-of-the-art translation scores on English-to-German and English-to-French benchmarks. Nearly every major AI model today - GPT, BERT, PaLM, LLaMA - is built on this foundation.

Source: arXiv

More from AI & Machine Learning

All AI & Machine Learning →