What did the phrase 'Attention Is All You Need' actually change about AI?
One 2017 paper killed the old guard of neural networks and launched the modern AI era.
The answer
Published by eight Google researchers in June 2017, 'Attention Is All You Need' introduced the Transformer architecture, which replaced recurrent and convolutional layers entirely with self-attention mechanisms. This allowed models to process all parts of an input sequence in parallel rather than one step at a time, dramatically speeding up training. The original Transformer achieved state-of-the-art translation scores on English-to-German and English-to-French benchmarks. Nearly every major AI model today - GPT, BERT, PaLM, LLaMA - is built on this foundation.
Source: arXiv
- transformers
- attention
- deep learning
More from AI & Machine Learning
- What radical question did Alan Turing ask in 1950 that launched the entire field of artificial intelligence?
- Can a machine fool you into thinking it's human - and does that mean it's truly intelligent?
- What happened at a small Dartmouth workshop in 1956 that gave artificial intelligence its name?
- How did a 1958 invention called the Perceptron convince people that machines could learn like brains?
- Could a 1966 chatbot really make people believe they were talking to a therapist?
- What was the first robot that could 'reason' about its own actions?