What actually makes Large Language Models work - is it just memorization?

The answer

Large Language Models work by learning statistical patterns across vast text corpora through a process called next-token prediction - given a sequence of words, predict what comes next. Through this deceptively simple objective applied at massive scale, LLMs develop emergent capabilities including reasoning, translation, and code generation that were never explicitly programmed. The key ingredients are the Transformer architecture's self-attention mechanism, enormous training datasets (often trillions of tokens), and massive compute budgets. Whether this constitutes genuine 'understanding' or sophisticated pattern matching remains one of AI's most contested philosophical debates.