Next-Token Prediction

What it is

Next-token prediction is the fundamental task language models are trained on: given a sequence of tokens, predict what token comes next. The model learns patterns from billions of examples until it can predict likely continuations for any text. Despite this simple objective, this process produces sophisticated language capabilities.

Why it matters

Understanding next-token prediction demystifies how LLMs work. They don't "understand" like humans—they predict statistically likely continuations. This explains both their strengths (producing fluent, coherent text) and limitations (confidently predicting plausible-sounding but incorrect information).