How Large Language Models Work

Large Language Models — the AI behind ChatGPT, Claude, and Gemini — work in a surprisingly elegant way. Understanding the core idea helps you use them more effectively.

The core task: predicting the next word

LLMs were trained by being given enormous amounts of text (the internet, books, articles) with one job: predict what word comes next in a sequence. Over trillions of examples, the model gets extremely good at this — and in the process, seems to learn grammar, facts, reasoning patterns, and writing styles.

Generation: one word at a time

When you ask an LLM a question, it does not "think" about an answer and then write it. It generates the response token by token, each time predicting the most likely continuation based on everything that has come before. This is why responses unfold gradually, and why the model cannot "go back and change" something it already wrote.

Why they can seem so capable

Being an extremely good predictor of language turns out to require encoding an enormous amount about how the world works. To predict that "The capital of France is ___" should be completed with "Paris", the model needs to have encoded that fact. To predict a continuation of a logical argument, it needs to have encoded patterns of logical reasoning.

Why they hallucinate

LLMs do not have a "fact database" they look things up in. They generate what they predict is likely text. Sometimes the most likely-seeming continuation happens to be wrong — a confident-sounding falsehood. This is fundamental to how they work, not a bug to be patched.

LLMs are prediction engines that have learned so much about language that they can do remarkable things — but they are not thinking, understanding, or checking facts. Keeping this in mind is the key to using them well.