What an LLM actually is
P10.llm-essentials.01 · Audience: guest, it-ml, language-pro
Take it one idea at a time: what a large language model is, what "predicting the next word" really means, what it did and did not learn from text, and why "sounds right" is not the same as "is right". Each step has a worked example. No maths required.
ⓘ Concept: One mental model to keep
Picture an extremely well-read autocomplete. It has read a huge amount of writing and learned how language tends to continue. Given everything so far, it guesses the most plausible next few words — again and again. That one idea explains most of its strengths and every one of its risks.
Why it matters — This single idea — plausible next-word prediction — explains why LLMs are fluent, why they are confidently wrong, and why a human must stay on meaning and truth.
A large language model (LLM) is a system trained to predict the most likely next chunk of text given everything before it. During training it read enormous amounts of writing and learned the statistical patterns of how language tends to continue.
At the moment you use it, it generates one small piece at a time, and each choice is shaped by the running context — your prompt plus what it has written so far. That is why the output reads as fluent and on-topic.
Worked example — watch it continue, word by word
Give the model the start of a sentence and ask only for the next word. Here is the kind of ranked guess it works from:
Prompt: “The interpreter took a deep breath before stepping into the …”
| Next word it considers | How plausible (rough) |
|---|---|
| booth | very likely |
| room | likely |
| meeting | possible |
| banana | almost never |
It picks a highly plausible word — say “booth” — appends it, and repeats: now it is predicting the word after “…stepping into the booth”, and so on. No sentence is planned in advance; the sentence emerges one plausible step at a time.
The crucial implication for language work: it optimises for plausibility, not truth. It produces what sounds like a good continuation. That usually lines up with being correct — but not always — and there is no built-in fact-checker inside it.
⚡ Interview Ref — the quick-scan Reference face
In plain language: what a large language model is, what "predicting the next word" really means, what it did and did not learn from text, and why "sounds right" is not the same as "is right". No maths required.
1 — What it is, and what "predicting the next word" means (l1)
A large language model (LLM) is a system trained to predict the most likely next chunk of text given everything before it. It learned the statistical patterns of how language tends to continue, and at use time it generates one small piece at a time, each choice shaped by the running context. The crucial implication: it optimises for plausibility, not truth — what sounds like a good continuation — with no built-in fact-checker inside it.
2 — What "trained on text" did and did not teach it (l5)
Training absorbed patterns, style, grammar, and world knowledge implicitly, by adjusting millions of internal settings. It learned how language tends to be expressed — but it did not build a tidy database of facts to look up; knowledge is baked in fuzzily. Two consequences: it can state wrong things confidently (a wrong answer can still be fluent), and it has a knowledge cutoff — after its training date it knows nothing unless you supply it in the prompt.
3 — Does it understand, or simulate? (l31)
These models manipulate language patterns without understanding meaning the way a person does — no beliefs, intentions, or grounded experience. Fluency and apparent comprehension can sit right next to shallow or absent real understanding. The point is to guard against over-trust: treat the model as a powerful language assistant, not an accountable expert, and keep a human on meaning and truth.
Interview one-liners
- An LLM is a well-read autocomplete: it predicts the most plausible next text, not the true text.
- It learned patterns, not a database — fuzzy knowledge, confident mistakes, a fixed cutoff date.
- It simulates understanding rather than having it — so keep a human on meaning and facts.
📚 Go Further
Plain-language explainers on what LLMs are and how they work.
| Type | Resource |
|---|---|
| Explainer | What Is ChatGPT Doing … and Why Does It Work? — Stephen Wolfram |
| Article | On the Dangers of Stochastic Parrots — Bender et al. (2021) — patterns vs meaning |
| In-app | P10.text-representation.01 — Mathematical Foundations, for the mechanics behind these ideas |
Ask the mentor about this module
Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.
🎓 Practice ladder
1 graded rung · ~8 minNow build the idea yourself. The workspace is three panels — instructions, a code editor, and live output with test results. Run checks the visible tests; Submit grades against hidden edge cases; the reference solution unlocks once you pass.
Explorer rung — Predict the next word
Loading exercise…
Where next?
Later in LLM Essentials
Go up a level