Generating text & context
P10.llm-essentials.02 · Audience: guest, it-ml, language-pro · Prerequisites: What an LLM actually is
Two things surprise almost everyone who starts working with these tools: ask the same question twice and the answer comes back worded differently, and hand over a long document and the model quietly loses track of parts of it. Neither is a malfunction — both follow directly from how a model generates text, and knowing the mechanism tells you when to expect them and what to do about them. This module explains the difference between models built to read and models built to write, why a writer's sampling makes the same prompt vary, and how the context window limits how much text — instructions, document, and reply together — the model can hold in mind at once. Every step has a worked example, and no maths is required.
ⓘ Concept: One mental model to keep
A model that reads a whole passage to understand it is doing a different job from a model that writes text left to right. Most chat assistants are writers — and a writer that makes fresh choices as it goes will not say things the exact same way twice.
Why it matters — Knowing whether you are dealing with a reader or a writer — and that a writer makes fresh choices as it goes — sets your expectations for reproducibility and drift.
BERT-style models are built to understand. They read the whole passage at once, looking both left and right — strong at classification, sentiment, search relevance, and pulling information out of a text.
GPT-style models are built to generate. They produce text left to right, one piece at a time — strong at writing, translating, summarising, and conversation.
Roughly: BERT is a reader, GPT is a writer. The chat tools you use directly are writers; readers usually work behind the scenes in search and analysis.
On model cards and in technical writing these two shapes carry technical names, and it is worth recognising them: a reader is an encoder-only model, a writer is a decoder-only model. There is a third shape, the encoder-decoder, which reads with one stack and writes with another — the shape of classic translation systems, a reader and a writer joined together. The names describe how text flows through the model; the jobs are still exactly reader, writer and translator.
Worked example — one sentence, two different jobs
Hand both kinds of model the same half-finished sentence and watch the job differ:
Input: “The training session was surprisingly …”
| Model | What it does with it | Typical use |
|---|---|---|
| Reader (BERT-style) | Fills the blank / judges tone — e.g. sentiment = positive | Sort support tickets, rank search hits, tag sentiment |
| Writer (GPT-style) | Continues it — “… engaging, and the interpreters left with new techniques.” | Draft the follow-up email, translate it, summarise the day |
Same words in, completely different task. Knowing which kind you're dealing with sets your expectations: a writer will happily keep going (and can drift or invent); a reader just labels or extracts and never volunteers new text.
⚡ Interview Ref — the quick-scan Reference face
In plain language: the difference between "reader" and "writer" models, why the same prompt can give different answers each time, and why a model can only hold so much text in mind at once. No maths required.
1 — Reader vs writer: BERT-style vs GPT-style (l4)
BERT-style models read the whole passage at once, both left and right — strong at classification, sentiment, search relevance, and extraction. GPT-style models produce text left to right, one piece at a time — strong at writing, translating, summarising, and conversation. Roughly, BERT is a reader, GPT is a writer. The chat tools you use directly are writers; readers usually work behind the scenes in search and analysis.
2 — Why the same prompt gives different answers (l22)
A writer generates by sampling among the likely next pieces of text; temperature controls how much randomness is allowed, so the same prompt can be worded differently each time. Higher randomness → varied, creative; lower → predictable, repeatable. For brainstorming, variability is a feature; for translation/legal/terminology work, turn it down and add a review step — you cannot assume two runs match.
3 — The context window and long documents (l6)
The context window is all the model can hold at once — instructions + document + reply, and everything must fit. Long documents must be chunked, summarised, or retrieved; even within the window, models attend less to the middle of a long input. Practical consequences: how you chunk, how you carry terminology across pieces, and why a final consistency pass is worth it.
Interview one-liners
- BERT reads, GPT writes — chat assistants are writers; readers hide behind search and analysis.
- Same prompt, different answers: the model samples; temperature is the creativity/repeatability dial — turn it down for precise work.
- The context window is all the model can hold at once (prompt + document + reply); long texts need chunking, summarising, or retrieval — plus a consistency pass.
📚 Go Further
Plain-language explainers on generation and context.
| Type | Resource |
|---|---|
| Article | Lost in the Middle: How Language Models Use Long Contexts — Liu et al. (2023) |
| In-app | P10.text-representation.04 — Byte-Pair Encoding, for how text becomes the tokens a window counts |
| In-app | P10.llm-essentials.03 — Reliability & grounding, for what to do when a long document outruns the window |
Ask the mentor about this module
Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.
Keeping your files on this device
Off by default. The mentor always gets your file; this only decides whether your own copy stays here. Copies live in this browser only - they do not follow you to another device, and clearing site data removes them.
🎓 Practice ladder
1 graded rung · ~10 minTurn the idea into code. The workspace is three panels — instructions, a code editor, and live output with test results. Run checks the visible tests; Submit grades against hidden edge cases; the reference solution unlocks once you pass.
Explorer rung — Same prompt, different answers
Loading exercise…
Open in Labs: One distribution, one token
My notes on this module
Loading your notes...
Where next?
Later in LLM Essentials
Go up a level