Generating text & context
P10.llm-essentials.02 · Audience: guest, it-ml, language-pro · Prerequisites: What an LLM actually is
Take it one idea at a time: the difference between a "reader" model and a "writer" model, why the same prompt can give a different answer each time, and why a model can only hold so much text in mind at once. Each step has a worked example. No maths required.
ⓘ Concept: One mental model to keep
A model that reads a whole passage to understand it is doing a different job from a model that writes text left to right. Most chat assistants are writers — and a writer that makes fresh choices as it goes will not say things the exact same way twice.
Why it matters — Knowing whether you are dealing with a reader or a writer — and that a writer makes fresh choices as it goes — sets your expectations for reproducibility and drift.
BERT-style models are built to understand. They read the whole passage at once, looking both left and right — strong at classification, sentiment, search relevance, and pulling information out of a text.
GPT-style models are built to generate. They produce text left to right, one piece at a time — strong at writing, translating, summarising, and conversation.
Roughly: BERT is a reader, GPT is a writer. The chat tools you use directly are writers; readers usually work behind the scenes in search and analysis.
Worked example — one sentence, two different jobs
Hand both kinds of model the same half-finished sentence and watch the job differ:
Input: “The training session was surprisingly …”
| Model | What it does with it | Typical use |
|---|---|---|
| Reader (BERT-style) | Fills the blank / judges tone — e.g. sentiment = positive | Sort support tickets, rank search hits, tag sentiment |
| Writer (GPT-style) | Continues it — “… engaging, and the interpreters left with new techniques.” | Draft the follow-up email, translate it, summarise the day |
Same words in, completely different task. Knowing which kind you're dealing with sets your expectations: a writer will happily keep going (and can drift or invent); a reader just labels or extracts and never volunteers new text.
⚡ Interview Ref — the quick-scan Reference face
In plain language: the difference between "reader" and "writer" models, why the same prompt can give different answers each time, and why a model can only hold so much text in mind at once. No maths required.
1 — Reader vs writer: BERT-style vs GPT-style (l4)
BERT-style models read the whole passage at once, both left and right — strong at classification, sentiment, search relevance, and extraction. GPT-style models produce text left to right, one piece at a time — strong at writing, translating, summarising, and conversation. Roughly, BERT is a reader, GPT is a writer. The chat tools you use directly are writers; readers usually work behind the scenes in search and analysis.
2 — Why the same prompt gives different answers (l22)
A writer generates by sampling among the likely next pieces of text; temperature controls how much randomness is allowed, so the same prompt can be worded differently each time. Higher randomness → varied, creative; lower → predictable, repeatable. For brainstorming, variability is a feature; for translation/legal/terminology work, turn it down and add a review step — you cannot assume two runs match.
3 — The context window and long documents (l6)
The context window is all the model can hold at once — instructions + document + reply, and everything must fit. Long documents must be chunked, summarised, or retrieved; even within the window, models attend less to the middle of a long input. Practical consequences: how you chunk, how you carry terminology across pieces, and why a final consistency pass is worth it.
Interview one-liners
- BERT reads, GPT writes — chat assistants are writers; readers hide behind search and analysis.
- Same prompt, different answers: the model samples; temperature is the creativity/repeatability dial — turn it down for precise work.
- The context window is all the model can hold at once (prompt + document + reply); long texts need chunking, summarising, or retrieval — plus a consistency pass.
📚 Go Further
Plain-language explainers on generation and context.
| Type | Resource |
|---|---|
| Article | Lost in the Middle: How Language Models Use Long Contexts — Liu et al. (2023) |
| In-app | P10.text-representation.04 — Byte-Pair Encoding, for how text becomes the tokens a window counts |
| In-app | P10.llm-essentials.03 — Reliability & grounding, for what to do when a long document outruns the window |
Ask the mentor about this module
Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.
🎓 Practice ladder
1 graded rung · ~10 minTurn the idea into code. The workspace is three panels — instructions, a code editor, and live output with test results. Run checks the visible tests; Submit grades against hidden edge cases; the reference solution unlocks once you pass.
Explorer rung — Same prompt, different answers
Loading exercise…
Where next?
Later in LLM Essentials
Go up a level