From rules to learned patterns, without the maths
P10.orientation.03 · Audience: guest, it-ml, language-pro
This is the idea that separates old language technology from the new kind. Instead of a person writing the rules, the computer learns the patterns from examples — a process called training. Along the way two more words show up that you will meet properly later: token and embedding. We introduce them here in plain language so they feel familiar when the maths arrives. Still no maths, no code. Your glossary is one click away.
🗣️ From a linguist's perspective: learning a language by immersion, not by rulebook
There are two ways to learn a language. One is the rulebook: memorise grammar tables and vocabulary lists. The other is immersion: hear thousands of real sentences until the patterns sink in and the right word simply sounds right. Old language software was built the rulebook way — people wrote the rules by hand. Modern language models are built the immersion way: they read an enormous amount of text and absorb the patterns. That single switch — from stated rules to absorbed patterns — is what training means.
For decades, language software worked from hand-written rules. To build a grammar checker, experts wrote out thousands of rules: a singular subject takes a singular verb; ‘their’ is possessive, ‘they’re’ is a contraction, and on and on. Early machine translation was rule tables mapping one language's structures onto another's.
This can work — but it is brittle. Language is endlessly irregular: idioms, exceptions, slang, tone, and living change. Every gap needs a new hand-written rule, the rules start to contradict one another, and no team can ever finish. If you have used an old grammar checker that flagged perfectly good sentences and missed obvious errors, you have felt the limits of the rulebook approach.
Here is that “map of meaning” in one picture — words used in similar ways land close together, which is exactly what an embedding gives the model:
📚 Go Further
The rules-to-patterns shift, in plain terms.
- Training — showing a model many examples so it adjusts itself to predict text better; patterns absorbed, not rules written. See your glossary.
- Token — the piece of text a model works in (a word or a word-chunk). See your glossary.
- Embedding — turning words into numbers so that similar use means nearby values. See your glossary.
- In-app: P10.text-representation zooms into tokens and embeddings properly, once you want the machinery.
Ask the mentor about this module
Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.
Where next?
Continue
Why there are limits: memory, compute and context →Orientation — concepts before the maths · P10
Later in Orientation — concepts before the maths