Multilingual & translation
P10.llm-language-practice.01 · Audience: guest, it-ml, language-pro
Take it one idea at a time: why the model doesn't see words the way you do, why other languages cost more and read less naturally, why it's weakest in low-resource languages, and where it beats — and loses to — traditional machine translation. Each step has a worked example. No maths required.
ⓘ Concept: One mental model to keep
The model reads text in tokens — sub-word pieces — and those pieces were carved up mostly from English. Almost every cross-language cost, quality, and fairness gap you meet traces back to that single fact.
Why it matters — Almost every cross-language cost, quality, and fairness gap on this page traces back to one fact — the tokenizer was carved up mostly from English.
A token is the unit the model actually reads and writes — usually a sub-word piece, not a whole word. ‘unbelievable’ might be split into ‘un’ / ‘believ’ / ‘able’. The model never sees letters or words the way we do; it sees sequences of these pieces.
Worked example — the same greeting, three languages
Watch the same meaning turn into very different token counts, because the tokenizer was built mostly on English:
| Language | The sentence | Rough tokens |
|---|---|---|
| English | I am very happy to meet you | 7 |
| Turkish | Sizinle tanıştığıma çok mutluyum | 13 |
| Thai | ยินดีที่ได้พบคุณ | 22 |
Same idea, roughly 3× more tokens in Thai than English. Because the model works in these pieces, that inflates cost (you pay per token), uses up the context window faster, and can subtly degrade quality. It also explains odd behaviour with spelling, rhyme, and counting characters — the model isn't working with letters.
⚡ Interview Ref — the quick-scan Reference face
In plain language: why the model doesn't see words the way you do, why other languages cost more and read less naturally, why it's weakest in low-resource languages, and where it beats — and loses to — traditional machine translation. No maths required.
1 — Tokens: why models don't see words as you do (l2)
A token is the unit the model actually reads and writes — usually a sub-word piece, not a whole word. Tokenizers are optimized mostly for English, so other languages — especially those with rich morphology or non-Latin scripts — get split into many more tokens. That inflates cost, uses up the context window faster, and can subtly degrade quality. It also explains odd behaviour with spelling, rhyme, and counting characters — the model isn't working with letters.
2 — Why other languages cost more and read less naturally (l16)
Because tokenizers are trained mostly on English, other languages get broken into more, smaller pieces — especially agglutinative languages and non-Latin scripts. More tokens means higher cost, faster consumption of the context window, and often lower quality. This is a real fairness and budget gap: factor it into cost estimates, chunking, and per-language quality expectations — do not assume one price or one quality bar across languages.
3 — Why it's confidently wrong in low-resource languages (l11)
Models are far stronger in languages with abundant, high-quality training text and weaker in low-resource languages where they saw little data. There you see less idiomatic fluency, more grammatical slips, more hallucination, and inconsistent terminology — while the output still looks confident. Never assume parity across languages: demand evidence of quality per language and locale rather than a general claim of "multilingual support".
4 — Where LLMs excel and break in machine translation (l15)
LLMs translate very fluently and use context well — tone, ambiguity, and whole-document coherence — and follow instructions like "keep it formal". They break on consistency and control: silently dropping or adding content, varying terminology, and hallucinating on names and numbers. The strongest workflows combine them — LLM/MT drafts, human post-editing owns quality, and glossaries + source grounding constrain the model.
Interview one-liners
- The model reads tokens (sub-word pieces), carved mostly from English — so non-English text costs more and can read less naturally.
- Never assume parity across languages — low-resource languages are confidently wrong; demand per-locale evidence.
- LLMs beat old MT on fluency & context, lose on consistency & control — best used with glossaries, grounding, and human post-editing.
📚 Go Further
Plain-language explainers on multilingual behaviour and translation.
| Type | Resource |
|---|---|
| Study | Language Model Tokenizers Introduce Unfairness Between Languages — Petrov et al. (2023) |
| In-app | P10.text-representation.04 — Byte-Pair Encoding, for how sub-word pieces are formed |
| In-app | P10.llm-language-practice.03 — Terminology & judging quality, for constraining and checking translations |
Ask the mentor about this module
Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.
🎓 Practice ladder
3 graded rungs · ~26 minThree short, scaffolded rungs turn "tokens cost money" into code you run. The workspace is three panels — instructions, a code editor, and live output with test results. Run checks the visible tests; Submit grades against hidden edge cases; the reference solution unlocks once you pass.
Explorer rung 1 — Count the real tokens
Loading exercise…
Explorer rung 2 — Why your language costs more
Loading exercise…
Explorer rung 3 — Will it fit the window?
Loading exercise…
Where next?
Later in LLM Language Practice
Go up a level