Your goals
What are you here for? We lead with your goals across the app - the dashboard, recommendations, and your weekly report follow them. Optional.
No goals yet - describe one in your own words, or pick a common goal below.
Start here
Understand Transformers & BERT
From zero to a working understanding of how modern language AI works - built for language professionals: just a little plain-language maths (no mathematician background needed) and no coding experience assumed to start.
Begin / ContinueNot this one?
Switch to a different path
You are reading Understand Transformers & BERT. Each of these replaces it with a separate path that starts from its own beginning -- not a later stage of the one above, and not a sampler.
- Reach DELF B2 in FrenchFor an adult who already speaks some French and wants B2 - all four epreuves worked, spoken-first, with the rules demonstrated rather than asserted. Start with the diagnostic: it finds where your progress is actually blocked, and the app reorders itself around the answer.Switch to this path
- Understand security: protocols, identity and secure architectureFor a technical learner who wants the mechanisms rather than a checklist - each idea worked from the primitive up, and every attack shown with the defence that answers it. Start with the crypto: everything after it is built on those few primitives.Switch to this path
- Learn to code from zero, on your own materialFor someone who has never written a line of code. Nothing here assumes you know what a variable, a function or an error is - each one is met when you need it, using words and texts from your own work rather than toy examples. Every step is listed, so you always know what comes next.Switch to this path
- Go from writing Python to engineering itFor someone who already ships Python and wants the depth behind the habits. Twelve doors rather than a reading list - take them in order for the full arc, or go straight to the one you came for. Each ends in a lab, a project or an interview set.Switch to this path
Optional -- tell us roughly where you're starting on each axis and we'll point Begin at the right stage. These are independent: knowing about LLMs does not assume you code, and coding does not assume you know the concepts. Nothing is hidden or locked -- every stage below stays open.
You'll be able to: Explain, in plain words, what a model, an algorithm and a language model are.
Begin from what you already know about language and build the core ideas -- model, algorithm, language model -- in plain words. Your glossary is one click away whenever a term is new.
You already work with language every day -- grammar, register, prediction, ambiguity. NLP is a machine doing a narrow version of that. We start from your expertise, not from maths.
You'll be able to: Use LLMs well for translation, terminology and register -- and know where they fail.
What these models can and can't do, and how to use them well for translation, terminology and register.
You'll be able to: Read the numbers and statistics in AI claims critically -- no formulas required.
The statistics literacy you actually need - correlation vs causation, reading numbers, uncertainty - no formulas required.
You'll be able to: Judge ML claims, trust and governance questions in your work -- with no maths.
The plain-language literacy to work with ML confidently: what it is and isn't, how to question and judge it, and the ethics - no maths.
You don't need to build models to work well with them -- you need to know what they can promise, how to question them, and where the risks lie.
A quick recap before the hands-on part. You have built the conceptual core: what a model, an algorithm and a language model are; how to use LLMs well and where they fail; how to read the numbers in AI claims; and how to judge ML in your work. Nothing here is graded -- just check the ideas landed.
In your own words, what is the difference between a model and an algorithm?
An algorithm is a fixed set of steps (a recipe). A model is a set of learned patterns that turns an input into an output. A language model is a model whose job is to predict likely next words.
Why can an LLM sound confident and still be wrong?
It predicts fluent, likely-sounding text from patterns -- it has no built-in check that a statement is true. Fluency is not accuracy, which is why grounding and verification matter.
Name one task where an LLM helps your work, and one where it is risky.
Helpful: drafting, rephrasing, terminology suggestions, a first-pass translation. Risky: anything needing guaranteed accuracy, confidentiality or accountability -- legal or medical wording, private data, final sign-off.
You'll be able to: Read, run and lightly edit a snippet of Python without fear.
Get comfortable reading, running and lightly editing code. You don't need to become a programmer.
A gate for the hands-on parts below - just like a new human language, you read and tweak real code before you ever write your own; read-and-edit first, no blank page, no need to become a programmer.
You'll be able to: Reuse code with functions and automate a small real task end to end.
Reuse code with functions and automate a small real task end to end.
You'll be able to: Read an embedding as a vector, measure similarity with the dot product and cosine, and see how softmax turns scores into attention weights.
Just enough plain-language maths to see how meaning becomes numbers and how attention decides what matters: vectors and the dot product (similarity), and softmax (turning scores into weights).
No mathematician background needed. These are the few ideas the machinery below rests on -- a word as an arrow, how closely two arrows point the same way (their meaning), and how a list of scores becomes a set of weights that add up to one.
You'll be able to: Follow how text becomes vectors, how attention works, and the Transformer behind BERT.
For the curious - see the machinery: how text becomes vectors, attention, and the Transformer that powers BERT.
Heads up - a little more maths here
These steps lean on a little more maths than the 'A little maths' stage above. You can follow the ideas without it, or take the on-ramp first: positional encoding uses sin/cos (trigonometry), and the encoder block adds matrix operations, layer-norm (variance) and -- for the feed-forward network -- backpropagation.
You'll be able to: Optional: reach the maths, hardware and neural-network foundations underneath it all.
If you want the foundations underneath it all: the maths, how computers actually run these models (memory, CPU, GPU and their limits), and how neural networks learn.
One last recap before you run the whole thing on your own text. You have seen how text becomes vectors, how attention decides what matters, and how the Transformer builds a sentence's meaning. Nothing here is graded -- just make sure the pipeline hangs together.
What does tokenization do, and why is it needed?
It splits text into the small units (tokens) the model actually reads -- whole words or word-pieces -- so language becomes a sequence of items that can each be turned into a vector.
How does the dot product relate to attention?
An attention score is the dot product of two tokens' vectors: a high value means they point the same way, so one token 'wants' what the other offers. Softmax then turns those scores into weights that sum to one.
What does a sentence embedding capture?
A single vector that summarises the whole sentence's meaning, built from the contextual token vectors -- so sentences with similar meaning sit close together in the space.
Want to revisit?
You'll be able to: See tokenisation, embeddings, attention and sentence meaning work on your own sentences.
The payoff: run the whole pipeline you have just met -- tokens, embeddings, attention, sentence meaning -- on a few sentences of your own, and watch the ideas work on your material.
Everything so far ran on a fixed demo. Now bring your own -- a line you translated, a tricky term, a sentence from your work -- and see the machine read it exactly the way we have been describing.
Not sure yet? Browse the full curriculum.