Attention

Read this sentence: "The animal didn't cross the street because it was tired." You resolved it — the animal, obviously — before you even noticed there was a question. Now change one word: "...because it was wide." Suddenly it means the street. Same grammar, same word order; the referent flipped on the meaning of a single adjective. For you this is effortless. For machines it was, for decades, close to impossible — and the mechanism that finally cracked it is the one this track takes apart.

The idea is almost rude in its simplicity: let every word look at every other word and decide, with fresh numbers each time, how much each one matters to it right now. Nothing is hard-wired — the weights are recomputed from the vectors themselves, which is exactly why it can lean on animal in one sentence and street in its near-twin. To make that decision, each token plays three roles at once: it asks (a query), it advertises what it holds (a key), and it hands over content (a value). Those three words carry the whole track.

You start with the intuition and no equations, then work through a complete attention pass — the dot products, the scaling, the softmax — at a size small enough to check every number against your own arithmetic. The final module answers the question the first two set up: why one attention pattern is never enough, and what changes when the Transformer runs several in parallel, each free to specialise in a different kind of relationship.

The sentences doing the attending are the ones below — put an ambiguity of your own into the corpus and watch which words the weights pick out.