Text Representation

A computer never sees words. By the time any model touches your sentence, the letters are gone — every piece of text has been replaced by lists of numbers, and everything the model will ever know about your sentence has to survive that conversion. This track is about the conversion itself: the least glamorous step in the whole pipeline, and the one every later trick silently depends on.

Here is the puzzle to carry with you. In "she sat on the bank of the river" and "she opened a bank account", the word bank is spelled identically — yet you never confuse the two. If turning words into numbers were a simple lookup table, both banks would get exactly the same vector, and the model would be blind to a distinction any eight-year-old handles without effort. So: one vector for bank, or two? This track builds both answers — the fixed lookup first, then the learned kind — and lets you watch exactly where the simple version breaks.

Along the way you make the same choices real systems make. You split sentences the way BERT actually splits them and discover why unhappiness comes apart into pieces no dictionary contains; you grow a vocabulary with BPE and WordPiece and see why its size is a genuine trade-off rather than a detail; and you place tokens into a space where distance means something, using nothing beyond the vectors and dot products the first module puts in your hands.

The sentences below are the track's shared working material. Swap in your own and every module downstream recomputes its tokens, vocabularies and vectors from what you wrote.