Numbers & Computer Representations
P2.math-foundations.01 · Audience: guest, it-ml, language-pro
A computer can only do one thing: process numbers. There is no such thing as a letter, a word, or a sentence inside a CPU — everything is first turned into a stream of numbers. For language, that means each word (or word-piece) becomes a whole number called a token ID, which is then looked up to retrieve a list of decimal numbers called an embedding vector. Every computation in a Transformer — attention, matrix multiplication, softmax — runs on these numbers.
Two kinds of number appear everywhere in this tutorial. An integer is a whole number with no decimal point (0, 1, 42, −5) — Python type int. Token IDs are integers. A float (floating-point) can have a decimal point (3.14, −0.87, 0.0) — Python type float. Embedding values are floats.
🗣️ From a linguist's perspective: Text as numerical encoding — a linguist's entry point
| Type | What it is | Role in Transformers | Example |
|---|---|---|---|
Integer (int) | Whole number, no decimal point | Token ID, vocabulary size, sequence length | 4231 |
Float (float) | Number with a decimal point — may be tiny or huge | Embedding values, attention weights, loss | 0.1245 |
Ask the mentor about this module
Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.
Try it yourself
A scratch console for this page's ideas — ungraded, nothing you run here is recorded.
Scratch console
A scratch console with the scientific stack (pandas, numpy, scikit-learn). Runs on the server — no network, resource-limited and measured.
Output appears here.
Where next?
Later in Mathematics Foundations