Vectors
P2.math-foundations.03 · Audience: guest, it-ml, language-pro · Prerequisites: Functions & Graphs
An embedding vector is nothing more mysterious than a list of numbers arranged in a fixed order. Once a word has become a token and then a list of decimal numbers, that list is called a vector, and everything a Transformer does — comparing words, blending meanings, scoring attention — is arithmetic on these lists. This module gives you both the picture (arrows on a page) and the three operations (add, scale, and the all-important dot product) that power every later step.
A vector is simply an ordered list of numbers. The order matters: the list [3, 1] is a different vector from [1, 3], just as the letters of a word cannot be shuffled without changing the word. You already use vectors every day — a GPS position is two numbers (latitude, longitude); a colour on a screen is three numbers (how much red, green, and blue).
Each number in the list sits in one slot, and each slot is called a dimension. A colour has 3 slots. A word stored by a language model may have hundreds of slots — a common model uses 768 — each holding one decimal that records how strongly the word relates to one learned shade of meaning, such as how animal-like, how big, or how positive it is. You never have to picture 768 directions; a dimension is just a numbered slot in a list.
🗣️ From a linguist's perspective: Vectors as ordered feature profiles for words
ⓘ Concept: A vector is an ordered list of numbers
Why it matters — Every stored word, and every intermediate result a Transformer produces while reading a sentence, is a vector. Vectors are the single basic building block of the whole machine — get comfortable with them and the rest is combinations of three simple moves.
Here is one concrete example — a 4-slot vector and the value in each slot.
| Slot number | Component | Value |
|---|---|---|
| 0 | first | 1.2 |
| 1 | second | −0.5 |
| 2 | third | 3.0 |
| 3 | fourth | 0.8 |
Practice
Work through these three graded exercises to lock in adding, length, and the dot product.
Ask the mentor about this module
Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.
Loading exercise…
Loading exercise…
Loading exercise…
Try it yourself
A scratch console for this page's ideas — ungraded, nothing you run here is recorded.
Scratch console
A scratch console with the scientific stack (pandas, numpy, scikit-learn). Runs on the server — no network, resource-limited and measured.
Output appears here.
Where next?
Later in Mathematics Foundations