Matrices & Linear Transformations

P2.math-foundations.04 · Audience: guest, it-ml, language-pro · Prerequisites: Vectors

A matrix is nothing more than a rectangular table of numbers arranged in rows and columns. You already know tables — a word-frequency table, a spreadsheet of co-occurrence counts, a bilingual glossary laid out in two columns. What makes a matrix special is not how it looks but what it does: a single matrix multiplication applies one systematic rule to a whole collection of word-vectors at once. Inside a Transformer, every Query, Key, and Value vector is produced by exactly this operation. This module builds the intuition with no maths background assumed.

Step 1 / 5 — A Matrix is a Table of Numbers

A matrix is a rectangular table of numbers arranged in rows and columns. An "m by n" matrix simply has m rows and n columns. A single row on its own — an ordered list of numbers — is the vector you already met; a matrix is just a stack of such rows, one per word.

🗣️ From a linguist's perspective: A matrix as a systematic transformation applied to every word-vector

Think of a grammatical rule applied across a whole paradigm — the way a single rule for forming the plural, or for conjugating a verb, applies uniformly to every word in its class. A matrix is that kind of rule made numerical: one fixed table of numbers that maps every word-vector from one description of meaning into another. A bilingual dictionary is the closest everyday analogy: it maps every word in language A to its equivalent in language B, and each entry is one small rule. A weight matrix does the same, mapping a vector from one space of features into another — and, like a translation, each output draws on all of the input at once, not just one piece of it.

ⓘ Concept: A matrix — the plain terminology

Why it matters — The Transformer's whole store of learned knowledge lives in matrices. The embedding table that turns each token into a vector is a big matrix — one row per word in the vocabulary. Every weight layer that reshapes meaning is a matrix too. Reading a matrix as 'a table of rules' is the single most useful mental model for the rest of this tutorial.

Here is the demo embedding matrix E: four words from our corpus, each described by four numbers. Every row is one word's vector; every column is one shared feature of meaning.

Word	dim 0	dim 1	dim 2	dim 3
lions	0.80	−0.30	0.50	0.20
animals	0.70	−0.20	0.60	0.10
language	−0.10	0.90	−0.20	0.80
word	−0.20	0.80	−0.10	0.70

Notice how the two animal words sit close together, and the two language words sit close together — the numbers already carry a hint of meaning.

Ask the mentor about this module

Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.

Ctrl/Cmd + Enter to send

Try it yourself

A scratch console for this page's ideas — ungraded, nothing you run here is recorded.

Scratch console

A scratch console with the scientific stack (pandas, numpy, scikit-learn). Runs on the server — no network, resource-limited and measured.

Output appears here.

Where next?

Continue

Probability & Statistics Basics →

Mathematics Foundations · P2

Later in Mathematics Foundations

Go up a level