Why LLMs Live in Data Centres
P3.computing-fundamentals.03 · Audience: guest, it-ml, language-pro · Prerequisites: GPUs & Parallel Computing
You can use a powerful AI tool on your phone, but you cannot train one on your desktop. Why? The answer is arithmetic you can do yourself: a model's size in bytes, the memory a GPU has to spare, and the extra memory training demands on top of that. This module turns abstract "billions of parameters" into concrete gigabytes — then lets you compute the fit for yourself in two short exercises.
A language model is a pile of parameters (also called weights) — the numbers learned during training. Each is usually a 32-bit floating-point number taking 4 bytes. So the size is simply:
model size (bytes) = number of parameters × bytes per parameter
| Model | Parameters | Size (FP32) | Fits a consumer laptop? |
|---|---|---|---|
| BERT-base | 110 million | ~440 MB | Yes (16 GB) |
| GPT-2 large | 774 million | ~3 GB | Yes |
| Llama 3 8B | 8 billion | ~32 GB | Barely (32 GB, tight) |
| GPT-3 | 175 billion | ~700 GB | No (needs ~10 GPUs) |
| GPT-4 (est.) | >1 trillion | >4,000 GB | No (a data centre) |
🗣️ From a linguist's perspective: Model size as a printed reference library
Quick recap — model scale
- Model size in bytes = parameters × 4 (FP32). GPT-3 is ~700 GB; BERT-base is ~440 MB.
- VRAM is the GPU's own memory — the whole model must fit there. Training needs ~4× the model's weight size (weights + gradients + optimiser state).
- Inference (using a model) is cheap and fits consumer hardware for small-to-mid models; training frontier models needs thousands of GPUs and months of compute.
Practice — Ladder E: does it fit?
Two short exercises turn the arithmetic of this module into code. The first sizes a model in gigabytes; the second decides whether a model fits a given GPU, for both inference and training. Both run in your browser — no setup, no data centre required.
Ask the mentor about this module
Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.
Loading exercise…
Loading exercise…
📚 Go Further
Where the numbers behind model scale come from.
| Type | Resource |
|---|---|
| Video | 3Blue1Brown — How large language models work (YouTube) |
| Reference | Tim Dettmers — Which GPU(s) to Get for Deep Learning (timdettmers.com) |
| Reference | Epoch AI — Trends in AI (epochai.org) — compute, dataset size, and training cost over time |