LLM Systems & Frontier

When you ask a chatbot a question, a datacentre GPU that costs as much as a car starts generating your answer one token at a time — and for every single token, it must consult billions of stored weights. Multiply one answer by hundreds of millions of users, and the puzzle writes itself: how is any of this affordable? The honest answer is that at list price it would not be — the whole discipline of LLM serving is the stack of tricks that closes the gap, and it opens this track.

Serving comes first because generation is one-token-at-a-time: latency and cost rule everything, and memory movement — not raw arithmetic — is usually the real bottleneck. The opening module works through the actual levers with small worked calculations: the KV cache and its variants, continuous batching with paged attention, quantisation, speculative decoding — each one a way to move fewer bytes per token or keep the GPU busy despite them. Then the other direction: what training at scale takes (data pipelines, GPU clusters, the parallelism strategies that split one model across thousands of chips) and the architectural bets the frontier labs are making — mixture-of-experts, long context, multimodality — each of which is, at bottom, a different answer to the same cost equation.

The closing modules connect research to operations: alignment and adaptation at scale (RLHF, LoRA and friends — how a finished model is steered without retraining it from scratch), then production LLM systems — routing, guardrails and evaluation for applications that must not fail in public.

This is an IT/ML sequence: expect numbers, trade-offs and back-of-envelope engineering throughout. Language professionals have their own applied route in LLM Language Practice; here, the audience is the person who will be asked "can we afford to run this, and can we trust it in production?" — and who wants to answer with arithmetic rather than a vendor's slide.