Account

P10 · Generative AI

language modelling → Transformer (reference tutorial) → fine-tuning, evaluation

Generative AI built from first principles: tokens, embeddings, attention and the full Transformer assembled step by step, culminating in a live BERT pipeline you can probe sentence by sentence.

The end-to-end picture first: every module in this pillar is one stage of the same journey — raw sentence in, meaning-bearing vectors out. The diagram below is the map; the corpus editor under it sets the sentences every lab computes on.

Raw Textstringstext-representationTokenizetoken IDs + masktext-representationEmbedseq × d_modelvectorspositional-encodingPositionalEncodeseq × d_model+ positionattentionAttentionseq × d_modelcontextualattentionMulti-HeadAttentionseq × d_modelmulti-contexttransformer-architectureEncoderBlockseq × d_modelencoder outputtransformer-architecturePool(d_model,)sentence vector
End-to-end: raw text becomes a sentence vector in eight stages. The Atlas module walks the inside of each one; Console & Inspect run the live BERT pipeline behind them.

Orientation — concepts before the maths

Text Representation

Positional Encoding

Attention

Transformer Architecture

LLM Essentials

LLM Language Practice

LLM Systems & Frontier

Pipeline console & Atlas (live BERT)