Console — run the live BERT pipeline

Everything this pillar teaches — tokenisation, positional encoding, attention, the encoder block — exists so that this can happen: a real BERT model trained from raw text, end to end. The console below drives the actual training engine (tlab_compute.bert_pipeline, PyTorch) through its six steps:

1. Collect — gather the raw text corpus.
2. Clean — normalise it into training sentences.
3. Train tokenizer — learn a BPE vocabulary from the corpus.
4. Build dataset — pack sentences into masked training sequences.
5. Train model — train TinyBERT with the MLM + NSP objectives.
6. Inspect — mask tokens in a held-out pair and check the predictions.

Each step consumes the previous step's artifacts, which live in your session on the server — one live session per account, evicted after idle time. The log pane streams every line the engine prints, exactly as you would see running it in a terminal.

The Tiny preset (bundled offline corpus, vocab 512, 2 layers, 1 epoch) runs the whole pipeline in seconds on CPU — small enough to re-run freely, real enough that every artifact is genuine. The Reference preset (~7 M parameters, a 300–700K-word corpus fetched from the network) is a command-line workload and is not served over the web.

Checking access…

Where to go next

Once step 5 has produced a checkpoint, the Inspect module lets you probe it with your own sentences, watch a fresh model learn iteration by iteration, and trace a single training step end to end.