Guided pathThis is part of Understand Transformers & BERTBack to the path

Build a vocabulary table

P42.from-paper-to-notebook.02 · Audience: guest, language-pro, it-ml · Prerequisites: Tally the words in a text

Real LLM grading for this pageLLM grading (this page):

An interpreter builds a glossary: jot down every specialist term, drop the repeats, put them in order. Done by hand it is fiddly — you cross out the same word twice, lose your alphabetical place, miss one. Two small tools do the whole clean-up for you: a set removes duplicates, and sorted puts what is left in order.

Step 1 / 3 — A set removes the duplicates
ⓘ Concept: set() keeps only the distinct values
Wrapping a list in set(...) throws away the repeats and keeps one of each. To de-duplicate a list of terms and read them back in a fixed order, sort the set:
UNIQUE = sorted(set(terms))
A raw list of seven jottings with three repeats becomes four distinct terms, in A–Z order — in one line.

Why it matters — De-duplicating by hand is exactly where a glossary goes wrong — the same term written twice, slightly differently. A set does it perfectly, every time.

Ask the mentor about this module

Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.

Images, PDF or text. Kept on this device only.
Keeping your files on this device

Off by default. The mentor always gets your file; this only decides whether your own copy stays here. Copies live in this browser only - they do not follow you to another device, and clearing site data removes them.

Ctrl/Cmd + Enter to send

🎓 Practice ladder

1 graded rung · ~7 min

Your turn: de-duplicate and sort the terms, then loop to build a vocabulary table pairing each term with its length. Press Run.

Explorer rung — build a vocabulary table

Loading exercise…

My notes on this module

Loading your notes...

Build a vocabulary table — TransformerLab