Adapting LLMs
P11.applied-llm-systems.02 · Audience: guest, it-ml, language-pro · Prerequisites: Agentic systems, tools & MCP
To make a general LLM good at a domain task you can pull three levers, in order
of escalating commitment: prompt, fine-tune, distill. Prompting
changes the input, fine-tuning changes the weights, distillation changes which
model serves. (Cross-link: P10.llm-systems-frontier.04 — alignment &
adaptation.) Adapting the model is one lever; which model to run per task, and
what each token costs, is covered in P11.applied-llm-systems.05.
| Lever | Changes | Data needed | Best when |
|---|---|---|---|
| Prompt | the input only (weights frozen) | a handful of examples | ship fast, little data, task fluid |
| Fine-tune | the weights (smaller model) | 100s–1000s of pairs | reliable format, shorter prompts, small-beats-large |
| Distill | which model serves | the teacher labels for you | hit quality with a costly model, must serve cheap/fast |
A fine-tuned small model can beat a prompted large one on a narrow task, while being cheaper to serve.
⚡ Interview Ref — the quick-scan Reference face
- Three levers: prompt (input, frozen weights, few examples) · fine-tune (weights, 100s–1000s pairs, reliable format / small-beats-large) · distill (which model serves; teacher labels; cheap/fast serving).
- Mental model: prompting changes the input, fine-tuning changes the weights, distillation changes which model serves.
- Decision factors: labelled-data volume · need for reliable format · latency/cost constraint · existing teacher.
- Distillation: teacher labels → student mimics → cheap fast serving at scale.
- MoE: disambiguate sparse-expert layer vs model-router. VLMs: image-side tasks; grounded eval; images = sensitive data.
Ask the mentor about this module
Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.
Try it yourself
A scratch console for this page's ideas — ungraded, nothing you run here is recorded.
Scratch console
A scratch console with the scientific stack (pandas, numpy, scikit-learn). Runs on the server — no network, resource-limited and measured.
Output appears here.
Where next?
Later in Applied LLM Systems & Agents
This module unlocks