LLM economics & agentic design
P11.applied-llm-systems.05 · Audience: guest, it-ml, language-pro · Prerequisites: Adapting LLMs
Every LLM call is billed in tokens, and the token is the unit of latency
as much as cost. But not every token costs the same: the same word can be cheap
or dear depending on whether the model processed it fresh, reused it from cache,
or generated it. This module is the economics behind the agents you built in
P11.applied-llm-systems.01 and adapted in P11.applied-llm-systems.02 — what
you actually pay for, and the three design levers that decide the bill: more
tools, more skills, and the right model for each task.
A token is the billing atom — roughly a word-piece. Every token in a request or response is counted, and each one falls into exactly one price class (call it a bucket) with its own per-token rate. Your bill is simply the sum over buckets of tokens × that bucket's price.
There are four buckets because the same token can cost wildly different amounts depending on what the provider had to do with it — process it fresh, store it, reuse a stored copy, or generate it.
🎯 The token is the unit of cost and latency; the four buckets are just the four things a provider can do with a token.
⚡ Interview Ref — the quick-scan Reference face
- Four token buckets: uncached input (1×) · cache-write (~1.25×/2×) · cache-read (~0.1×) · output (~5×). Reasoning tokens bill as output; output dominates the bill.
- Caching = prefix match: stable content first, volatile last; one changed byte invalidates the rest. Cache read ~10× cheaper than fresh input. This is context construction, not prompt engineering.
- Effort lever: thinking tokens are output; match depth to task difficulty, not minimise — more planning can lower total cost on hard agentic tasks.
- Tools vs skills: tools = capability (context + selection-accuracy cost; use tool-search; promote bash → typed tool to gate/render/audit/parallelise) · skills = deterministic procedure via progressive disclosure. Skill for inconsistency; tool for a missing capability.
- Model selection: tier per task; escalate, don't default high; cheap sub-agents for fan-out; cache is model-scoped (spawn a sub-agent, don't swap mid-loop).
- Build an agent only if: complex · valuable · viable · errors recoverable. Otherwise a single call or a code-orchestrated workflow.
Ask the mentor about this module
Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.