Learning from Reward

P9.rl-foundations.01 · Audience: guest, it-ml, language-pro

Uplift & Heterogeneous Effects Bandits & Explore/Exploit

Every pillar of the Machine Learning domain so far has quietly assumed that the data is already there — a pile of examples someone collected, waiting to be fitted or evaluated. This pillar drops that assumption. Here there is no dataset handed to you: an agent must act to generate its own experience, and the only feedback it gets is a reward — a number saying "that was good" or "that was bad", never the correct answer spelled out. Learning to walk, to play, to hold a conversation: none of these come with a labelled answer key. They come with consequences.

Step 1 / 5 — A third way to learn

Machine learning has three broad families, and they differ by the kind of feedback the learner gets. Two of them you have already met.

Paradigm	What it is given	What it learns
Supervised	Examples, each with the correct answer (a label)	A mapping from input to that answer
Unsupervised	Examples with no answers at all	Structure — clusters, patterns, compression
Reinforcement	No examples — only rewards for the actions it tries	A way of behaving that earns the most reward

The jump to the third row is bigger than it looks. Supervised learning has a teacher who, for every question, writes the right answer in the margin. Reinforcement learning has no such teacher — only a world that reacts. You try something, the world responds with a good or bad outcome, and you are left to work out for yourself what you should have done. Nobody ever tells you the right move; you only ever find out how well the move you made turned out.

ⓘ Concept: Reinforcement learning is learning by acting

supervised: given (input, correct answer) · reinforcement: given (situation → your action → a reward)

Why it matters — In supervised learning the data is fixed and given; the learner is a passive fitter. In reinforcement learning the learner's own choices decide what data it ever sees — act timidly and you never discover the good outcomes that live behind the doors you never opened. This is why 'the data is given' finally breaks: the agent is not fitting a dataset, it is generating one, one action at a time, and its behaviour and its data shape each other.

✅ Key takeaways

This pillar drops the assumption that the data is given. A reinforcement-learning agent generates its own experience by acting, and its only feedback is a reward — a score, never the correct answer.
Reinforcement is the third learning paradigm alongside supervised (learn from labelled examples) and unsupervised (find structure with no answers). It learns a way of behaving that earns the most reward.
Every RL story is the same loop: the agent observes a state, takes an action, and the environment returns a reward and a next state — round and round. A dog earning treats and a child learning to ride a bike are both this loop.
A reward is evaluative, not instructive: it scores the action you took without revealing the better action you did not. It can also be delayed (the blunder is punished many moves later) and sparse (silent for long stretches) — which is what makes RL genuinely hard.
Because no one names the best action, every agent must trade off exploring untested actions against exploiting the best known one — a tension with no equivalent in supervised learning, and the subject of the next module.

Next: P9.rl-foundations.02 — Bandits & Explore/Exploit

Ask the mentor about this module

Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.

Ctrl/Cmd + Enter to send

Try it yourself

A scratch console for this page's ideas — ungraded, nothing you run here is recorded.

Scratch console

A scratch console with the scientific stack (pandas, numpy, scikit-learn). Runs on the server — no network, resource-limited and measured.

Output appears here.

Where next?

Continue

Bandits & Explore/Exploit →

RL Foundations · P9

Later in RL Foundations

Go up a level