REINFORCEMENT LEARNING · START HERE
Learning starts with a choice
Imagine teaching an explorer to find a treasure. You do not hand it the route. It tries a move, sees what happens, and uses that experience to make its next choice.
That is the starting idea behind reinforcement learning (RL). A learner takes actions, receives rewards, and tries to collect more reward over time. A good choice may help later—even if it does not pay off right now.
Begin with the story. Try an experiment. Then meet the equation that explains what you just saw. You can follow the main idea before learning every symbol.
Try the ideas outside RL first. In Random Dots and Fair Comparisons, use a circle to discover Monte Carlo. Continue with importance sampling or entropy and KL divergence. In Small Steps Down a Hill, compare full gradients with SGD and mini-batches on the same known objective.
Choose a starting point
01 · A SINGLE CHOICE
Which button should I try?
Meet rewards, exploration and uncertainty. Watch new evidence change a belief.
02 · A JOURNEY
Which route should I learn?
Keep the world the same. Change the learning rule. Compare paths, scores and estimates.
03 · THE REASONING
What do the equations say?
Build from averages and probabilities to the Bellman equation, Monte Carlo and TD learning.
The learning path
Each step answers a question the previous idea could not handle.
- Bandits: Which action is worth trying when rewards are uncertain?
- Bayesian beliefs: How should evidence change what I believe?
- MDPs: What changes when my action affects where I go next?
- Returns and values: How do I measure a whole journey, not just one move?
- Bellman: Can I split a long future into one step and what comes after it?
- DP, Monte Carlo and TD: Do I use a model, wait for a complete trip, or learn after one step?
- SARSA and Q-learning: Which next action belongs in my prediction?
Later chapters extend these ideas to large problems: function approximation → DQN; policy gradients → actor–critic → GAE, TRPO and PPO. GRPO belongs to a later branch about language-model post-training. These advanced topics extend the foundation lessons as the series grows.
Every color has a job
Equation symbols keep their meaning across lessons. Text labels explain each role, too.
A method’s path color identifies that method within a comparison. Curve colors distinguish the prior, likelihood and posterior in the Bayesian charts. These chart keys are separate from the equation symbol key.
A small math toolbox, opened when needed
To talk about uncertainty
Probability → conditional probability → Bayes’ rule → prior, likelihood and posterior.
To learn from experience
Random variables → expectation → sampling → sample average → Monte Carlo estimation.
To read the symbols
One number → vector → matrix → tensor. A function is a rule; its stored representation might be a table or a network.
You do not need to finish a separate math course first. Each equation in the foundation lessons has its own symbol dictionary and a small numerical example.
Be First to Comment