-
Deep Q-Networks
DQN is the algorithm that taught a single neural network to beat human players across 49 Atari games from raw pixels. This is the preface: what Q-learning is, why a network breaks it, and the three tricks that put it back together.
-
MoB: Mixture of Bandits for Sequential Exploring Problems with Stateless Agents and Limited Input
A stateless agent with a noisy local sensor is a blind, deaf, one-legged minesweeper — fine when the world is uniform, systematically wrong when the world is clustered. The fix is not a second expert, but a learned gate that knows when to disable the planner's long leap.
-
The Sequential Recommender: Fitted Q-Iteration
The moment an action changes what the next decision can see or do, the bandit is myopic and the problem becomes sequential reinforcement learning.
-
Building the Bandit Recommender
The practical companion to the bandits essay. We state the recommendation problem the way a product team would, use the fishing simulator as a running analogy, dig into the hashing trick that makes the engine streamable, and compare ε-greedy against LinUCB on a real workload. With a brief survey of the rest of the bandit family.
-
Contextual Bandits and Their Ethical Use Cases
A contextual bandit is a machine-learning algorithm that makes billions of dollars a year and is roughly one-tenth as hyped as GPT. The TikTok doomscroll, the ad that follows you around for a week, the song Spotify queues next, Addictive Games — all of these are bandit-family algorithms.
-
The Essence of Context Bandits
What a contextual bandit actually optimises, in one diagram and a handful of equations. With live math and a flow of the explore-exploit loop.
-
Primer to Gradient Descent
Gradient descent is the most used algorithm in machine learning, and the idea behind it is almost embarrassingly simple. This is the preface: without it, you won't understand 90% of ML.
-
Quartiles, Ridge, and Lasso
Three fundamentals that look unrelated but tell one story: how we describe data, and how we keep a model from believing it too much.