AIM-05-001-P · Primer — What Is Reinforcement Learning

The idea in one sentence

Reinforcement learning is learning to make good decisions by trial and error, guided by rewards.

Why it exists

Think of a child learning to ride a bicycle: the parent doesn’t tell the child exactly how to balance each second, but they praise the child when they stay upright for a few meters. The child receives a burst of encouragement only after a successful ride, not for every tiny pedal push. In the same way, reinforcement learning tackles problems where we cannot label every correct action, only signal success after a sequence—like teaching a robot to navigate a warehouse where the only feedback is “you reached the goal” versus “you got stuck”. The core need is to automate decision‑making in environments where feedback is delayed and sparse.

How it works

An agent starts in an initial state of the environment, observes its features, and selects an action. The environment then returns a reward—perhaps +1 for winning a game—and transitions to a new state. The agent repeats this loop many times, accumulating rewards over a trajectory. By comparing the total reward of different action sequences, it updates its policy to favor actions that lead to higher long‑term returns.

Worked example: Imagine a 3‑step grid world where the agent starts at cell A, can move right (R) or down (D), and receives a reward of +10 only when it reaches the goal cell G at the bottom‑right corner. All other moves give 0. The optimal path is A→R→R→D→D (5 moves). Suppose the agent initially picks actions at random and experiences the following episode:

A → R (reward 0, now at B)
B → D (reward 0, now at E)
E → R (reward 0, now at F)
F → D (reward 0, now at G) → receives +10. The total return for this episode is 10. Using Q‑learning, the agent updates the Q‑value for each state‑action pair visited, e.g., Q(A,R) ← 0 + α[10 + γ·0 − 0]. After many such episodes, the Q‑values for the optimal moves (R from A, R from B, D from C, etc.) become higher than the sub‑optimal ones, and the policy converges to always choose the path that yields the +10 reward.

A major variation is Q‑learning, which estimates the value of each state‑action pair instead of directly learning a policy. Another related framework is policy gradient methods, which adjust the policy parameters directly based on reward feedback.

See it — the agent-environment loop

flowchart LR
  A["Agent"] -->|action| B["Environment"]
  B -->|state| A
  B -->|reward| A

The one gotcha

Beginners often mistake the visible, immediate score for the true learning signal. Reinforcement learning optimizes cumulative future reward, so an agent may deliberately take a short‑term loss (e.g., a -1 penalty) to reach a state that yields a +20 payoff later. Ignoring this long‑term perspective leads to policies that chase instant points but fail to achieve the overall objective.

The one-line takeaway

Reinforcement learning teaches machines to act wisely by learning from the consequences of their own actions.

Where you'll meet it

Practitioners encounter reinforcement learning in game AI, robotics, and recommendation systems, often using libraries like OpenAI Gym, TensorFlow Agents, or RLlib. In production, engineers monitor reward curves and policy performance to ensure the agent is truly improving over time.