Q-Learning

Model-free reinforcement learning algorithm that learns optimal action-value functions.

intermediate reinforcement learning reinforcement-learningcontrol
bellman-equationtemporal-differenceexplorationoptimisation

Author @Praneeth-Suresh Updated 2026-06-24

Overview

Q-Learning iteratively estimates the optimal action-value function via the Bellman update:

Q(s,a)←Q(s,a)+α[r+γmax⁡a′Q(s′,a′)−Q(s,a)]Q(s, a) \leftarrow Q(s, a) + \alpha\left[r + \gamma \max_{a'} Q(s', a') - Q(s, a)\right]

Exploration vs Exploitation

ϵ\epsilon-greedy balances exploration with exploitation: take a random action with probability ϵ\epsilon, otherwise act greedily.

Q-Learning is optimised using ideas from Gradient Descent and extended by deep Q-networks built on Neural Networks.

Learning Path

What should I learn before Q-Learning?

Start with gradient-descent before studying Q-Learning.

How does Q-Learning connect to other AI topics?

Q-Learning belongs to Reinforcement Learning and connects to reinforcement-learning, control, bellman-equation, temporal-difference, exploration, optimisation.

Further Reading

  1. Reinforcement Learning: An Introduction – Sutton & Barto