Implementing Q-Table in Python
A simple example of Q-learning.
Implementing Q-Table in Python is a free Learn AI with Python lesson on CoddyKit — lesson 3 of 5. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Learn AI with Python learning path, one of 5 lessons in the course, and your progress syncs across the web and the CoddyKit app.
1
Implementing Q-Learning in Python
In this lesson, we will implement a simple Q-Learning algorithm in Python. The agent will learn to navigate a grid world to maximize its rewards.
2
Defining the Environment
We define a simple 4×4 grid world where the agent starts at the top-left corner and must reach the bottom-right corner to receive a reward.
import numpy as np
# Define the environment
num_states = 16 # 4x4 grid
num_actions = 4 # Up, Down, Left, Right
# Define rewards (-1 for each step, +10 for the goal)
rewards = np.zeros(num_states)
rewards[-1] = 103
Initializing the Q-Table
The Q-Table is initialized with zeros for all state-action pairs.
# Initialize the Q-Table
q_table = np.zeros((num_states, num_actions))
print("Initial Q-Table:")
print(q_table)4
Implementing the Q-Learning Algorithm
We use the following parameters:
- α (Learning Rate): Controls how much new information overrides old.
- γ (Discount Factor): Weights future rewards relative to immediate rewards.
- ε (Exploration Rate): Balances exploration and exploitation.
# Parameters
alpha = 0.1 # Learning rate
gamma = 0.9 # Discount factor
epsilon = 0.1 # Exploration rate
# Q-Learning loop
num_episodes = 1000
for episode in range(num_episodes):
state = 0 # Start at the first state
done = False
while not done:
# Exploration or exploitation
if np.random.rand() < epsilon:
action = np.random.randint(num_actions) # Explore
else:
action = np.argmax(q_table[state]) # Exploit
# Transition to the next state
next_state = (state + action) % num_states # Simplified transition logic
reward = rewards[next_state]
# Update Q-value
q_table[state, action] = q_table[state, action] + alpha * (
reward + gamma * np.max(q_table[next_state]) - q_table[state, action]
)
# Move to the next state
state = next_state
done = state == num_states - 15
Evaluating the Learned Policy
After training, we evaluate the policy by following the optimal actions stored in the Q-Table:
# Evaluate the policy
state = 0
optimal_path = [state]
while state != num_states - 1:
action = np.argmax(q_table[state])
state = (state + action) % num_states # Simplified transition logic
optimal_path.append(state)
print("Optimal Path:", optimal_path)6
Visualizing the Q-Table
We can visualize the Q-Table to understand the learned values for each state-action pair:
import matplotlib.pyplot as plt
import seaborn as sns
# Visualize Q-Table
plt.figure(figsize=(10, 8))
sns.heatmap(q_table, annot=True, fmt=".2f", cmap="coolwarm")
plt.title("Q-Table Heatmap")
plt.xlabel("Actions")
plt.ylabel("States")
plt.show()7
Advantages of Q-Learning
Q-Learning offers several benefits:
- Simple and easy to implement.
- Can handle stochastic environments.
- Converges to the optimal policy with sufficient exploration.
8
9
Limitations of Q-Learning
Q-Learning has some limitations:
- Does not scale well to environments with large state-action spaces.
- Learning can be slow in complex environments.
- Requires a well-defined state-action representation.
10
Summary and Next Steps
In this lesson, we:
- Implemented a simple Q-Learning algorithm in Python.
- Trained an agent to navigate a grid world using a Q-Table.
- Evaluated the learned policy and visualized the Q-Table.
Next, we will explore Deep Q-Learning, which uses neural networks to handle environments with large state-action spaces.
Frequently asked questions
Is the “Implementing Q-Table in Python” lesson free?
Yes — the full text of “Implementing Q-Table in Python” is free to read here on the web, and the Learn AI with Python course includes 5 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Learn AI with Python course, upgrade to CoddyKit PRO.
What will I learn in “Implementing Q-Table in Python”?
A simple example of Q-learning. You practise Learn AI with Python with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Learn AI with Python?
No prior experience is required. Learn AI with Python on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 5, so you can start here or from the beginning and move at your own pace.
How long does the “Implementing Q-Table in Python” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Learn AI with Python lesson?
Yes. Every Learn AI with Python lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Basic Concepts in Reinforcement Learning
- Q-Table Concept
- Implementing Q-Table in Python
- Deep Q-Learning
- Exploring OpenAI Gym