Implementing Q-Table in Python
A simple example of Q-learning.
Implementing Q-Table in Python is a free Python Academy lesson on CoddyKit. This is lesson 3 of 5. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Python Academy learning path, and your progress syncs across the web and the CoddyKit app. The Python Academy course includes 5 lessons in total.
Implementing Q-Learning in Python
In this lesson, we will implement a simple Q-Learning algorithm in Python. The agent will learn to navigate a grid world to maximize its rewards.
Defining the Environment
We define a simple 4×4 grid world where the agent starts at the top-left corner and must reach the bottom-right corner to receive a reward.
import numpy as np
# Define the environment
num_states = 16 # 4x4 grid
num_actions = 4 # Up, Down, Left, Right
# Define rewards (-1 for each step, +10 for the goal)
rewards = np.zeros(num_states)
rewards[-1] = 10Initializing the Q-Table
The Q-Table is initialized with zeros for all state-action pairs.
# Initialize the Q-Table
q_table = np.zeros((num_states, num_actions))
print("Initial Q-Table:")
print(q_table)Implementing the Q-Learning Algorithm
We use the following parameters:
- α (Learning Rate): Controls how much new information overrides old.
- γ (Discount Factor): Weights future rewards relative to immediate rewards.
- ε (Exploration Rate): Balances exploration and exploitation.
# Parameters
alpha = 0.1 # Learning rate
gamma = 0.9 # Discount factor
epsilon = 0.1 # Exploration rate
# Q-Learning loop
num_episodes = 1000
for episode in range(num_episodes):
state = 0 # Start at the first state
done = False
while not done:
# Exploration or exploitation
if np.random.rand() < epsilon:
action = np.random.randint(num_actions) # Explore
else:
action = np.argmax(q_table[state]) # Exploit
# Transition to the next state
next_state = (state + action) % num_states # Simplified transition logic
reward = rewards[next_state]
# Update Q-value
q_table[state, action] = q_table[state, action] + alpha * (
reward + gamma * np.max(q_table[next_state]) - q_table[state, action]
)
# Move to the next state
state = next_state
done = state == num_states - 1Evaluating the Learned Policy
After training, we evaluate the policy by following the optimal actions stored in the Q-Table:
# Evaluate the policy
state = 0
optimal_path = [state]
while state != num_states - 1:
action = np.argmax(q_table[state])
state = (state + action) % num_states # Simplified transition logic
optimal_path.append(state)
print("Optimal Path:", optimal_path)Visualizing the Q-Table
We can visualize the Q-Table to understand the learned values for each state-action pair:
import matplotlib.pyplot as plt
import seaborn as sns
# Visualize Q-Table
plt.figure(figsize=(10, 8))
sns.heatmap(q_table, annot=True, fmt=".2f", cmap="coolwarm")
plt.title("Q-Table Heatmap")
plt.xlabel("Actions")
plt.ylabel("States")
plt.show()Advantages of Q-Learning
Q-Learning offers several benefits:
- Simple and easy to implement.
- Can handle stochastic environments.
- Converges to the optimal policy with sufficient exploration.
Limitations of Q-Learning
Q-Learning has some limitations:
- Does not scale well to environments with large state-action spaces.
- Learning can be slow in complex environments.
- Requires a well-defined state-action representation.
Summary and Next Steps
In this lesson, we:
- Implemented a simple Q-Learning algorithm in Python.
- Trained an agent to navigate a grid world using a Q-Table.
- Evaluated the learned policy and visualized the Q-Table.
Next, we will explore Deep Q-Learning, which uses neural networks to handle environments with large state-action spaces.
Learn Python with an AI tutor — free
Write and run real code in your browser, get instant help from a 24/7 AI tutor, and pick up where you left off on the web or in the app.
- Courses
- 76
- Lessons
- 320
Frequently Asked Questions
Is the “Implementing Q-Table in Python” lesson free?
Yes — the full text of “Implementing Q-Table in Python” is free to read here on the web. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Python Academy course, upgrade to CoddyKit PRO. The Python Academy course includes 5 lessons in total.
What will I learn in “Implementing Q-Table in Python”?
A simple example of Q-learning. You practise Python Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Python Academy?
No prior experience is required. Python Academy on CoddyKit is structured for beginners through advanced learners, so you can start here or from the beginning and move at your own pace. This is lesson 3 of 5.
How long does the “Implementing Q-Table in Python” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Python Academy lesson?
Yes. Every Python Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Basic Concepts in Reinforcement Learning
- Q-Table Concept
- Implementing Q-Table in Python
- Deep Q-Learning
- Exploring OpenAI Gym