0Pricing
Learn AI with Python · Lesson

Custom Gymnasium Environments

gym.Env subclass, observation/action spaces, step/reset/render, registering custom env.

Custom Gymnasium Environments is a free Learn AI with Python lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Learn AI with Python learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

What is Gymnasium?

Gymnasium (the maintained successor to OpenAI Gym) is the standard API for RL environments. Any agent that follows its interface works with any compatible environment, so building your own unlocks RL for custom problems.

pip install gymnasium

import gymnasium as gym

Subclassing gymnasium.Env

A custom environment subclasses gymnasium.Env and implements four things: the action and observation spaces, plus reset and step. That contract is all an agent needs.

class GridWorld(gym.Env):
    def __init__(self):
        super().__init__()

The Action Space

The action_space declares what actions are valid. Discrete(n) means n choices (e.g. up/down/left/right); Box defines continuous ranges for actions like steering.

self.action_space = gym.spaces.Discrete(4)  # 4 moves

The Observation Space

The observation_space describes what the agent sees. A Box space sets the shape and value bounds of the observation vector, so algorithms know the input dimensions.

self.observation_space = gym.spaces.Box(
    low=0, high=10, shape=(2,), dtype=float
)  # agent (x, y) position

The reset Method

reset starts a new episode and returns a tuple (observation, info). It accepts a seed for reproducibility. Always return the initial observation here.

def reset(self, seed=None, options=None):
    super().reset(seed=seed)
    self.pos = [0, 0]
    obs = self._get_obs()
    info = {}
    return obs, info

The step Method

step(action) advances the environment one tick. It returns five values: (obs, reward, terminated, truncated, info). Getting this signature right is essential for compatibility.

def step(self, action):
    self._move(action)
    obs = self._get_obs()
    ...

terminated vs truncated

Two separate flags: terminated means the task ended naturally (goal reached or failed); truncated means it was cut off (e.g. time limit). Splitting them lets algorithms handle each correctly.

    terminated = (self.pos == self.goal)
    truncated = (self.steps >= self.max_steps)
    reward = 1.0 if terminated else -0.01
    return obs, reward, terminated, truncated, {}

Designing the Reward

The reward function defines the goal. Shape it carefully: a small step penalty plus a goal bonus encourages reaching the target quickly. Bad reward design is the most common cause of RL failure.

Registering the Environment

Register your env with an ID so you can create it via gym.make, matching the convention of built-in environments.

gym.register(id="GridWorld-v0", entry_point=GridWorld)
env = gym.make("GridWorld-v0")

Using It with an Agent

Because it follows the standard API, your custom env plugs straight into libraries like Stable-Baselines3, no extra glue required.

from stable_baselines3 import PPO

env = gym.make("GridWorld-v0")
model = PPO("MlpPolicy", env, verbose=1)
model.learn(total_timesteps=50_000)

The Standard Interaction Loop

Every RL loop looks the same: reset to get the first observation, then repeatedly choose an action, call step, and stop when terminated or truncated. This uniformity is Gymnasium's whole point.

obs, info = env.reset()
done = False
while not done:
    action = env.action_space.sample()
    obs, reward, terminated, truncated, info = env.step(action)
    done = terminated or truncated

Quick Check

Test your Gymnasium knowledge.

Recap: Custom Gymnasium Environments

You built a custom env by subclassing gymnasium.Env, defining action_space and observation_space, implementing reset() (returns obs, info) and step() (returns obs, reward, terminated, truncated, info), and designing a reward. Following the standard API lets any RL library train on it.

Frequently asked questions

Is the “Custom Gymnasium Environments” lesson free?

Yes — the full text of “Custom Gymnasium Environments” is free to read here on the web, and the Learn AI with Python course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Learn AI with Python course, upgrade to CoddyKit PRO.

What will I learn in “Custom Gymnasium Environments”?

gym.Env subclass, observation/action spaces, step/reset/render, registering custom env. You practise Learn AI with Python with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start Learn AI with Python?

No prior experience is required. Learn AI with Python on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Custom Gymnasium Environments” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this Learn AI with Python lesson?

Yes. Every Learn AI with Python lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. Policy Gradient Methods: REINFORCE
  2. Actor-Critic Methods (A2C)
  3. Proximal Policy Optimization (PPO)
  4. Custom Gymnasium Environments
← Back to Learn AI with Python