0Pricing
AI Agents · Lesson

World Models and Predictive Planning

Agents that simulate future states before acting: mental simulation for AI.

World Models and Predictive Planning is a free AI Agents lesson on CoddyKit — lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

What Is a World Model?

A world model is the agent's internal representation of the environment: what objects exist, what their states are, what rules govern how they change, and what the agent can do to affect them. It is the agent's mental simulation of the world.

Without a world model, the agent can only react to observations. With one, it can predict and plan.

World State Representation

A world state is a snapshot of all relevant facts about the environment at a moment in time. For an IoT agent, this might be current sensor readings and device states. For a software agent, it might be the state of files, APIs, and databases it manages.

from dataclasses import dataclass, field
from datetime import datetime
from typing import Any

@dataclass
class WorldState:
    timestamp: str = ''
    entities: dict = field(default_factory=dict)  # entity_id -> attributes
    relationships: list = field(default_factory=list)
    agent_position: str = 'idle'
    pending_actions: list = field(default_factory=list)

    def __post_init__(self):
        if not self.timestamp:
            self.timestamp = datetime.utcnow().isoformat()

    def update_entity(self, entity_id: str, attributes: dict):
        current = self.entities.get(entity_id, {})
        self.entities[entity_id] = {**current, **attributes}

    def snapshot(self) -> dict:
        return {
            'timestamp': self.timestamp,
            'entities': dict(self.entities),
            'agent_position': self.agent_position
        }

# Example:
world = WorldState()
world.update_entity('server_room', {'temp_c': 22.5, 'humidity': 55})
world.update_entity('hvac_unit_1', {'status': 'off', 'setpoint': 22})
print(world.snapshot())

Transition Model

A transition model defines how the world state changes in response to an action: next_state = transition(current_state, action). It lets the agent predict what the world will look like after taking an action without actually taking it.

import copy

def transition(state: WorldState, action: dict) -> WorldState:
    """
    Predict next state given current state and action.
    Returns a new WorldState (does not modify original).
    """
    next_state = copy.deepcopy(state)

    action_type = action.get('type')
    target = action.get('target')
    params = action.get('params', {})

    if action_type == 'TURN_ON_HVAC':
        next_state.update_entity(target, {
            'status': 'on',
            'setpoint': params.get('setpoint', 22)
        })
        # Simulate temperature effect (simplified linear model)
        room = action.get('room', 'server_room')
        current_temp = next_state.entities.get(room, {}).get('temp_c', 25)
        next_state.update_entity(room, {
            'temp_c': max(current_temp - 2, params.get('setpoint', 22))
        })

    elif action_type == 'SEND_ALERT':
        next_state.pending_actions.append({'alert_sent': True})

    return next_state

# Predict effect of turning on HVAC:
action = {'type': 'TURN_ON_HVAC', 'target': 'hvac_unit_1',
          'room': 'server_room', 'params': {'setpoint': 20}}
next_world = transition(world, action)
print('After action:', next_world.entities.get('server_room'))

Lookahead Planning

Lookahead planning simulates N steps into the future before committing to an action. The agent explores multiple action sequences, evaluates each resulting state, and chooses the sequence that leads to the best predicted outcome.

def evaluate_state(state: WorldState) -> float:
    """
    Compute a scalar utility score for a world state.
    Higher = better for the agent's goals.
    """
    score = 1.0
    server_room = state.entities.get('server_room', {})
    temp = server_room.get('temp_c', 25)
    # Penalty for high temperature
    if temp > 30:
        score -= (temp - 30) * 0.2
    # Penalty for very low temperature (over-cooling)
    if temp < 18:
        score -= (18 - temp) * 0.1
    return max(0.0, score)

def lookahead(state: WorldState, possible_actions: list, depth: int = 3) -> dict:
    """Evaluate top-level actions by simulating depth steps ahead."""
    best_action = None
    best_score = -float('inf')

    for action in possible_actions:
        simulated = transition(state, action)
        # Simulate depth more steps (simplified: no branching)
        for _ in range(depth - 1):
            simulated = transition(simulated, {'type': 'NOOP'})
        score = evaluate_state(simulated)
        if score > best_score:
            best_score = score
            best_action = action

    return {'best_action': best_action, 'expected_score': round(best_score, 3)}

LLM as World Model

For complex, hard-to-formalise environments, the LLM itself can serve as the world model. Ask it to predict what will happen if the agent takes a specific action. This trades precision for generality — the LLM can reason about arbitrary domains.

import anthropic
import json

client = anthropic.Anthropic(api_key='YOUR_API_KEY')

def llm_predict(
    current_state: dict,
    proposed_action: dict,
    domain_description: str = ''
) -> dict:
    prompt = (
        f'Domain: {domain_description}\n\n'
        f'Current state: {json.dumps(current_state, indent=2)}\n\n'
        f'Proposed action: {json.dumps(proposed_action, indent=2)}\n\n'
        'Predict the next world state after this action. '
        'Consider direct effects and second-order effects.\n'
        'Return JSON: {"predicted_state": dict, '
        '"confidence": float, "side_effects": [str]}'
    )
    response = client.messages.create(
        model='claude-opus-4-5', max_tokens=512,
        messages=[{'role': 'user', 'content': prompt}]
    )
    return json.loads(response.content[0].text)

Model-Based vs Model-Free Planning

Model-based: agent has an explicit world model, simulates future states, and plans. More sample-efficient, generalises better. Model-free: agent learns action-value pairs from experience without a model. Simpler but requires more interactions to learn a policy.

# Comparison table
COMPARISON = {
    'model_based': {
        'pros': [
            'Can plan before acting (lookahead)',
            'Works well with limited data',
            'Interpretable — you can inspect the world model',
            'Can simulate counterfactuals'
        ],
        'cons': [
            'Model can be wrong (model mismatch)',
            'Complex to build for arbitrary domains',
            'Planning is computationally expensive'
        ],
        'examples': 'AlphaGo, MCTS, agent with explicit WorldState'
    },
    'model_free': {
        'pros': [
            'No need to model the world explicitly',
            'Learns from raw experience',
            'Simpler architecture'
        ],
        'cons': [
            'Requires many interactions to learn',
            'Poor generalisation to new scenarios',
            'Cannot reason about unseen situations'
        ],
        'examples': 'Q-learning, policy gradient, basic RLHF'
    }
}

for approach, info in COMPARISON.items():
    print(f'{approach}: {info["examples"]}')

Uncertainty in World Models

World models are approximations. Account for uncertainty by maintaining a probability distribution over possible states rather than a single point estimate. When uncertainty is high, take conservative actions or gather more information before acting.

from statistics import mean, stdev

@dataclass
class UncertainState:
    entity_id: str
    attribute: str
    samples: list  # multiple estimates

    def mean(self) -> float:
        return mean(self.samples) if self.samples else 0.0

    def uncertainty(self) -> float:
        if len(self.samples) < 2:
            return float('inf')
        return stdev(self.samples)

    def should_gather_more_data(self, threshold: float = 2.0) -> bool:
        return self.uncertainty() > threshold

# Example: uncertain temperature estimate from multiple sensors
temp_state = UncertainState(
    entity_id='server_room',
    attribute='temp_c',
    samples=[22.1, 22.8, 21.9, 34.5, 22.3]  # one outlier
)
print(f'Mean temp: {temp_state.mean():.1f}C')
print(f'Uncertainty (stdev): {temp_state.uncertainty():.2f}')
print(f'Gather more data: {temp_state.should_gather_more_data()}')

Updating the World Model

After each action, the agent observes the actual outcome and updates its world model. If the prediction was wrong, the model is updated to be more accurate. This is the observe-update-plan cycle that drives model-based agent learning.

class WorldModelAgent:
    def __init__(self):
        self.world = WorldState()
        self.prediction_errors = []

    def act(self, action: dict) -> dict:
        # 1. Predict next state
        predicted = transition(self.world, action)

        # 2. Take the action in the real world
        actual_observation = self._execute_action(action)

        # 3. Measure prediction error
        for entity_id, attrs in actual_observation.items():
            for attr, actual_val in attrs.items():
                predicted_val = predicted.entities.get(entity_id, {}).get(attr)
                if predicted_val is not None:
                    error = abs(actual_val - predicted_val)
                    self.prediction_errors.append(error)

        # 4. Update world model with actual observation
        for entity_id, attrs in actual_observation.items():
            self.world.update_entity(entity_id, attrs)

        return actual_observation

    def _execute_action(self, action: dict) -> dict:
        # Placeholder — in production this calls real APIs/sensors
        return {'server_room': {'temp_c': 23.0}}

    def model_accuracy(self) -> float:
        if not self.prediction_errors:
            return 1.0
        return max(0.0, 1.0 - mean(self.prediction_errors) / 10)

Planning with the LLM: Simulate 3 Steps

For natural language domains, ask the LLM to simulate 3 steps of a proposed plan before executing it. The LLM evaluates each step's predicted outcome and flags potential problems — preventing the agent from taking actions with bad predicted consequences.

import anthropic
import json

client = anthropic.Anthropic(api_key='YOUR_API_KEY')

def simulate_plan_steps(goal: str, proposed_steps: list) -> dict:
    steps_str = json.dumps(proposed_steps, indent=2)
    prompt = (
        f'Goal: {goal}\n\n'
        f'Proposed plan steps:\n{steps_str}\n\n'
        'Simulate executing each step in order. For each step predict:\n'
        '- What changes in the world?\n'
        '- What could go wrong?\n'
        '- Is this step reversible?\n\n'
        'Return JSON: {"step_simulations": [{"step": int, '
        '"predicted_outcome": str, "risks": [str], "reversible": bool}], '
        '"plan_safe": bool, "recommendation": str}'
    )
    response = client.messages.create(
        model='claude-opus-4-5', max_tokens=1024,
        messages=[{'role': 'user', 'content': prompt}]
    )
    result = json.loads(response.content[0].text)
    if not result['plan_safe']:
        print(f'Plan safety concern: {result["recommendation"]}')
    return result

When to Trust the World Model

A world model is only as good as the accuracy of its transition function and initial state. In rapidly changing environments, the model becomes stale quickly. Implement a staleness threshold: if the model has not been updated in N seconds and an action is high-stakes, re-observe the environment first.

from datetime import datetime, timedelta

class StaleAwareWorldModel:
    def __init__(self, max_age_seconds: int = 60):
        self.state = WorldState()
        self.max_age = timedelta(seconds=max_age_seconds)
        self.last_observation_time = datetime.utcnow()

    def is_stale(self) -> bool:
        age = datetime.utcnow() - self.last_observation_time
        return age > self.max_age

    def refresh(self, observation_fn):
        new_observations = observation_fn()
        for entity_id, attrs in new_observations.items():
            self.state.update_entity(entity_id, attrs)
        self.last_observation_time = datetime.utcnow()
        print(f'World model refreshed')

    def plan_action(self, action: dict, observation_fn) -> dict:
        if self.is_stale():
            print('World model stale — refreshing before planning')
            self.refresh(observation_fn)
        return lookahead(self.state, [action], depth=3)

Counterfactual Reasoning

A world model enables counterfactual reasoning: 'What would have happened if I had taken action A instead of action B?' This is powerful for learning from mistakes without repeating them — the agent can simulate the counterfactual to understand why its choice led to a suboptimal outcome.

import anthropic
import json

client = anthropic.Anthropic(api_key='YOUR_API_KEY')

def counterfactual_analysis(
    actual_state_before: dict,
    action_taken: dict,
    actual_outcome: dict,
    alternative_action: dict
) -> dict:
    prompt = (
        'Analyse a counterfactual scenario:\n\n'
        f'Initial state: {json.dumps(actual_state_before, indent=2)}\n'
        f'Action taken: {json.dumps(action_taken, indent=2)}\n'
        f'Actual outcome: {json.dumps(actual_outcome, indent=2)}\n\n'
        f'Alternative action considered: {json.dumps(alternative_action, indent=2)}\n\n'
        'What would likely have happened with the alternative action?\n'
        'Return JSON: {"counterfactual_outcome": str, '
        '"better_choice": str, "lesson": str}'
    )
    response = client.messages.create(
        model='claude-opus-4-5', max_tokens=512,
        messages=[{'role': 'user', 'content': prompt}]
    )
    return json.loads(response.content[0].text)

Knowledge Check

What is the main advantage of a model-based agent over a model-free agent?

Recap: World Models and Predictive Planning

Great work! Key takeaways:

  • World state: snapshot of all relevant entity attributes at a moment in time
  • Transition model: next_state = transition(state, action) — predicts effects
  • Lookahead: simulate N steps ahead, evaluate outcomes, pick best action
  • LLM as world model: works for hard-to-formalise domains
  • Uncertainty: maintain distributions, not point estimates; avoid action under high uncertainty
  • Staleness: re-observe before high-stakes actions if model is old

Next: alignment challenges — why building truly beneficial autonomous agents is hard.

Frequently asked questions

Is the “World Models and Predictive Planning” lesson free?

Yes — the full text of “World Models and Predictive Planning” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “World Models and Predictive Planning”?

Agents that simulate future states before acting: mental simulation for AI. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 2 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “World Models and Predictive Planning” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. From Assistant to Autonomous Agent
  2. World Models and Predictive Planning
  3. Alignment Challenges in Autonomous Agents
  4. Research Frontiers: AGI and Beyond
← Back to AI Agents