AI Agents · Lesson

Backtesting Agent Decisions

Simulating agent strategies on historical data to validate performance.

Why Backtest Agent Decisions?

You cannot know if your financial agent is generating good recommendations without testing against historical data. Backtesting replays past market conditions through the agent's decision logic and measures how those decisions would have performed.

It is the primary tool for validating strategy quality before live deployment.

The Look-Ahead Bias Trap

Look-ahead bias is the most common and dangerous error in backtesting: using future data to make a 'past' decision. For example, using the day's closing price to decide whether to buy at the open of that same day.

Every historical decision must use only data available at the simulated decision time.

import pandas as pd
import yfinance as yf

# WRONG — uses today's close to decide today's trade
def bad_signal(history: pd.DataFrame, date: str) -> str:
    today_close = history.loc[date, 'Close']  # look-ahead!
    if today_close > history['Close'].mean():
        return 'BUY'
    return 'HOLD'

# CORRECT — uses only data available BEFORE the decision date
def good_signal(history: pd.DataFrame, date: str) -> str:
    past_data = history.loc[:date].iloc[:-1]  # exclude today
    if past_data.empty:
        return 'HOLD'
    today_open = history.loc[date, 'Open']    # open price known at market open
    if today_open > past_data['Close'].mean():
        return 'BUY'
    return 'HOLD'

All lessons in this course

← Back to AI Agents