AI Agents · Lesson

Timeouts and Circuit Breakers

Bound LLM and tool calls with strict timeouts, and use circuit breakers to stop hammering a sick dependency.

Timeouts and Circuit Breakers is a free AI Agents lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Bound Every External Call

Every network call must have a timeout. Without one, a hung connection can freeze the whole agent.

HTTP Timeouts

import requests
response = requests.get(url, timeout=(3, 10))   # 3s connect, 10s read

LLM Timeouts

from openai import OpenAI
client = OpenAI(timeout=30.0)

# Or per call:
response = client.chat.completions.create(
    model='gpt-4o-mini',
    messages=messages,
    timeout=30.0
)

Pick Realistic Timeouts

Measure real-world p99 latency and add 25%. Setting timeout = 3s when p99 is 4s causes constant false failures.

Total Agent Timeout

Wrap the whole agent run with a deadline:

import asyncio

async def run_agent_with_deadline(query, deadline_s=60):
    try:
        return await asyncio.wait_for(run_agent(query), timeout=deadline_s)
    except asyncio.TimeoutError:
        return 'Sorry, this is taking too long. Please try again.'

What Is a Circuit Breaker?

Pattern: when a downstream service starts failing, "open" the circuit — stop calling it for a while. Recover automatically after a cool-down.

States: CLOSED (normal) -> OPEN (fail fast) -> HALF_OPEN (probe) -> CLOSED (recovered).

Implementing with pybreaker

# pip install pybreaker
import pybreaker

breaker = pybreaker.CircuitBreaker(fail_max=5, reset_timeout=60)

@breaker
def fragile_call():
    return requests.get(url, timeout=5).json()

After 5 Failures

Once fail_max errors hit in a row, the breaker opens. All subsequent calls fail immediately for reset_timeout seconds — saving you from hammering a sick service.

Returning Sensible Fallbacks

When the breaker is open, return a graceful fallback instead of crashing the agent:

try:
    result = fragile_call()
except pybreaker.CircuitBreakerError:
    result = {'error': 'service-unavailable', 'fallback': 'cached or stale data'}

Per-Tool Breakers

Each external dependency should have its own breaker — one bad service should not affect calls to a healthy one.

Bulkheads

Limit the number of concurrent calls to any one tool with a semaphore — prevents one slow dependency from exhausting all workers:

from asyncio import Semaphore
search_sem = Semaphore(10)

async def search(query):
    async with search_sem:
        return await tavily_search(query)

Health Checks

Periodically ping each tool to detect issues before users do:

async def health_check():
    for tool in [tavily, openai, vector_db]:
        try:
            await tool.ping()
        except Exception as e:
            alert(f'{tool.name} unhealthy: {e}')

Surface Status to Users

When a critical tool is down, tell the user:

if tavily_breaker.current_state == 'open':
    return 'Web search is temporarily unavailable.'

Why Circuit Break?

Why use a circuit breaker on a flaky external service?

Recap

Timeouts on every call. Per-tool circuit breakers. Bulkheads for concurrency. Health checks. Graceful fallback messages.

Frequently asked questions

Is the “Timeouts and Circuit Breakers” lesson free?

Yes — the full text of “Timeouts and Circuit Breakers” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “Timeouts and Circuit Breakers”?

Bound LLM and tool calls with strict timeouts, and use circuit breakers to stop hammering a sick dependency. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Timeouts and Circuit Breakers” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to AI Agents