AI Agents · Lesson

Retries with Exponential Backoff

Use tenacity to retry transient failures with exponentially-increasing delays and jitter.

Retries with Exponential Backoff is a free AI Agents lesson on CoddyKit — lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Why Retries?

External calls (LLMs, search APIs, vector DBs) fail transiently. A simple retry recovers from most flakes without bothering the user.

What NOT to Do

Do not retry instantly in a tight loop — you will hammer the service and the rate-limiter will lock you out:

# BAD
for _ in range(10):
    try:
        return call_api()
    except Exception:
        pass

Exponential Backoff

Wait longer after each failure. The exponent of 2 is the convention:

import time

def call_with_backoff(fn, attempts=5):
    for i in range(attempts):
        try:
            return fn()
        except TransientError:
            if i == attempts - 1:
                raise
            time.sleep(2 ** i)   # 1, 2, 4, 8, 16 seconds

Add Jitter

Without jitter, retries from many clients align — creating thundering-herd traffic. Add random offset:

import random
sleep = (2 ** i) + random.uniform(0, 1)
time.sleep(sleep)

Use Tenacity

The tenacity library implements this for you:

from tenacity import retry, wait_exponential_jitter, stop_after_attempt, retry_if_exception_type
import requests

@retry(
    wait=wait_exponential_jitter(initial=1, max=30),
    stop=stop_after_attempt(5),
    retry=retry_if_exception_type((requests.Timeout, requests.ConnectionError))
)
def search(query):
    return requests.get('https://api.tavily.com/search', timeout=10).json()

Retry Only Transient Errors

Never retry on:

4xx errors (bad request — retrying will not help)
Authentication failures
Validation errors

Retry on:

Timeouts
5xx server errors
Rate limits (with longer waits)

Rate-Limit-Aware Retries

Honour the Retry-After header:

if response.status_code == 429:
    wait = int(response.headers.get('Retry-After', 5))
    time.sleep(wait)
    continue

Bound Total Retry Time

Cap how long retries can take overall — usually 30-60 seconds for a user-facing call:

from tenacity import stop_after_delay

@retry(
    wait=wait_exponential_jitter(initial=1, max=10),
    stop=stop_after_delay(60)
)
def call_api():
    ...

Idempotency Headers

For non-idempotent APIs (Stripe, etc.), include an idempotency key on every retry so duplicates are collapsed:

headers = {'Idempotency-Key': f'agent-call-{run_id}-{tool_id}'}

Retry the LLM Call Too

OpenAI and Anthropic SDKs auto-retry 1-2 times. You can configure:

client = OpenAI(max_retries=5, timeout=30.0)

Async Retries

For async code:

from tenacity import AsyncRetrying

async for attempt in AsyncRetrying(wait=wait_exponential_jitter(), stop=stop_after_attempt(5)):
    with attempt:
        result = await call_api()

Logging Retries

Every retry should log: attempt number, sleep duration, error reason. Otherwise debugging "why so slow?" is impossible.

Retries vs Timeouts

Set a tight timeout on each attempt (e.g. 10s) and a total cap via stop_after_delay (e.g. 60s). Without both, a stuck connection can hang forever.

Why Jitter?

Why add random jitter to retry delays?

Recap

Exponential backoff + jitter + bounded total time + retry only on transient errors. Use tenacity to skip the boilerplate.

Frequently asked questions

Is the “Retries with Exponential Backoff” lesson free?

Yes — the full text of “Retries with Exponential Backoff” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “Retries with Exponential Backoff”?

Use tenacity to retry transient failures with exponentially-increasing delays and jitter. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 2 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Retries with Exponential Backoff” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to AI Agents