0Pricing
AI Agents · Lesson

Blue-Green and Canary Deploys for Agents

Roll new prompts and models to 5% of traffic, watch metrics, then ramp — never big-bang.

Blue-Green and Canary Deploys for Agents is a free AI Agents lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Deploying Agents Is Different

Traditional code deploys are deterministic — a unit test pass means the change is safe. LLM agent deploys are NOT — even with the same code, swapping a prompt or model can subtly degrade quality.

Use deployment strategies designed to detect regressions early.

Blue-Green Deploys

Two identical environments:

  • Blue — current production
  • Green — new version

Switch traffic 0% -> 100% in one step after testing green.

Canary Deploys

Gradually shift traffic to the new version:

  1. 5% -> watch metrics
  2. 20% -> watch
  3. 50%
  4. 100%

Roll back at any step if metrics degrade.

What to Watch

  • Error rate
  • p95 latency
  • Cost per request
  • User-feedback signals (thumbs up/down ratio)
  • Eval pass rate on a live-traffic shadow set

Routing With a Feature Flag

import random

def pick_agent_version(user_id):
    if launchdarkly.variation('use-v2', user_id, default=False):
        return v2
    if random.random() < canary_pct:
        return v2
    return v1

# Roll canary_pct from 0 to 1.0 gradually.

Per-Tenant Rollout

Roll out per tenant in B2B SaaS — internal users first, then friendly customers, then everyone:

if user.org_id in EARLY_ACCESS_ORGS:
    return v2
return v1

Shadow Mode

Run the new version IN PARALLEL but don't serve its output:

v1_result = run_v1(query)
async def shadow():
    v2_result = await run_v2(query)
    log_compare(v1_result, v2_result)
asyncio.create_task(shadow())
return v1_result

Replay Production Traffic

Capture production queries to a buffer; replay nightly on the new version. Compare answer quality before promoting.

A/B Tests

For long-running quality experiments, split traffic randomly and statistically compare metrics over weeks.

Prompt Versioning

Treat prompts like code — pin a version, ship side-by-side with the model, roll out via flag:

PROMPTS = {
    'qa-v3': '...',
    'qa-v4': '...'
}
active = 'qa-v4' if feature_flag_on() else 'qa-v3'

Model Pinning

Pin the model version (gpt-4o-2024-08-06) so providers updating their default models doesn't silently change behavior overnight.

Automated Rollback

If a metric crosses a threshold, automatically pull the canary back to 0% and alert:

if p95_latency_now > 1.5 * p95_latency_baseline:
    flag.set('canary_pct', 0)
    alert('Auto-rollback triggered')

Document Each Release

Every release notes:

  • What changed (prompt, model, retrieval)
  • Eval results before/after
  • Canary timeline
  • Whether rolled back

Why Canary?

Why use canary deploys for agents instead of one-step releases?

Recap

Feature-flag traffic, ramp 5% -> 100%, watch error / latency / cost / feedback, roll back fast if needed. Add shadow mode and replay for confidence.

Frequently asked questions

Is the “Blue-Green and Canary Deploys for Agents” lesson free?

Yes — the full text of “Blue-Green and Canary Deploys for Agents” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “Blue-Green and Canary Deploys for Agents”?

Roll new prompts and models to 5% of traffic, watch metrics, then ramp — never big-bang. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Blue-Green and Canary Deploys for Agents” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. Serving Agents Behind an API
  2. Async Workflows and Background Jobs
  3. Rate Limiting and Quota Management
  4. Blue-Green and Canary Deploys for Agents
← Back to AI Agents