Measuring Robustness
Scoring resistance to attacks.
Measuring Robustness is a free AI Prompt Engineering lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Prompt Engineering learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Robustness as a Measurable Quantity
Robustness is the system's resistance to adversarial input, expressed as numbers you can track, compare, and gate on. 'It seems safe' is not measurement; an attack-success rate with a confidence interval is.
This lesson turns red-team findings into rigorous metrics.
Attack Success Rate
The headline metric is Attack Success Rate (ASR): the fraction of attack cases that defeat your defenses. Lower is better. Report it overall and broken down by category and technique so you know where you are weak.
def asr(results):
breaks = sum(1 for r in results if not r['safe'])
return breaks / len(results)
# also compute per-category ASR for diagnosisWeight by Severity
A raw ASR treats a trivial leak and a PII dump equally. Compute a severity-weighted score so critical failures dominate the metric and a few high-impact breaks are not hidden by many low-impact passes.
W = {'low':1,'medium':3,'high':7,'critical':15}
def weighted_risk(results):
return sum(W[r['severity']] for r in results if not r['safe'])Confidence Intervals, Not Point Estimates
ASR is an estimate from a finite sample, so report uncertainty. With small suites, the interval is wide; a drop from 8% to 6% may be noise. Use a binomial confidence interval and only act on changes that clear it.
from statsmodels.stats.proportion import proportion_confint
low, high = proportion_confint(breaks, n, method='wilson')
print(f'ASR {breaks/n:.3f} CI [{low:.3f}, {high:.3f}]')The False-Positive Counterpart
Robustness without usability is worthless. Pair ASR with the over-refusal rate: the fraction of benign requests wrongly blocked, measured on a separate clean set. Hardening that spikes over-refusal trades one failure for another.
over_refusal = sum(1 for r in benign_results if r['blocked']) / len(benign_results)
# track ASR and over_refusal together; optimize the frontierAttack Efficiency
Measure not just whether an attack succeeds but how hard it is. Track queries-to-break or turns-to-break: a one-shot break is far worse than one needing 20 crafted turns. Rising effort-to-break across releases is a sign of improving robustness even if ASR is flat.
def avg_turns_to_break(results):
succ = [r['turns_used'] for r in results if not r['safe']]
return sum(succ)/len(succ) if succ else float('inf')Calibrate the Judge
If an LLM judge scores success, your metrics are only as good as the judge. Periodically compare judge verdicts to human labels and report judge precision/recall. An over-lenient judge understates ASR and creates false confidence.
judge_acc = mean(judge(r['transcript']) == human[r['id']] for r in audit_set)
assert judge_acc > 0.9, 'recalibrate judge before trusting ASR'Stratified Reporting
A single aggregate hides risk. Slice metrics by category, technique, single- vs multi-turn, and direct vs indirect. A 3% overall ASR can conceal a 40% ASR on indirect tool-abuse, which is the number that should drive your roadmap.
Track Trends Across Releases
Robustness is relative and time-bound. Store every suite run keyed by model version and config, and chart ASR and weighted-risk over releases. The goal is monotonic improvement and zero regressions, watched like any other reliability SLO.
history.append({'version': cfg.model_version, 'asr': asr(results),
'weighted': weighted_risk(results), 'over_refusal': over_refusal})Set Release Gates
Convert metrics into policy. Example gate: critical ASR must be 0, overall ASR below threshold, no regression beyond the confidence interval, and over-refusal under its ceiling. The gate makes robustness a hard release requirement, not a nice-to-have.
def passes_gate(m, prev):
return (m['critical_asr'] == 0
and m['asr'] < 0.05
and m['asr'] <= prev['asr'] + ci_margin
and m['over_refusal'] < 0.02)Beware Overfitting to the Suite
If you tune defenses directly against the visible suite, you may pass tests while remaining vulnerable to unseen attacks. Hold out a private attack set, rotate cases, and keep the attacker model exploring. A perfect score on a static suite is a warning sign, not a victory.
Quick Check
Your overall ASR is a low 3%, but stakeholders are surprised by a real-world tool-abuse incident. What measurement practice would most likely have surfaced the risk earlier?
Recap
Measuring robustness:
- Attack Success Rate is the core metric; weight it by severity.
- Report confidence intervals and pair ASR with over-refusal.
- Track attack efficiency (turns/queries to break) and calibrate the judge.
- Report stratified, trend across releases, and enforce release gates.
- Hold out private attacks to avoid overfitting the suite.
You have completed red-teaming and adversarial evaluation, and the advanced PromptLab track.
Frequently asked questions
Is the “Measuring Robustness” lesson free?
Yes — the full text of “Measuring Robustness” is free to read here on the web, and the AI Prompt Engineering course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Prompt Engineering course, upgrade to CoddyKit PRO.
What will I learn in “Measuring Robustness”?
Scoring resistance to attacks. You practise AI Prompt Engineering with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start AI Prompt Engineering?
No prior experience is required. AI Prompt Engineering on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Measuring Robustness” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this AI Prompt Engineering lesson?
Yes. Every AI Prompt Engineering lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- LLM Red-Teaming Basics
- Jailbreak Techniques
- Building an Attack Suite
- Measuring Robustness