Hardening: Security, Caching, and Reliability
Add prompt injection defenses, semantic caching, circuit breaker fallback to a secondary model, structured tracing, and per-request cost tracking to production-harden the system.
What Production Hardening Means
Production hardening is the process of making a working system safe, cost-efficient, and resilient enough to handle real users and adversarial inputs. A system that works in a demo can fail in production due to prompt injection from malicious users, excessive API costs from repeated queries, or cascading failures when a provider goes down. Hardening addresses all three dimensions: security, cost, and reliability.
Prompt Injection Defense Layer
Add a two-stage injection filter before any user input reaches the LLM. The first stage is a fast rule-based check using pattern matching for common injection phrases like 'ignore previous instructions', 'system:', or 'DAN mode'. The second stage, triggered only when the first stage detects suspicious patterns, uses a small LLM classifier to decide whether the input is a genuine injection attempt or a false positive from the rule-based filter.
import re
INJECTION_PATTERNS = [
r'ignore\s+(all\s+)?previous\s+instructions',
r'you\s+are\s+now\s+in\s+(DAN|developer|jailbreak)\s+mode',
r'system\s*prompt\s*:\s*',
r'override\s+(your\s+)?(instructions|system|safety)',
r'SYSTEM\s*:',
]
def fast_injection_check(user_input: str) -> bool:
text = user_input.lower()
return any(re.search(p, text, re.IGNORECASE) for p in INJECTION_PATTERNS)
async def injection_guard(user_input: str) -> tuple:
if fast_injection_check(user_input):
# Secondary LLM check for false positive reduction
verdict = await llm_injection_classifier(user_input)
if verdict.is_injection:
return False, 'Input rejected by security filter.'
return True, user_inputAll lessons in this course
- Designing the Production Architecture
- Implementing Core RAG and Agent Features
- Hardening: Security, Caching, and Reliability
- Evaluation, Deployment, and Retrospective