Managing Configuration and Secrets in Deployment
Learn to externalize configuration and safely inject secrets like API keys into deployed LLM applications using environment variables, config maps, and secret managers.
Managing Configuration and Secrets in Deployment is a free LLM Apps in Production (RAG + Vector DB + Caching) lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the LLM Apps in Production (RAG + Vector DB + Caching) learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Config Belongs Outside Code
The same LLM app image runs in dev, staging, and production. The only difference should be configuration, not the code. Hard-coding endpoints or keys forces a rebuild for every environment.
This is the core idea of config externalization.
Config vs Secrets
Two related but distinct concepts:
- Config — non-sensitive settings: model name, temperature, log level
- Secrets — sensitive values: API keys, DB passwords, tokens
Secrets need stricter handling and must never be logged.
Environment Variables
The simplest portable mechanism is environment variables.
import os
model = os.environ.get('LLM_MODEL', 'gpt-mini')
temp = float(os.environ.get('LLM_TEMPERATURE', '0.2'))
print('Using', model, 'at temp', temp)The Twelve-Factor Approach
The twelve-factor methodology says store config in the environment. This keeps the build artifact identical across environments and avoids accidentally committing secrets into version control.
Kubernetes ConfigMaps
In Kubernetes, non-sensitive config lives in a ConfigMap and is injected as env vars or files.
apiVersion: v1
kind: ConfigMap
metadata:
name: llm-config
data:
LLM_MODEL: 'gpt-mini'
LLM_TEMPERATURE: '0.2'Kubernetes Secrets
Sensitive values go in a Secret object, kept separate from ConfigMaps and mounted with tighter access controls. Base64 encoding is not encryption, so enable encryption at rest.
apiVersion: v1
kind: Secret
metadata:
name: llm-secrets
type: Opaque
stringData:
OPENAI_API_KEY: 'set-via-pipeline'Dedicated Secret Managers
For production, use a dedicated secret manager:
- HashiCorp Vault
- AWS Secrets Manager
- GCP Secret Manager
They offer rotation, audit logs, and fine-grained access far beyond plain env vars.
Fetching Secrets at Runtime
Apps can pull secrets at startup from a manager instead of baking them in. This centralizes rotation.
def load_secret(name):
store = {'OPENAI_API_KEY': 'sk-demo'}
if name not in store:
raise KeyError('missing secret: ' + name)
return store[name]
print(load_secret('OPENAI_API_KEY')[:7])Validating Config at Startup
Fail fast: validate that all required config and secrets are present when the app boots, not when the first request arrives. A clear startup error beats a confusing 500 in production.
Avoiding Secret Leaks
Common leak vectors to guard against:
- Logging full request objects that include keys
- Echoing env vars in debug endpoints
- Committing .env files
- Exposing secrets in error stack traces
Rotation and Per-Environment Keys
Use separate keys per environment and rotate them on a schedule. With a secret manager, rotation updates one place and all instances pick it up without a redeploy.
Quick Check
Test your understanding of Kubernetes config.
Recap
You learned to externalize config from code and separate it from secrets. Use environment variables and ConfigMaps for settings, Secrets and dedicated managers for sensitive values, validate everything at startup, and rotate keys per environment without leaking them in logs.
Frequently asked questions
Is the “Managing Configuration and Secrets in Deployment” lesson free?
Yes — the full text of “Managing Configuration and Secrets in Deployment” is free to read here on the web, and the LLM Apps in Production (RAG + Vector DB + Caching) course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the LLM Apps in Production (RAG + Vector DB + Caching) course, upgrade to CoddyKit PRO.
What will I learn in “Managing Configuration and Secrets in Deployment”?
Learn to externalize configuration and safely inject secrets like API keys into deployed LLM applications using environment variables, config maps, and secret managers. You practise LLM Apps in Production (RAG + Vector DB + Caching) with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start LLM Apps in Production (RAG + Vector DB + Caching)?
No prior experience is required. LLM Apps in Production (RAG + Vector DB + Caching) on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Managing Configuration and Secrets in Deployment” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this LLM Apps in Production (RAG + Vector DB + Caching) lesson?
Yes. Every LLM Apps in Production (RAG + Vector DB + Caching) lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Containerizing LLM Applications with Docker
- Orchestration with Kubernetes for Scalability
- CI/CD for LLM Application Deployment
- Managing Configuration and Secrets in Deployment