Token Usage & Cost Monitoring
Measure how many tokens your agents consume and what they cost in real time, set budgets, and find the expensive steps so you can optimize spend in production.
Token Usage & Cost Monitoring is a free AI Agents with LangChain & Autonomous Workflows lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents with LangChain & Autonomous Workflows learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Why Track Tokens and Cost
Agents that loop, retry, or use long context can quietly become expensive. Without visibility you only discover the bill at month end.
Token and cost monitoring turns spend into a metric you can watch, alert on, and optimize.
Prompt vs Completion Tokens
Every call splits into:
- Prompt tokens: everything you send in (system prompt, context, history)
- Completion tokens: what the model generates
They are usually priced differently, so track them separately.
The Callback Approach
LangChain exposes usage via callbacks. get_openai_callback aggregates tokens and cost for everything inside its context block.
from langchain_community.callbacks import get_openai_callback
with get_openai_callback() as cb:
result = agent.invoke({'input': 'Summarize the report'})
print(cb.total_tokens, cb.total_cost)Reading the Breakdown
The callback object also exposes the split, which is what you log per request.
print('prompt:', cb.prompt_tokens)
print('completion:', cb.completion_tokens)
print('cost USD:', cb.total_cost)Usage Inside Responses
Many chat models also attach a usage_metadata field to the response, useful when you call the model directly without a callback.
resp = llm.invoke('Hello there')
print(resp.usage_metadata)Estimating Before You Send
To stay under a budget, estimate tokens before calling the model using a tokenizer like tiktoken. This catches oversized prompts early.
import tiktoken
enc = tiktoken.encoding_for_model('gpt-4o-mini')
n = len(enc.encode(prompt_text))
print('approx tokens:', n)Setting Budgets
Define a per-request and per-user token budget. If an estimate exceeds it, trim context, lower k in retrieval, or reject the request before paying for it.
MAX_TOKENS = 6000
if n > MAX_TOKENS:
raise ValueError('Request exceeds token budget')Per-Step Attribution
Agents make many sub-calls: tool selection, tool output processing, final answer. Wrapping each step's callback shows which step dominates cost, so you optimize the right one.
Logging to a Dashboard
Emit tokens and cost as structured logs or metrics (e.g. to Prometheus or LangSmith). Tag them with user, model, and route so you can slice spend.
log.info('llm_usage', extra={
'tokens': cb.total_tokens,
'cost': cb.total_cost,
'route': 'support_agent'
})Common Savings
Once you can see spend, the biggest wins are usually:
- Smaller models for simple steps
- Caching repeated calls
- Trimming history and retrieved context
- Stopping runaway agent loops with iteration limits
Alerting on Anomalies
Set alerts for sudden cost spikes — often a sign of a prompt-injection loop or a misbehaving tool. Catching it in minutes beats finding it on the invoice.
Quick Check
Test your cost monitoring knowledge.
Recap
You learned to observe agent spend:
- Split prompt vs completion tokens
- Use
get_openai_callbackandusage_metadata - Estimate with
tiktokenand enforce budgets - Attribute cost per agent step
- Log to dashboards and alert on spikes
Visibility into cost is the foundation for optimizing production agents.
Frequently asked questions
Is the “Token Usage & Cost Monitoring” lesson free?
Yes — the full text of “Token Usage & Cost Monitoring” is free to read here on the web, and the AI Agents with LangChain & Autonomous Workflows course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents with LangChain & Autonomous Workflows course, upgrade to CoddyKit PRO.
What will I learn in “Token Usage & Cost Monitoring”?
Measure how many tokens your agents consume and what they cost in real time, set budgets, and find the expensive steps so you can optimize spend in production. You practise AI Agents with LangChain & Autonomous Workflows with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start AI Agents with LangChain & Autonomous Workflows?
No prior experience is required. AI Agents with LangChain & Autonomous Workflows on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Token Usage & Cost Monitoring” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this AI Agents with LangChain & Autonomous Workflows lesson?
Yes. Every AI Agents with LangChain & Autonomous Workflows lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- LangSmith for Tracing & Monitoring
- Debugging Agent Thought Processes
- Evaluating Agent Performance
- Token Usage & Cost Monitoring