AI Agents · Lesson

Visualising Agent Runs (Langfuse, LangSmith)

Drop into Langfuse or LangSmith UIs to inspect a trace tree and replay failed runs.

Visualising Agent Runs (Langfuse, LangSmith) is a free AI Agents lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Two Leading Tools

You can build tracing yourself, but two products do it well out of the box:

Langfuse — OSS, self-hostable, framework-agnostic
LangSmith — closed source, from the LangChain team

Plus Helicone, Phoenix, Weights & Biases for some workflows.

Langfuse Overview

Langfuse gives you:

Trace tree view with timing
Per-span inputs/outputs
Cost and token rollups
Dataset and eval support
Prompt versioning
OSS — run locally with Docker

Langfuse Quick Start

# pip install langfuse openai
from langfuse.openai import openai

response = openai.chat.completions.create(
    model='gpt-4o-mini',
    messages=[{'role': 'user', 'content': 'Hi'}],
    name='greeting'   # span name
)
# Trace automatically appears in Langfuse UI

Manual Spans in Langfuse

For tool calls or custom logic:

from langfuse import Langfuse
lf = Langfuse()

trace = lf.trace(name='agent-run', user_id='u-42')
span = trace.span(name='search-tool', input={'query': 'shipping'})
results = search_kb('shipping')
span.end(output=results)

LangSmith Quick Start

# pip install langsmith
import os
os.environ['LANGCHAIN_TRACING_V2'] = 'true'
os.environ['LANGCHAIN_API_KEY'] = '...'

# Any LangChain code automatically logs to LangSmith
from langchain_openai import ChatOpenAI
llm = ChatOpenAI()
llm.invoke('Hello')
# Trace appears in https://smith.langchain.com

LangSmith Without LangChain

You can use LangSmith from any code with the @traceable decorator:

from langsmith import traceable

@traceable
def ask_agent(query: str):
    # your custom agent code here
    return final_answer

What the UI Shows

List of recent traces with status (ok/error)
One trace -> tree of spans
Per-span: model, latency, tokens, cost, input/output
Replay a trace with edits

Tagging and Filtering

Add tags to traces so you can filter:

trace = lf.trace(name='agent-run', tags=['production', 'tenant:acme', 'tool:rag'])

User Feedback

Capture thumbs up/down from end-users; attach to the trace:

trace.score(name='user-rating', value=1)   # thumbs up
trace.score(name='user-rating', value=0)   # thumbs down

Datasets and Replays

Both tools let you turn traces into eval datasets. Pick 50 production traces that went wrong, save as a dataset, and rerun with a new prompt to compare.

Self-Hosting Langfuse

# docker-compose with Postgres + Clickhouse
version: '3'
services:
  langfuse:
    image: langfuse/langfuse:latest
    ports: ['3000:3000']
    environment:
      DATABASE_URL: postgresql://...
      NEXTAUTH_SECRET: '...'
      SALT: '...'
# Then point LANGFUSE_HOST=http://localhost:3000

Compare Tools

	Langfuse	LangSmith	Helicone
OSS	Yes	No	Yes
Framework	Any	LangChain-first	Any
Eval	Yes	Yes	Basic

Self-Hosted?

Which of these tracing tools can be self-hosted with Docker?

Recap

Use a tracing tool — Langfuse for OSS / self-hosted, LangSmith for tight LangChain integration. Capture user feedback. Build a dataset from real traces for evals.

Frequently asked questions

Is the “Visualising Agent Runs (Langfuse, LangSmith)” lesson free?

Yes — the full text of “Visualising Agent Runs (Langfuse, LangSmith)” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “Visualising Agent Runs (Langfuse, LangSmith)”?

Drop into Langfuse or LangSmith UIs to inspect a trace tree and replay failed runs. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Visualising Agent Runs (Langfuse, LangSmith)” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

Why You Need Tracing for Agents
Logging Tool Calls and Inputs/Outputs
Latency and Cost per Step
Visualising Agent Runs (Langfuse, LangSmith)

← Back to AI Agents