AI Agents · Lesson

Data Collection: Trajectories and Trace Replay

Replay successful agent traces to build a training set of (state, action) pairs.

Data Collection: Trajectories and Trace Replay is a free AI Agents lesson on CoddyKit — lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Data Is the Product

Fine-tuning is 10% modelling, 90% data. The biggest quality improvements come from better datasets, not bigger models.

What an Agent Trajectory Looks Like

A trajectory captures one complete agent run:

trajectory = {
    'task': 'Refactor the auth module',
    'messages': [
        {'role': 'system', 'content': '...'},
        {'role': 'user', 'content': 'Task...'},
        {'role': 'assistant', 'content': 'Plan...', 'tool_calls': [...]},
        {'role': 'tool', 'content': '...'},
        ...
    ],
    'outcome': 'success'
}

Mining Production Traces

Your best data source is real usage:

for trace in production_traces:
    if trace.outcome == 'success' and trace.user_feedback == 'thumbs_up':
        save_to_training_set(trace)

Trace Replay

Replay a successful trace to verify reproducibility, and to use as a training example:

def replay(trace):
    messages = trace.messages[:1]   # just the initial user msg
    for expected_msg in trace.messages[1:]:
        actual = agent.step(messages)
        if actual.role == 'assistant':
            messages.append(actual)
        elif expected_msg.role == 'tool':
            messages.append(expected_msg)   # replay tool result

Filtering for High-Quality Traces

Outcome is success (tests pass, user happy)
Trace is short (no thrashing)
Tool calls are sensible
No security alerts triggered

Distillation from Big Models

Use a strong model (GPT-4o) to generate trajectories for a tuning target (Llama 8B):

for task in task_list:
    traj = big_model_agent.run(task)
    if traj.success:
        save_for_training(traj)

# Now fine-tune Llama 8B on the GPT-4o traces.

Pruning Bad Trajectories

One bad example poisons training. Aggressive filtering:

Drop traces with errors
Drop traces with tool hallucinations
Drop traces > N tool calls
Drop traces that contradict policy

Format for Fine-Tuning

OpenAI expects JSONL with messages:

{"messages": [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}
{"messages": [...]}

Tool Calls in Training Data

Include tool_calls and tool messages — the model learns the full agent loop:

{
  "messages": [
    {"role": "user", "content": "Weather in Paris?"},
    {"role": "assistant", "tool_calls": [{...}]},
    {"role": "tool", "tool_call_id": "...", "content": "{...}"},
    {"role": "assistant", "content": "It is 18C and sunny."}
  ]
}

Negative Examples

Some training methods (DPO, KTO) benefit from BAD examples too:

preferences = [
    {'prompt': '...', 'chosen': good_response, 'rejected': bad_response}
]

Dataset Size Calculator

Rough volume needed by training method:

SFT for format: 500-2000
SFT for new capability: 5k-50k
DPO: 1k-10k preference pairs
RLHF / RLAIF: 10k-100k+

Versioning the Dataset

Treat your dataset like code. Version it; track which traces were included; reproducible builds.

Human-in-the-Loop Curation

The best datasets pass through a human review queue. Tools like Argilla, Label Studio, Snorkel make this efficient.

Best Trace Source

What's the highest-signal source of training trajectories?

Recap

Trajectories from real successful runs are gold. Distill from strong models when needed. Filter aggressively. Version your dataset.

Frequently asked questions

Is the “Data Collection: Trajectories and Trace Replay” lesson free?

Yes — the full text of “Data Collection: Trajectories and Trace Replay” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “Data Collection: Trajectories and Trace Replay”?

Replay successful agent traces to build a training set of (state, action) pairs. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 2 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Data Collection: Trajectories and Trace Replay” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

When Fine-Tuning Beats Prompting
Data Collection: Trajectories and Trace Replay
LoRA and QLoRA for Cost-Efficient Tuning
Evaluating Tuned Models vs Base

← Back to AI Agents