AI Agents · Lesson

Summarisation as Compression

Periodically replace old turns with a running summary, trading exact recall for context window space.

Summarisation as Compression is a free AI Agents lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Compression as Memory

Once a conversation gets long, replace old turns with a short summary. You trade exact recall for context space.

The summary captures the gist (facts, decisions, pending items) — verbatim text is gone.

When to Summarise

Trigger summarisation when:

Total tokens > threshold (e.g. 8k)
Turn count > N (e.g. 20)
User starts a new topic

Summarise the Older Turns

Keep recent turns verbatim; summarise only what fell off:

def compact(messages, keep_recent=10):
    system = messages[0]
    old = messages[1:-keep_recent]
    recent = messages[-keep_recent:]
    if not old:
        return messages
    summary = summarise(old)
    return [
        system,
        {'role': 'system', 'content': f'Earlier conversation summary:\n{summary}'},
        *recent
    ]

Summary Prompt

Use a separate, cheap model call to make the summary:

def summarise(messages):
    text = '\n'.join(f'{m["role"]}: {m["content"]}' for m in messages)
    response = client.chat.completions.create(
        model='gpt-4o-mini',
        messages=[
            {'role': 'system', 'content': 'Summarise this conversation in 5 bullet points. Preserve facts, decisions, and pending TODOs.'},
            {'role': 'user', 'content': text}
        ],
        max_tokens=300,
    )
    return response.choices[0].message.content

Running Summaries

Maintain a single growing summary instead of re-summarising from scratch each time:

running_summary = ''

def extend_summary(new_messages):
    global running_summary
    response = client.chat.completions.create(
        model='gpt-4o-mini',
        messages=[
            {'role': 'system', 'content': 'Extend the existing summary with the new turns. Keep it under 200 words.'},
            {'role': 'user', 'content': f'Existing summary:\n{running_summary}\n\nNew turns:\n{format(new_messages)}'}
        ],
        max_tokens=500,
    )
    running_summary = response.choices[0].message.content

Structured Summaries

Force the summary to extract specific fields:

summary_schema = {
    'topics_discussed': '...',
    'user_facts': {'name': '...', 'preferences': '...'},
    'open_questions': ['...'],
    'decisions_made': ['...']
}
# Use json_object response_format to enforce.
import json
print(json.dumps(summary_schema, indent=2))

Hierarchical Summaries

For very long histories:

Per-100-turn summaries (mid-level)
Per-1000-turn summaries (high-level)
Lifetime summary (extreme)

Inject only the appropriate level into the context.

Losing Detail

Summaries lose specific quotes and exact wording. If the user asks "what exactly did I say about pricing earlier?" the model cannot answer.

For verbatim recall, you need either bigger context or a vector-search archive.

Vector-Archive Both

Best of both worlds:

Summarise old turns into the system message
ALSO embed each turn into a vector store
If the user asks for specifics, retrieve verbatim from the vector store

Summary Quality Matters

A garbage summary is worse than no summary. Use a cheap-but-capable model (gpt-4o-mini, haiku) — not the cheapest tier — for summarisation calls.

Summary Updates Are Expensive

Summarising 8000 tokens of conversation is a real LLM call — 1-2 seconds, 0.5 cents. Do it asynchronously between turns rather than on the critical path.

Summary Trade-off

What is the main trade-off when using summarisation as memory?

Recap

Summarisation is the cheap way to extend memory. Combine with vector retrieval for exact-quote recall.

Frequently asked questions

Is the “Summarisation as Compression” lesson free?

Yes — the full text of “Summarisation as Compression” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “Summarisation as Compression”?

Periodically replace old turns with a running summary, trading exact recall for context window space. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Summarisation as Compression” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to AI Agents