0Pricing
AI Prompt Engineering · Lesson

Understanding the Chat Interface

How LLM chat UIs work: roles, turns, and session context.

Understanding the Chat Interface is a free AI Prompt Engineering lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Prompt Engineering learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

The Three Roles

Every conversation with an LLM is built on three roles: system, user, and assistant.

The system role sets the rules and persona before the conversation starts. The user role is you sending messages. The assistant role is the model replying.

import anthropic

client = anthropic.Anthropic(api_key='sk-ant-your-key-here')

message = client.messages.create(
    model='claude-opus-4-5',
    max_tokens=256,
    system='You are a helpful cooking assistant.',
    messages=[
        {'role': 'user', 'content': 'What is mise en place?'}
    ]
)
print(message.content[0].text)

The System Message

The system message is the model's instruction manual. It runs before the first user turn and stays active for the whole session.

Use it to set persona, tone, domain restrictions, or output format. The model treats it as its guiding context throughout the conversation.

import openai

client = openai.OpenAI(api_key='sk-your-key-here')

response = client.chat.completions.create(
    model='gpt-4o',
    messages=[
        {
            'role': 'system',
            'content': 'You are a senior Python engineer. '
                       'Always include type hints and docstrings in your examples.'
        },
        {
            'role': 'user',
            'content': 'Show me a function that parses JSON safely.'
        }
    ]
)
print(response.choices[0].message.content)

Turns and Conversation Flow

A conversation is a sequence of turns. Each turn alternates: user speaks, assistant replies, user speaks again.

The model sees the entire turn history on every request. That is what makes it feel like a continuous dialogue — but technically each API call is stateless and receives the full context each time.

import openai

client = openai.OpenAI(api_key='sk-your-key-here')

# Each call sends the FULL conversation history
history = [
    {'role': 'system', 'content': 'You are a geography tutor.'},
    {'role': 'user', 'content': 'What is the capital of France?'},
    {'role': 'assistant', 'content': 'The capital of France is Paris.'},
    {'role': 'user', 'content': 'And its population?'}   # follow-up turn
]

response = client.chat.completions.create(
    model='gpt-4o',
    messages=history
)
print(response.choices[0].message.content)

Session Context

Session context is the running memory of your conversation. Every user and assistant message accumulates in the history array.

The model has no separate memory store — it reasons purely from what it sees in the current context window. If you start a new session, the model has no memory of previous sessions unless you re-inject that information.

import anthropic

client = anthropic.Anthropic(api_key='sk-ant-your-key-here')

# Simulating 3-turn session context
conversation = [
    {'role': 'user', 'content': 'My name is Alex.'},
    {'role': 'assistant', 'content': 'Nice to meet you, Alex!'},
    {'role': 'user', 'content': 'What is 5 times 7?'},
    {'role': 'assistant', 'content': '5 times 7 is 35.'},
    {'role': 'user', 'content': 'Can you repeat my name?'}   # uses session context
]

response = client.messages.create(
    model='claude-opus-4-5',
    max_tokens=128,
    messages=conversation
)
print(response.content[0].text)  # expects: Your name is Alex.

Message History Structure

Message history is simply a list of dictionaries, each with a role and content key.

You manage this list yourself in code. After each assistant reply you append its response, then append the next user message — and send the whole list again on the next call.

import openai

client = openai.OpenAI(api_key='sk-your-key-here')

def chat(history, user_message):
    history.append({'role': 'user', 'content': user_message})
    response = client.chat.completions.create(
        model='gpt-4o',
        messages=history
    )
    reply = response.choices[0].message.content
    history.append({'role': 'assistant', 'content': reply})
    return reply, history

history = [{'role': 'system', 'content': 'You are a math tutor.'}]
reply, history = chat(history, 'What is a prime number?')
print(reply)
print('History length:', len(history))

Token Limits Explained

Every model has a context window measured in tokens. A token is roughly 4 characters or 0.75 words in English.

Both your input (system + all history) and the model's output count toward this limit. GPT-4o supports 128k tokens; Claude Opus 4.5 supports 200k. When the limit is reached, older messages must be removed or summarized.

import tiktoken

# Count tokens before sending to avoid exceeding the limit
encoding = tiktoken.encoding_for_model('gpt-4o')

messages = [
    {'role': 'system', 'content': 'You are a helpful assistant.'},
    {'role': 'user', 'content': 'Explain quantum entanglement in simple terms.'}
]

total_tokens = 0
for msg in messages:
    total_tokens += len(encoding.encode(msg['content']))
    total_tokens += 4  # overhead per message

print(f'Estimated input tokens: {total_tokens}')
print(f'GPT-4o limit: 128,000 tokens')
print(f'Budget remaining: {128000 - total_tokens:,} tokens')

What Happens at the Token Limit

When conversation history grows beyond the context window, you have three options:

  • Truncate — drop the oldest messages
  • Summarize — ask the model to compress earlier turns into a brief summary
  • Sliding window — keep only the last N messages

Choosing the wrong strategy can cause the model to lose crucial context and give incoherent replies.

def trim_history(history, max_messages=10, keep_system=True):
    '''Keep the system message and the last N non-system messages.'''
    system_msgs = [m for m in history if m['role'] == 'system']
    non_system  = [m for m in history if m['role'] != 'system']

    if len(non_system) > max_messages:
        non_system = non_system[-max_messages:]
        print(f'Trimmed to last {max_messages} messages.')

    return system_msgs + non_system if keep_system else non_system

history = [{'role': 'system', 'content': 'You are a tutor.'}]
for i in range(15):
    history.append({'role': 'user',      'content': f'Question {i}'})
    history.append({'role': 'assistant', 'content': f'Answer {i}'})

trimmed = trim_history(history, max_messages=6)
print('Trimmed history length:', len(trimmed))

Multi-turn Context in Practice

Let's see context in action. In a multi-turn conversation the model uses every prior message to answer follow-up questions correctly.

This is why you can say 'explain that more simply' without repeating what 'that' was — the model sees the full history and knows what you referred to.

import openai

client = openai.OpenAI(api_key='sk-your-key-here')

history = [{'role': 'system', 'content': 'You are a science teacher.'}]

# Turn 1
history.append({'role': 'user', 'content': 'What is photosynthesis?'})
r1 = client.chat.completions.create(model='gpt-4o', messages=history)
reply1 = r1.choices[0].message.content
history.append({'role': 'assistant', 'content': reply1})

# Turn 2 — refers to prior answer without repeating it
history.append({'role': 'user', 'content': 'Can you explain that using only 3 bullet points?'})
r2 = client.chat.completions.create(model='gpt-4o', messages=history)
print(r2.choices[0].message.content)

Stateless API, Stateful UX

Here is a key insight: the API is completely stateless. The server remembers nothing between calls.

Chat products like ChatGPT create the illusion of memory by storing conversation history in their own database and injecting it back into every API call. You can build the same mechanism yourself.

# Illustration: two isolated API calls vs one with history
import openai

client = openai.OpenAI(api_key='sk-your-key-here')

# ❌ WITHOUT history — model has no memory
r1 = client.chat.completions.create(
    model='gpt-4o',
    messages=[{'role': 'user', 'content': 'My dog is named Biscuit.'}]
)
r2 = client.chat.completions.create(
    model='gpt-4o',
    messages=[{'role': 'user', 'content': "What's my dog's name?"}]
)
print('Without history:', r2.choices[0].message.content)  # will not know

# ✅ WITH history injected
r3 = client.chat.completions.create(
    model='gpt-4o',
    messages=[
        {'role': 'user',      'content': 'My dog is named Biscuit.'},
        {'role': 'assistant', 'content': 'Got it, your dog is named Biscuit!'},
        {'role': 'user',      'content': "What's my dog's name?"}
    ]
)
print('With history:', r3.choices[0].message.content)

Practical Tips for Chat Interface Use

To get the most from the chat interface:

  • Put persistent instructions in the system message, not repeated in every user turn
  • Keep earlier turns concise — verbose history burns tokens fast
  • Start a new session when switching topics to avoid context pollution
  • Inject only the relevant subset of past context when resuming a long project
# Good practice: compact system message + focused history
import anthropic

client = anthropic.Anthropic(api_key='sk-ant-your-key-here')

system = (
    'You are a concise Python code reviewer. '
    'Reply with: 1) one-line verdict, 2) top 3 issues, 3) fixed snippet.'
)

response = client.messages.create(
    model='claude-opus-4-5',
    max_tokens=512,
    system=system,
    messages=[
        {'role': 'user', 'content': 'def add(a,b): return a+b\nprint(add(1,2))'}
    ]
)
print(response.content[0].text)

Recap: The Chat Interface

Let's review the key ideas about how the chat interface works:

  • Three roles: system sets rules, user sends prompts, assistant replies
  • Full history is sent on every API call — the server is stateless
  • Token limits cap total context; trim or summarize when approaching them
  • Session context is just a list you manage in your own code
  • Start fresh sessions when switching topics to keep context clean
# Summary: minimal chat loop skeleton
import openai

client = openai.OpenAI(api_key='sk-your-key-here')
history = [{'role': 'system', 'content': 'You are a helpful assistant.'}]

def ask(question):
    history.append({'role': 'user', 'content': question})
    res = client.chat.completions.create(model='gpt-4o', messages=history)
    answer = res.choices[0].message.content
    history.append({'role': 'assistant', 'content': answer})
    return answer

print(ask('Hello! What can you help me with?'))

Knowledge Check

Test your understanding of the chat interface mechanics.

A user starts a brand-new session and asks the AI: 'What did we discuss yesterday?' The AI has no memory of yesterday's session. Why?

What You've Learned

You now understand the foundation of every AI chat interaction:

  • The system/user/assistant role structure that shapes every conversation
  • How full message history is sent on every call to simulate memory
  • Why the API is stateless and how chat products build memory on top of it
  • How token limits constrain context and how to manage them

This mental model will help you craft better prompts and build smarter AI-powered applications.

Frequently asked questions

Is the “Understanding the Chat Interface” lesson free?

Yes — the full text of “Understanding the Chat Interface” is free to read here on the web, and the AI Prompt Engineering course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Prompt Engineering course, upgrade to CoddyKit PRO.

What will I learn in “Understanding the Chat Interface”?

How LLM chat UIs work: roles, turns, and session context. You practise AI Prompt Engineering with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Prompt Engineering?

No prior experience is required. AI Prompt Engineering on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Understanding the Chat Interface” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Prompt Engineering lesson?

Yes. Every AI Prompt Engineering lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. Understanding the Chat Interface
  2. Types of Requests AI Can Handle
  3. How AI Generates Responses
  4. What AI Cannot Do
← Back to AI Prompt Engineering