0Pricing
AI Engineering Academy · Lesson

The Chat Completions Endpoint

Understand the messages array with system, user, and assistant roles, craft your first prompt, and interpret the response object that comes back from the API.

The Chat Completions Endpoint is a free AI Engineering Academy lesson on CoddyKit — lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Engineering Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

The Messages Array Architecture

The Chat Completions endpoint runs on a messages array: a list of turns, each with a role (system, user, or assistant). The model is stateless, so you send the history every time.

from openai import OpenAI

client = OpenAI()

response = client.chat.completions.create(
    model='gpt-4o-mini',
    messages=[
        {'role': 'system', 'content': 'You are a concise Python tutor.'},
        {'role': 'user', 'content': 'What is a list comprehension?'}
    ]
)

print(response.choices[0].message.content)

The System Role: Defining Behavior

The system message is your biggest lever. It sets the model's persona, rules, and format before the user types a word. Invest time here — it shapes everything. See the code.

system_prompt = '''You are a customer support agent for TechShop.
You help customers with: order tracking, returns, and product questions.
You do NOT discuss pricing changes or competitor products.
Always respond in 2-3 sentences maximum.
If you cannot help, say: 'Let me connect you with a human agent.'
'''

response = client.chat.completions.create(
    model='gpt-4o-mini',
    messages=[
        {'role': 'system', 'content': system_prompt},
        {'role': 'user', 'content': 'Where is my order #12345?'}
    ]
)

Multi-Turn Conversation Management

To keep a conversation going, you append each turn to the messages array and resend it all. That's how the model seems to remember — you're feeding it the full history.

history = [
    {'role': 'system', 'content': 'You are a helpful assistant.'}
]

def chat(user_message):
    history.append({'role': 'user', 'content': user_message})
    response = client.chat.completions.create(
        model='gpt-4o-mini',
        messages=history
    )
    assistant_reply = response.choices[0].message.content
    history.append({'role': 'assistant', 'content': assistant_reply})
    return assistant_reply

print(chat('My name is Alice.'))
print(chat('What is my name?'))  # model remembers 'Alice'

Anatomy of the API Response

The response is an object, not just text. choices holds the reply, finish_reason says why it stopped, and usage counts tokens — which is your cost. Log these in production.

response = client.chat.completions.create(
    model='gpt-4o-mini',
    messages=[{'role': 'user', 'content': 'Say hello in one word.'}]
)

# Accessing response fields
print('Content:', response.choices[0].message.content)
print('Finish reason:', response.choices[0].finish_reason)  # 'stop'
print('Model:', response.model)  # exact version like gpt-4o-mini-2024-07-18
print('Prompt tokens:', response.usage.prompt_tokens)
print('Completion tokens:', response.usage.completion_tokens)
print('Total tokens:', response.usage.total_tokens)

Understanding finish_reason

finish_reason tells you why generation stopped. 'stop' means done; 'length' means it hit max_tokens and got cut off mid-answer. Always check it — truncation is a silent bug.

def safe_completion(messages, max_tokens=500):
    response = client.chat.completions.create(
        model='gpt-4o-mini',
        messages=messages,
        max_tokens=max_tokens
    )
    choice = response.choices[0]
    
    if choice.finish_reason == 'length':
        print(f'WARNING: Response was truncated at {max_tokens} tokens!')
    elif choice.finish_reason == 'content_filter':
        print('WARNING: Response blocked by content filter!')
        return None
    
    return choice.message.content

Selecting the Right Model

Pick the right model for the job. gpt-4o is the powerhouse for hard reasoning; gpt-4o-mini is far cheaper and handles most tasks well. Benchmark before assuming bigger wins.

# Model comparison guidance
models = {
    'gpt-4o': {
        'use_for': 'Complex reasoning, code generation, nuanced analysis',
        'input_cost_per_1M': 2.50,  # USD
        'output_cost_per_1M': 10.00
    },
    'gpt-4o-mini': {
        'use_for': 'Classification, extraction, summarization, Q&A',
        'input_cost_per_1M': 0.15,
        'output_cost_per_1M': 0.60
    }
}
# gpt-4o is ~17x more expensive on input tokens

Content Types in Messages

A message's content can be more than text. For vision models like gpt-4o, you pass a list mixing text and images — so you can ask questions about charts or screenshots.

# Sending an image to a vision-capable model
response = client.chat.completions.create(
    model='gpt-4o',
    messages=[
        {
            'role': 'user',
            'content': [
                {
                    'type': 'text',
                    'text': 'What is in this image? Describe in one sentence.'
                },
                {
                    'type': 'image_url',
                    'image_url': {'url': 'https://example.com/photo.jpg'}
                }
            ]
        }
    ]
)

The n Parameter: Multiple Completions

The n parameter returns several completions for one prompt. Handy for picking the best, or for confidence: if all n agree, the model is sure; if they clash, be wary.

response = client.chat.completions.create(
    model='gpt-4o-mini',
    messages=[{'role': 'user', 'content': 'Name the capital of Germany.'}],
    n=3,  # generate 3 independent completions
    temperature=0.5
)

for i, choice in enumerate(response.choices):
    print(f'Completion {i+1}: {choice.message.content}')

# Check if all completions agree (confidence signal)
answers = [c.message.content.strip() for c in response.choices]
print('All agree:', len(set(answers)) == 1)

Handling the Response as a String

To grab the reply as text, the path is always response.choices[0].message.content. Wrap it in a helper — and guard for None, which happens on tool calls or filters.

def get_completion(prompt, system='You are a helpful assistant.', model='gpt-4o-mini'):
    '''Simple helper that returns the response text as a string.'''
    response = client.chat.completions.create(
        model=model,
        messages=[
            {'role': 'system', 'content': system},
            {'role': 'user', 'content': prompt}
        ]
    )
    content = response.choices[0].message.content
    if content is None:
        raise ValueError(f'No content in response. Finish reason: {response.choices[0].finish_reason}')
    return content

result = get_completion('Explain recursion in one sentence.')
print(result)

Inspecting the Raw Request and Response

Debugging odd replies? Inspect the raw request and response. Setting OPENAI_LOG=debug prints the full body to your terminal — the fastest way to see what's on the wire.

import json
import httpx

# Enable debug logging (shows full request/response)
import os
os.environ['OPENAI_LOG'] = 'debug'

# Or use a custom logging client:
class LoggingClient(httpx.Client):
    def send(self, request, *args, **kwargs):
        print('REQUEST:', request.method, request.url)
        print('BODY:', json.loads(request.content))
        response = super().send(request, *args, **kwargs)
        print('STATUS:', response.status_code)
        return response

Building a Minimal Chat Loop

Now you can build a minimal chat loop: keep a messages list, append each turn, send it all, repeat. That simple pattern powers every chat app on the API. The code shows it.

import openai

client = openai.OpenAI()

SYSTEM_PROMPT = 'You are a helpful assistant. Be concise.'

def simple_chat_loop():
    messages = [{'role': 'system', 'content': SYSTEM_PROMPT}]
    print('Chat started. Type "quit" to exit.')

    while True:
        user_input = input('You: ').strip()
        if user_input.lower() == 'quit':
            break
        if not user_input:
            continue

        messages.append({'role': 'user', 'content': user_input})

        response = client.chat.completions.create(
            model='gpt-4o-mini',
            messages=messages,
            max_tokens=500
        )

        assistant_reply = response.choices[0].message.content
        messages.append({'role': 'assistant', 'content': assistant_reply})
        print(f'Assistant: {assistant_reply}\n')

print('Example chat loop defined. Run simple_chat_loop() to start.')

Quick Check

Test your understanding of AI Engineering concepts from this lesson.

Lesson Recap

You learned the chat core: the messages array controls the conversation, and the response carries content, finish_reason, and token counts. Next: parameters.

Frequently asked questions

Is the “The Chat Completions Endpoint” lesson free?

Yes — the full text of “The Chat Completions Endpoint” is free to read here on the web, and the AI Engineering Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Engineering Academy course, upgrade to CoddyKit PRO.

What will I learn in “The Chat Completions Endpoint”?

Understand the messages array with system, user, and assistant roles, craft your first prompt, and interpret the response object that comes back from the API. You practise AI Engineering Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Engineering Academy?

No prior experience is required. AI Engineering Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 2 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “The Chat Completions Endpoint” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Engineering Academy lesson?

Yes. Every AI Engineering Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. Setting Up Your Python Environment
  2. The Chat Completions Endpoint
  3. Controlling Model Behavior with Parameters
  4. Error Handling and Rate Limits
← Back to AI Engineering Academy