Calling OpenAI API: chat.completions

Why chat.completions?

Install the SDK

# pip install openai
from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY from env

Your First Call

response = client.chat.completions.create(
    model='gpt-4o-mini',
    messages=[
        {'role': 'system', 'content': 'You are concise.'},
        {'role': 'user', 'content': 'Capital of France?'},
    ],
)
print(response.choices[0].message.content)
# 'Paris'

Response Object Anatomy

response.choices[0].message.content       # the text
response.choices[0].finish_reason          # 'stop' / 'length' / 'tool_calls'
response.usage.prompt_tokens               # input tokens
response.usage.completion_tokens           # output tokens
response.usage.total_tokens                # sum
response.id                                # 'chatcmpl-...' for tracing

Key Parameters

Temperature: Pick by Task

Stop Sequences

response = client.chat.completions.create(
    model='gpt-4o-mini',
    messages=messages,
    stop=['\n\n', 'END'],
)
# Stops as soon as a blank line or 'END' appears.

Multiple Choices

response = client.chat.completions.create(
    model='gpt-4o-mini',
    messages=messages,
    n=3,  # three samples
)
for choice in response.choices:
    print(choice.message.content)

Error Types to Handle

Async Calls

from openai import AsyncOpenAI
import asyncio

client = AsyncOpenAI()

async def ask(q):
    r = await client.chat.completions.create(
        model='gpt-4o-mini',
        messages=[{'role': 'user', 'content': q}],
    )
    return r.choices[0].message.content

results = asyncio.run(asyncio.gather(*[ask(q) for q in questions]))

Timeout and Retries Built-In

client = OpenAI(
    timeout=30.0,
    max_retries=3,
)

Finish Reasons

Recap