OpenAI API: chat.completions and Streaming

The openai Python package is the official way to talk to OpenAI models like GPT-4o. Install it with pip install openai and you get a single OpenAI client class that handles authentication, retries, and request building for you.

Every call you make goes through this client, so creating it once and reusing it is the standard pattern.

Instantiate the client with OpenAI(). By default it reads your key from the OPENAI_API_KEY environment variable, which keeps secrets out of your source code.

You can also pass the key explicitly, but environment variables are the recommended approach for production apps.

Chat models are driven by a list of messages. Each message is a dict with a role and content. The three core roles are system (instructions), user (the human), and assistant (the model).

This list is the full conversation context the model sees on every call.

Call client.chat.completions.create() with a model and your messages. The reply text lives at response.choices[0].message.content.

The API returns a list of choices, but for normal use you read the first one.

Two parameters shape the response: temperature (0 = deterministic, higher = more creative) and max_tokens (caps the reply length). Lower temperatures suit factual tasks; higher suits brainstorming.

Without streaming, you wait for the entire reply before seeing anything. With streaming, tokens arrive as they are generated, so a UI can display text in real time like ChatGPT does.

You enable it with stream=True, which changes the return value from a single object into an iterator.

Pass stream=True to create(). Instead of a completed response, you now get a generator that yields chunks as the model produces them.

Loop over the stream. Each chunk carries a partial piece of text in chunk.choices[0].delta.content. Early and final chunks may have None there, so guard against it before printing.

In a normal response you read message.content (the whole reply). In a stream you read delta.content (just the new piece). The word delta means "the change" since the last chunk.

Concatenating all deltas reconstructs the full message.

To keep a conversation going, append the assistant's reply back into messages and then add the next user message. The model is stateless, so YOU maintain the history.

Non-streamed responses include a usage object with prompt_tokens, completion_tokens, and total_tokens. This is how you track and budget API cost, since billing is per token.

Test your understanding of streaming.

You learned to create an OpenAI client, build a messages list with system/user/assistant roles, and call chat.completions.create(). You read normal replies from choices[0].message.content.

With stream=True you iterate chunks and read delta.content for real-time output, maintain history by appending replies, and track cost via the usage object.