0Pricing
AI Engineering Academy · Lesson

Structured Outputs with Pydantic

Define Pydantic models as your output schema, pass them to the API via the new structured outputs feature, and automatically deserialize responses into typed Python objects.

Structured Outputs with Pydantic is a free AI Engineering Academy lesson on CoddyKit — lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Engineering Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Why Pydantic for LLM Output?

Pydantic is a Python data validation library that defines data schemas using Python type hints. It excels at validating and deserializing data from external sources — and LLM output is one of the most unreliable external sources you will encounter. Combining Pydantic schemas with OpenAI's structured outputs gives you type-safe, validated, auto-deserialized responses from an AI model.

Instead of writing data = json.loads(response) followed by manual field extraction and type casting, you get a fully typed Python object where every field is guaranteed to have the correct type, complete with auto-completion in your IDE and runtime validation. This is how professional AI engineering teams handle structured extraction.

Defining a Basic Pydantic Schema

A Pydantic model is a class inheriting from BaseModel with fields defined using Python type annotations. Field types can be Python primitives, other Pydantic models for nesting, or types from the typing module for lists, optionals, and unions.

from pydantic import BaseModel, Field
from typing import Optional, List
from enum import Enum

class Sentiment(str, Enum):
    positive = 'positive'
    negative = 'negative'
    neutral = 'neutral'

class ReviewAnalysis(BaseModel):
    sentiment: Sentiment
    confidence: float = Field(ge=0.0, le=1.0, description='Confidence score 0-1')
    key_themes: List[str] = Field(description='Main topics mentioned in the review')
    summary: str = Field(max_length=200, description='One-sentence summary')
    product_name: Optional[str] = Field(default=None, description='Product mentioned, if any')
    would_recommend: Optional[bool] = None

# Pydantic validates types and constraints at instantiation
example = ReviewAnalysis(
    sentiment=Sentiment.positive,
    confidence=0.95,
    key_themes=['fast delivery', 'good quality'],
    summary='Customer loves the product and quick shipping.',
    product_name='Wireless Headphones',
    would_recommend=True
)
print(example.model_dump_json(indent=2))

Pydantic with OpenAI Structured Outputs

Pass your Pydantic model class directly to the response_format parameter of client.beta.chat.completions.parse(). The SDK automatically converts the model to JSON Schema, sends it to the API, and deserializes the response back into a typed Python object.

import openai
from pydantic import BaseModel
from typing import List, Optional
from enum import Enum

client = openai.OpenAI()

class Sentiment(str, Enum):
    positive = 'positive'
    negative = 'negative'
    neutral = 'neutral'

class ReviewAnalysis(BaseModel):
    sentiment: Sentiment
    confidence: float
    key_themes: List[str]
    summary: str
    would_recommend: Optional[bool]

review_text = '''
I bought this laptop for my design work and I am blown away. It handles Photoshop
like a dream, the screen colors are beautiful, and it has not slowed down once in
three months. Battery life could be better but overall highly recommend!
'''

result = client.beta.chat.completions.parse(
    model='gpt-4o-mini',
    messages=[
        {'role': 'system', 'content': 'Analyze the customer review and extract structured information.'},
        {'role': 'user', 'content': review_text}
    ],
    response_format=ReviewAnalysis
)

analysis = result.choices[0].message.parsed
print(f'Sentiment: {analysis.sentiment.value}')
print(f'Confidence: {analysis.confidence}')
print(f'Themes: {analysis.key_themes}')
print(f'Recommend: {analysis.would_recommend}')

Nested Pydantic Models

Pydantic schemas can reference other Pydantic models, enabling arbitrarily nested structured outputs. This is ideal for extracting hierarchical data from documents like contracts, invoices, resumes, and medical records.

from pydantic import BaseModel
from typing import List, Optional

class Address(BaseModel):
    street: Optional[str]
    city: str
    country: str
    postal_code: Optional[str]

class ContactInfo(BaseModel):
    email: Optional[str]
    phone: Optional[str]
    address: Optional[Address]

class Person(BaseModel):
    full_name: str
    age: Optional[int]
    job_title: Optional[str]
    contact: ContactInfo
    skills: List[str]

# When you pass Person to response_format, the API generates:
# {
#   "full_name": "...",
#   "contact": {
#     "email": "...",
#     "address": { "city": "...", "country": "..." }
#   },
#   "skills": ["...", "..."]
# }
print('Nested model defined - pass to response_format for extraction')

Field Validation with Pydantic Validators

Pydantic validators let you add custom validation logic beyond simple type checking. You can validate that a confidence score is between 0 and 1, that a price is not negative, or that a date string is in the correct format. When the LLM returns a value that fails validation, Pydantic raises a ValidationError that you can catch and handle.

from pydantic import BaseModel, Field, field_validator
from typing import Optional
import re

class ExtractedContact(BaseModel):
    name: str
    email: Optional[str] = None
    phone: Optional[str] = None
    confidence: float = Field(ge=0.0, le=1.0)

    @field_validator('email')
    @classmethod
    def validate_email(cls, v):
        if v is not None:
            # Basic email format check
            if not re.match(r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$', v):
                raise ValueError(f'Invalid email format: {v}')
        return v

    @field_validator('phone')
    @classmethod
    def normalize_phone(cls, v):
        if v is not None:
            # Remove non-digit characters for normalization
            digits = re.sub(r'[^0-9+]', '', v)
            return digits
        return v

try:
    contact = ExtractedContact(name='Alice', email='not-an-email', confidence=0.9)
except Exception as e:
    print(f'Validation error: {e}')

Extracting Lists of Objects

A common pattern is extracting multiple instances of the same entity from a document — all line items from an invoice, all action items from a meeting transcript, all entities from a news article. Wrap your model in a container model with a list field to handle this case cleanly.

import openai
from pydantic import BaseModel
from typing import List

client = openai.OpenAI()

class ActionItem(BaseModel):
    task: str
    assignee: str
    due_date: str  # or use datetime with proper parsing
    priority: str  # high / medium / low

class MeetingNotes(BaseModel):
    meeting_title: str
    action_items: List[ActionItem]
    key_decisions: List[str]

meeting_transcript = '''
Q3 Planning Meeting - June 2025
Decision: Launch new feature in July.
Decision: Extend free trial to 30 days.
Action: Alice to finalize designs by June 30th - High priority.
Action: Bob to write API docs by July 5th - Medium priority.
Action: Carol to set up staging environment by June 28th - High priority.
'''

result = client.beta.chat.completions.parse(
    model='gpt-4o-mini',
    messages=[
        {'role': 'system', 'content': 'Extract structured data from meeting notes.'},
        {'role': 'user', 'content': meeting_transcript}
    ],
    response_format=MeetingNotes
)
notes = result.choices[0].message.parsed
for item in notes.action_items:
    print(f'[{item.priority.upper()}] {item.task} -> {item.assignee} by {item.due_date}')

Optional Fields and Defaults

Real-world documents are incomplete. A resume may not list a phone number; an invoice may not have an invoice number; a product review may not mention the product name. Design your Pydantic models to handle missing data gracefully using Optional fields with appropriate defaults.

A field annotated as Optional[str] = None tells both Pydantic and the LLM that this field may be absent. The model will return null in JSON for fields it cannot extract, and Pydantic will deserialize that as Python's None, letting you handle it cleanly downstream without KeyError exceptions.

Converting Pydantic Models to JSON Schema

The Pydantic model you define is automatically converted to a JSON Schema when passed to the API. You can inspect this schema to understand exactly what the API will enforce, which is helpful for debugging cases where the model is not returning the structure you expect.

from pydantic import BaseModel, Field
from typing import List, Optional
import json

class ProductExtraction(BaseModel):
    name: str = Field(description='Product name as mentioned in the text')
    price_usd: Optional[float] = Field(default=None, description='Price in USD')
    features: List[str] = Field(default_factory=list)
    in_stock: bool = Field(description='Whether the product is currently available')

# See the JSON Schema that will be sent to the API
schema = ProductExtraction.model_json_schema()
print(json.dumps(schema, indent=2))
# This shows exactly what constraints the API will enforce

Handling Extraction Failures

Even with structured outputs, extraction can fail in two ways: the model refuses to respond (returns a refusal), or the document genuinely does not contain the requested information so the model returns nulls for required fields — triggering a Pydantic validation error because required fields cannot be null.

The safest approach is to make all fields Optional with defaults, accept null values for missing data, and apply your own business-logic validation after extraction. This separates the extraction concern (getting data out of text) from the validation concern (checking that the data meets your requirements).

import openai
from pydantic import BaseModel, ValidationError
from typing import Optional

client = openai.OpenAI()

class ContactExtraction(BaseModel):
    name: Optional[str] = None
    email: Optional[str] = None
    phone: Optional[str] = None

try:
    result = client.beta.chat.completions.parse(
        model='gpt-4o-mini',
        messages=[
            {'role': 'system', 'content': 'Extract contact information.'},
            {'role': 'user', 'content': 'I would like to discuss partnership opportunities.'}
        ],
        response_format=ContactExtraction
    )
    msg = result.choices[0].message
    if msg.refusal:
        print('Refused:', msg.refusal)
    else:
        contact = msg.parsed
        if not any([contact.name, contact.email, contact.phone]):
            print('No contact information found in text')
        else:
            print(contact.model_dump())
except ValidationError as e:
    print('Validation failed:', e)

Using Pydantic with instructor Library

The instructor library is a popular third-party package that patches the OpenAI client to support Pydantic-based extraction with automatic retry on validation failure. If the model returns output that fails your Pydantic validation, instructor automatically retries with a prompt that includes the validation error, giving the model a chance to correct itself.

This is especially useful in batch extraction pipelines where you cannot manually review each result, and you want the system to self-correct without human intervention.

# pip install instructor
import instructor
import openai
from pydantic import BaseModel, Field
from typing import Optional

# Patch the OpenAI client with instructor
client = instructor.from_openai(openai.OpenAI())

class ProductInfo(BaseModel):
    name: str
    price_usd: float = Field(gt=0, description='Price must be positive')
    brand: Optional[str] = None

# instructor automatically retries if Pydantic validation fails
product = client.chat.completions.create(
    model='gpt-4o-mini',
    messages=[
        {'role': 'user', 'content': 'The Sony WH-1000XM5 headphones cost $279.99 at Best Buy.'}
    ],
    response_model=ProductInfo,  # instructor-specific parameter
    max_retries=3
)
print(f'{product.name}: ${product.price_usd} by {product.brand}')

Discriminated Unions and Dynamic Schemas

Pydantic supports discriminated unions — a schema where the structure depends on the value of a discriminator field. This is useful when different document types share a common base but have different additional fields. For example, an expense report might have either a flight receipt (with departure/arrival) or a hotel receipt (with check-in/check-out dates).

By using Union types with a Literal discriminator field, you can define a single extraction schema that handles multiple document variants, with the model selecting the correct subtype based on the document content. Pydantic automatically validates against the correct subtype based on the discriminator value.

from pydantic import BaseModel
from typing import Union, Literal, Optional

class FlightExpense(BaseModel):
    expense_type: Literal['flight']
    airline: str
    departure_city: str
    arrival_city: str
    amount_usd: float

class HotelExpense(BaseModel):
    expense_type: Literal['hotel']
    hotel_name: str
    check_in: str
    check_out: str
    amount_usd: float

class MealExpense(BaseModel):
    expense_type: Literal['meal']
    restaurant: Optional[str]
    amount_usd: float

class ExpenseReport(BaseModel):
    submitter: str
    expenses: list[Union[FlightExpense, HotelExpense, MealExpense]]
    total_usd: float

print('Discriminated union schema - model selects correct subtype per item')

Quick Check

Test your understanding of AI Engineering concepts from this lesson.

Lesson Recap

In this lesson you learned: Pydantic BaseModel subclasses define strongly typed extraction schemas that OpenAI's structured outputs enforce at the API level, nested models, Optional fields, and List types handle complex real-world document structures, and the instructor library adds automatic validation-failure retry for robust batch extraction pipelines. Next up we build a complete information extraction pipeline for unstructured text sources.

Frequently asked questions

Is the “Structured Outputs with Pydantic” lesson free?

Yes — the full text of “Structured Outputs with Pydantic” is free to read here on the web, and the AI Engineering Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Engineering Academy course, upgrade to CoddyKit PRO.

What will I learn in “Structured Outputs with Pydantic”?

Define Pydantic models as your output schema, pass them to the API via the new structured outputs feature, and automatically deserialize responses into typed Python objects. You practise AI Engineering Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Engineering Academy?

No prior experience is required. AI Engineering Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 2 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Structured Outputs with Pydantic” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Engineering Academy lesson?

Yes. Every AI Engineering Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. JSON Mode and response_format
  2. Structured Outputs with Pydantic
  3. Extracting Data from Unstructured Text
  4. Validating and Retrying Bad Outputs
← Back to AI Engineering Academy