Schema-Driven Data Extraction
Providing JSON schemas in prompts to guarantee structured output format.
Schema-Driven Data Extraction is a free AI Prompt Engineering lesson on CoddyKit — lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Prompt Engineering learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Why Schema-Driven Extraction?
When you tell a model extract the important data, you get inconsistent, unpredictable output. When you provide a JSON schema and say extract data matching this exact schema, you get machine-readable, consistent, type-safe output every time.
Schema-driven extraction is the pattern used in production systems that process invoices, contracts, medical records, meeting notes, and any document where structured data must be reliably extracted from unstructured text.
Providing the Schema in the Prompt
The schema lives directly in the prompt. The model uses it as the output contract:
import anthropic, json
client = anthropic.Anthropic(api_key='YOUR_API_KEY')
INVOICE_SCHEMA = '''
{
"invoice_number": "string",
"vendor_name": "string",
"vendor_address": "string or null",
"invoice_date": "YYYY-MM-DD",
"due_date": "YYYY-MM-DD or null",
"line_items": [
{
"description": "string",
"quantity": "number",
"unit_price": "number",
"total": "number"
}
],
"subtotal": "number",
"tax": "number or null",
"total_amount": "number",
"currency": "3-letter ISO code e.g. USD"
}
'''
def extract_invoice(invoice_text):
prompt = f'Extract structured data from this invoice.\nReturn JSON matching this schema exactly:\n{INVOICE_SCHEMA}\n\nInvoice:\n{invoice_text}'
r = client.messages.create(model='claude-opus-4-5', max_tokens=500, messages=[{'role': 'user', 'content': prompt}])
return json.loads(r.content[0].text)
print('Invoice schema defined.')Invoice Extraction Example
Applying the schema to extract structured data from a real invoice text:
invoice_text = '''
INVOICE #INV-2025-0342
From: Acme Software Ltd.
123 Tech Street, San Francisco, CA 94105
Date: March 15, 2025
Due: April 14, 2025
Items:
- Annual Pro License (5 seats) x1 @ $2,400.00 = $2,400.00
- Setup & Onboarding x2 @ $300.00 = $600.00
Subtotal: $3,000.00
Tax (8.5%): $255.00
TOTAL DUE: $3,255.00 USD
'''
result = extract_invoice(invoice_text)
print(f'Invoice: {result["invoice_number"]}')
print(f'Vendor: {result["vendor_name"]}')
print(f'Total: {result["currency"]} {result["total_amount"]}')
print(f'Line items: {len(result["line_items"])}')Meeting Notes Extraction
Schema-driven extraction applied to meeting notes — a less structured document type:
MEETING_SCHEMA = '''
{
"meeting_title": "string",
"date": "YYYY-MM-DD",
"attendees": ["string"],
"decisions": ["string"],
"action_items": [
{
"task": "string",
"owner": "string or null",
"due_date": "YYYY-MM-DD or null"
}
],
"next_meeting": "string or null"
}
'''
meeting_notes = '''
Product Sync - March 20, 2025
Attendees: Sarah (PM), Jake (Engineering), Priya (Design)
Decided to push the v2.0 launch to April 15.
Will not include the analytics dashboard in v2.0.
Actions:
- Jake to fix the login bug by March 25
- Priya to finalize mockups by March 22
- Sarah to send updated roadmap to stakeholders (no date set)
Next sync: March 27, same time.
'''
print(f'Meeting schema: {len(MEETING_SCHEMA)} chars')
print(f'Notes length: {len(meeting_notes)} chars')Product Spec Extraction
Extracting structured product specifications from a catalog description:
PRODUCT_SCHEMA = '''
{
"product_name": "string",
"sku": "string or null",
"category": "string",
"price": {"amount": "number", "currency": "string"},
"dimensions": {
"length_cm": "number or null",
"width_cm": "number or null",
"height_cm": "number or null",
"weight_kg": "number or null"
},
"colors": ["string"],
"materials": ["string"],
"features": ["string"],
"in_stock": true | false
}
'''
product_text = 'AlphaDesk Pro standing desk. SKU: AD-PRO-001. $899. Available in white and black. 120x60x75cm, 35kg. Steel frame, bamboo top. Features: memory height, anti-collision, app control. In stock.'
prompt = f'Extract product specs. Return JSON:\n{PRODUCT_SCHEMA}\n\nProduct: {product_text}'
r = client.messages.create(model='claude-opus-4-5', max_tokens=400, messages=[{'role': 'user', 'content': prompt}])
print(json.loads(r.content[0].text))Handling Optional Fields
Schemas must handle optional fields gracefully. Use null as the default for missing data rather than omitting the field — this keeps the output structure consistent:
prompt_optional = '''
Extract the data. For fields not present in the source text,
use null — do NOT omit the field.
Every field in the schema must appear in the output.
Schema:
{
"company": "string",
"ceo": "string or null",
"founded": "YYYY or null",
"revenue": "string or null",
"employees": "number or null"
}
Text: Vertex AI Solutions is a B2B SaaS company.
'''
# Expected output: ceo, founded, revenue, employees all set to null
# NOT omitted — null fields are still present in the JSON
print(prompt_optional)Multi-Document Extraction with the Same Schema
The same schema can be applied across many documents consistently. This is how you build a structured database from unstructured documents at scale:
def extract_many(documents, schema):
results = []
for i, doc in enumerate(documents):
try:
r = client.messages.create(
model='claude-opus-4-5', max_tokens=400,
messages=[{'role': 'user', 'content': f'Extract data. Return JSON matching schema:\n{schema}\n\nDocument:\n{doc}'}]
)
parsed = json.loads(r.content[0].text)
parsed['_source_doc'] = i
parsed['_extraction_ok'] = True
results.append(parsed)
except (json.JSONDecodeError, Exception) as e:
results.append({'_source_doc': i, '_extraction_ok': False, '_error': str(e)})
return results
invoices = ['Invoice from Acme, March 2025, $500', 'Invoice from Beta Corp, April 2025, $1200']
results = extract_many(invoices, INVOICE_SCHEMA)
print(f'Processed: {len([r for r in results if r["_extraction_ok"]])} success, {len([r for r in results if not r["_extraction_ok"]])} failed')Schema Validation After Extraction
Validate extracted data against the expected schema using Python's jsonschema library or custom validators:
def validate_extracted(data, required_fields, type_checks):
errors = []
# Check required fields
for field in required_fields:
if field not in data or data[field] is None:
errors.append(f'Required field missing or null: {field}')
# Check types
for field, expected_type in type_checks.items():
if field in data and data[field] is not None:
if not isinstance(data[field], expected_type):
errors.append(f'{field}: expected {expected_type.__name__}, got {type(data[field]).__name__}')
return errors
extracted = {'invoice_number': 'INV-001', 'total_amount': 3255.0, 'vendor_name': 'Acme', 'invoice_date': '2025-03-15'}
required = ['invoice_number', 'total_amount', 'vendor_name']
types = {'total_amount': float, 'invoice_number': str, 'line_items': list}
errors = validate_extracted(extracted, required, types)
print('Validation errors:', errors)Iterative Schema Refinement
Schemas evolve through iterative testing. The process:
- Define initial schema based on domain knowledge
- Run extraction on 20 sample documents
- Review output — which fields are consistently wrong or missing?
- Refine the schema description and add field definitions
- Re-run on the same 20 documents
- Repeat until quality meets threshold
Adding Field Descriptions to the Schema
When a field is ambiguous, add a description comment to guide the model:
ANNOTATED_SCHEMA = '''
{
"invoice_number": "string // The unique identifier for this invoice, e.g., INV-2025-001",
"invoice_date": "YYYY-MM-DD // Date the invoice was issued",
"due_date": "YYYY-MM-DD or null // Payment due date; null if not specified",
"subtotal": "number // Amount before tax, as a decimal number",
"tax": "number or null // Tax amount as a decimal; null if tax is not listed",
"total_amount": "number // Final amount to pay, including tax",
"payment_terms": "string or null // e.g., Net 30, Due on receipt; null if not mentioned"
}
'''
print('Annotated schema adds context per field.')
print(f'Schema length: {len(ANNOTATED_SCHEMA)} chars')Confidence Scores for Extracted Fields
For production systems, include a confidence score per field. Low-confidence extractions can be routed to human review:
SCHEMA_WITH_CONFIDENCE = '''
{
"fields": {
"invoice_number": {"value": "string", "confidence": "high|medium|low"},
"total_amount": {"value": "number", "confidence": "high|medium|low"},
"due_date": {"value": "YYYY-MM-DD or null", "confidence": "high|medium|low"}
},
"overall_confidence": "high|medium|low",
"extraction_notes": "string or null // Any ambiguities encountered"
}
'''
prompt = f'Extract invoice data with confidence scores.\nReturn JSON:\n{SCHEMA_WITH_CONFIDENCE}\n\nInvoice: Payment due within 30 days. Total is approximately $500.'
r = client.messages.create(model='claude-opus-4-5', max_tokens=300, messages=[{'role': 'user', 'content': prompt}])
result = json.loads(r.content[0].text)
print('Overall confidence:', result.get('overall_confidence'))
print('Notes:', result.get('extraction_notes'))Quick Check
When a required field is not present in the source document, what should a schema-driven extraction prompt instruct the model to return for that field?
Schema-Driven Extraction — Key Takeaways
Schema-driven extraction is the standard for reliable document processing in production:
- Provide the exact JSON schema in the prompt — the model uses it as an output contract
- Add field descriptions for ambiguous fields to guide model interpretation
- Always instruct: missing fields return null, never omit them
- Apply the same schema across many documents for consistent database-ready output
- Include confidence scores per field to enable human review routing
- Validate extracted data programmatically after every extraction
- Refine schemas iteratively: extract 20 samples, review, improve, repeat
Frequently asked questions
Is the “Schema-Driven Data Extraction” lesson free?
Yes — the full text of “Schema-Driven Data Extraction” is free to read here on the web, and the AI Prompt Engineering course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Prompt Engineering course, upgrade to CoddyKit PRO.
What will I learn in “Schema-Driven Data Extraction”?
Providing JSON schemas in prompts to guarantee structured output format. You practise AI Prompt Engineering with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start AI Prompt Engineering?
No prior experience is required. AI Prompt Engineering on CoddyKit is structured for beginners through advanced learners; this is — lesson 2 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Schema-Driven Data Extraction” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this AI Prompt Engineering lesson?
Yes. Every AI Prompt Engineering lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Named Entity Extraction Prompts
- Schema-Driven Data Extraction
- LLM as Text Classifier
- Confidence and Uncertainty in Classification