AI Engineering Academy · Lesson

Generating Embeddings with OpenAI

Use the text-embedding-3-small and text-embedding-3-large models to embed sentences, paragraphs, and documents, and compare their trade-offs in quality and cost.

Generating Embeddings with OpenAI is a free AI Engineering Academy lesson on CoddyKit — lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Engineering Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

OpenAI Embedding Models Overview

OpenAI offers two production embedding models: text-embedding-3-small and text-embedding-3-large. The small model produces 1536-dimensional vectors and is 5x cheaper per token, while the large model produces 3072-dimensional vectors with higher accuracy on benchmarks. For most RAG applications, the small model is the right starting point.

Making Your First Embedding Call

The client.embeddings.create() method takes a model name and an input string or list. It returns a response object with a data list, where each item has an embedding attribute containing the vector as a Python list of floats.

Always store your API key in an environment variable — never hardcode it in source files.

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ['OPENAI_API_KEY'])

response = client.embeddings.create(
    model='text-embedding-3-small',
    input='What is retrieval-augmented generation?'
)

vector = response.data[0].embedding
print(f'Vector length: {len(vector)}')      # 1536
print(f'Type: {type(vector[0])}')           # float
print(f'Sample values: {vector[:3]}')       # [-0.02, 0.01, ...]

Embedding Batches Efficiently

Passing a list of strings to input embeds them all in one API round-trip. This is the recommended approach when indexing a document corpus. The response data list preserves the original order, making it easy to build a lookup dictionary.

Batches are limited to 2048 items per request and the total tokens across all inputs must stay within the model's token limit.

from openai import OpenAI

client = OpenAI()

documents = [
    'RAG stands for Retrieval-Augmented Generation.',
    'Embeddings convert text to numerical vectors.',
    'Pinecone is a managed vector database service.'
]

response = client.embeddings.create(
    model='text-embedding-3-small',
    input=documents
)

embeddings = [item.embedding for item in response.data]
print(f'Got {len(embeddings)} embeddings')
print(f'Each has {len(embeddings[0])} dimensions')

Reducing Embedding Dimensions

Both text-embedding-3 models support a dimensions parameter that truncates the output vector. For example, setting dimensions=256 returns a 256-element vector instead of 1536. Shorter vectors use less storage and compute, with a modest accuracy tradeoff.

This is useful when you want to experiment quickly or when storage cost matters more than peak retrieval quality.

from openai import OpenAI

client = OpenAI()

response = client.embeddings.create(
    model='text-embedding-3-small',
    input='Shorter vectors save storage and query time.',
    dimensions=256   # truncate from 1536 to 256
)

print(len(response.data[0].embedding))  # 256

Encoding Format: Base64 vs Float List

By default the API returns embeddings as a JSON array of floats (encoding_format='float'). You can request encoding_format='base64' to receive a compact base64-encoded binary blob, which transfers faster over the network for large batches.

When using base64, you need to decode it with NumPy: np.frombuffer(base64.b64decode(b64_str), dtype='float32').

import base64
import numpy as np
from openai import OpenAI

client = OpenAI()

response = client.embeddings.create(
    model='text-embedding-3-small',
    input='Base64 encoding transfers faster.',
    encoding_format='base64'
)

b64 = response.data[0].embedding
vector = np.frombuffer(base64.b64decode(b64), dtype='float32')
print(f'Decoded {len(vector)} floats')

Tracking Token Usage and Cost

Every embedding response includes a usage object with prompt_tokens. You pay per token: text-embedding-3-small costs $0.02 per million tokens and text-embedding-3-large costs $0.13 per million tokens (as of mid-2024).

Tracking usage lets you estimate how much it costs to index your entire corpus before you commit to a production run.

from openai import OpenAI

client = OpenAI()

texts = ['Document one content here.', 'Document two content here.']

response = client.embeddings.create(
    model='text-embedding-3-small',
    input=texts
)

tokens_used = response.usage.prompt_tokens
cost_usd = tokens_used * 0.00000002  # $0.02 per 1M tokens
print(f'Tokens: {tokens_used}, Cost: ${cost_usd:.6f}')

Comparing text-embedding-3-small vs large

The choice between small and large depends on your accuracy requirements and budget:

text-embedding-3-small: 1536D, faster, 5x cheaper, great for most RAG workloads
text-embedding-3-large: 3072D, higher MTEB benchmark scores, better for multilingual content and complex semantic tasks

A common pattern is to prototype and validate retrieval quality with the small model, then switch to large only if evaluation metrics fall below your target.

Normalizing Embeddings for Dot Product Search

OpenAI returns L2-normalized embeddings, meaning each vector already has a magnitude of 1. This means you can use the dot product as a fast approximation of cosine similarity without an extra normalization step, which is important when searching millions of vectors where micro-optimizations add up.

import numpy as np
from openai import OpenAI

client = OpenAI()

response = client.embeddings.create(
    model='text-embedding-3-small',
    input='Are OpenAI embeddings normalized?'
)

vec = np.array(response.data[0].embedding)
magnitude = np.linalg.norm(vec)
print(f'Vector magnitude: {magnitude:.6f}')  # very close to 1.0

Handling Large Documents

The text-embedding-3-small model accepts up to 8191 tokens per input. If your document exceeds this limit, the API will return an error. The standard solution is to chunk the document first (typically 200-500 tokens per chunk) and embed each chunk separately.

This chunking is not just a technical requirement — it also produces better retrieval because embedding smaller focused pieces returns more precise results than embedding entire pages.

import tiktoken

enc = tiktoken.encoding_for_model('text-embedding-3-small')

def count_tokens(text):
    return len(enc.encode(text))

max_tokens = 8191
text = 'Very long document content...'  # pretend this is huge

if count_tokens(text) > max_tokens:
    print('Document too long — chunk before embedding')
else:
    print(f'Safe to embed: {count_tokens(text)} tokens')

Async Embedding for High Throughput

When indexing thousands of documents, use the async OpenAI client with asyncio.gather to send multiple embedding requests concurrently. This is much faster than sequential calls because the bottleneck is network latency, not CPU.

Always add rate limit handling with exponential backoff when running concurrent requests to avoid hitting the tokens-per-minute limit.

import asyncio
from openai import AsyncOpenAI

async def embed_batch(texts):
    client = AsyncOpenAI()
    response = await client.embeddings.create(
        model='text-embedding-3-small',
        input=texts
    )
    return [item.embedding for item in response.data]

async def main():
    batches = [['doc1', 'doc2'], ['doc3', 'doc4']]
    results = await asyncio.gather(*[embed_batch(b) for b in batches])
    all_embeddings = [emb for batch in results for emb in batch]
    print(f'Embedded {len(all_embeddings)} documents')

asyncio.run(main())

Caching Embeddings to Save Cost

Generating embeddings for the same text multiple times wastes money and time. Cache embeddings in a dictionary keyed by the text string (or a hash of it) and persist this cache to disk between sessions.

For production systems, store embeddings in a vector database that deduplicates by document ID. Only re-embed a document when its content actually changes, not on every indexing run.

import json, hashlib, os
from openai import OpenAI

CACHE_FILE = '/tmp/embedding_cache.json'
client = OpenAI()

try:
    cache = json.load(open(CACHE_FILE))
except FileNotFoundError:
    cache = {}

def get_embedding(text):
    key = hashlib.md5(text.encode()).hexdigest()
    if key not in cache:
        r = client.embeddings.create(model='text-embedding-3-small', input=text)
        cache[key] = r.data[0].embedding
        json.dump(cache, open(CACHE_FILE, 'w'))
    return cache[key]

vec = get_embedding('Caching saves API costs.')
print(f'Retrieved {len(vec)}-dim embedding')

Quick Check

Test your understanding of AI Engineering concepts from this lesson.

Lesson Recap

In this lesson you learned: text-embedding-3-small is the cost-effective choice for most RAG workloads, batch embedding multiple texts in one API call is more efficient than sequential calls, and the dimensions parameter can reduce vector size to trade accuracy for storage savings. Next up we build a semantic search system using these embeddings and NumPy.

Frequently asked questions

Is the “Generating Embeddings with OpenAI” lesson free?

Yes — the full text of “Generating Embeddings with OpenAI” is free to read here on the web, and the AI Engineering Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Engineering Academy course, upgrade to CoddyKit PRO.

What will I learn in “Generating Embeddings with OpenAI”?

Use the text-embedding-3-small and text-embedding-3-large models to embed sentences, paragraphs, and documents, and compare their trade-offs in quality and cost. You practise AI Engineering Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Engineering Academy?

No prior experience is required. AI Engineering Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 2 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Generating Embeddings with OpenAI” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Engineering Academy lesson?

Yes. Every AI Engineering Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to AI Engineering Academy