What Embeddings Are (Vector Representations)
An embedding is a list of numbers that represents the meaning of text — similar texts have similar vectors.
What Embeddings Are (Vector Representations) is a free AI Agents lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
From Text to Numbers
An embedding is a list of numbers (a vector) that represents the meaning of a piece of text. Texts with similar meaning have vectors that are close together in vector space.
Why Vectors?
Computers cannot do math on words but they can on numbers. Embeddings give every text a location in a high-dimensional space so we can:
- Measure similarity between texts
- Cluster related items
- Search semantically (not just by keyword)
An Embedding Example
The OpenAI text-embedding-3-small model returns 1536 numbers per text. For "hello world", you get something like:
embedding = [0.0123, -0.0456, 0.0789, 0.0234, -0.0567, 0.0089, -0.0321]
# Real embeddings have ~1536 floats, each between roughly -1 and 1; this is a shortened example.
print("Example embedding (7 of ~1536 dimensions shown):", embedding)
Similar Texts, Similar Vectors
The pairs below get vectors close to each other:
- "king" and "queen"
- "happy" and "joyful"
- "car" and "automobile"
The pair "king" and "pizza" get vectors that are far apart.
Dimensionality
More dimensions = more nuance but more storage. Common sizes:
- OpenAI text-embedding-3-small: 1536
- OpenAI text-embedding-3-large: 3072
- Cohere embed-multilingual-v3: 1024
- BGE-small (OSS): 384
Semantic vs Lexical
Lexical search (Elasticsearch, BM25) matches keywords. Semantic search (embeddings) matches meaning.
"How to fix my car?" matches "auto repair tips" semantically — even though they share zero words.
Embeddings for Documents
You can embed:
- Short queries
- Sentences
- Paragraphs (~500 tokens is the sweet spot)
- Entire documents (with quality loss)
Embeddings for Code
There are code-specific embedding models (e.g. Voyage Code, Jina Code) that understand programming syntax and outperform general text embeddings on code search.
Multimodal Embeddings
You can embed images, audio, and video into the same space as text — enabling cross-modal search. CLIP is the famous example for images.
Bias in Embeddings
Embeddings reflect their training data. They encode bias: gender-occupation associations, racial stereotypes, etc. Be aware when using them for ranking decisions involving people.
What Embeddings Cannot Do
- Cannot solve math problems
- Cannot do logical inference
- Cannot generate text
- Cannot reason about cause and effect
They are a way to MEASURE similarity, not REASON.
Embeddings Definition
What is an embedding?
Recap
Embeddings turn text into vectors. Vectors enable semantic search, clustering, and RAG. Next we generate them with the OpenAI API.
Frequently asked questions
Is the “What Embeddings Are (Vector Representations)” lesson free?
Yes — the full text of “What Embeddings Are (Vector Representations)” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.
What will I learn in “What Embeddings Are (Vector Representations)”?
An embedding is a list of numbers that represents the meaning of text — similar texts have similar vectors. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start AI Agents?
No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “What Embeddings Are (Vector Representations)” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this AI Agents lesson?
Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- What Embeddings Are (Vector Representations)
- Generating Embeddings with text-embedding-3
- Cosine Similarity for Retrieval
- Embedding Models Compared (OpenAI vs Cohere vs OSS)