什么是向量嵌入？
了解神经网络如何将文本映射到高维空间中的稠密向量，为什么语义相似的文本会产生相近的向量，以及余弦相似度所衡量的内容。
什么是向量嵌入？ 是 CoddyKit 上的免费 AI Engineering Academy 课时。 这是第 1 节课，共 4 节。 你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。 这是 AI Engineering Academy 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 AI Engineering Academy 课程共包含 4 节课。
本课时的部分内容尚未翻译，以英文显示。
The Meaning Behind Numbers
A vector embedding is a list of numbers (a vector) that represents the meaning of a piece of text. Instead of storing words as raw strings, neural networks learn to encode semantic meaning into dense numerical arrays. Two texts with similar meaning will produce vectors that are close together in this high-dimensional space.
High-Dimensional Vector Space
Modern embeddings typically have 768 to 3072 dimensions. Each dimension captures some latent feature of meaning — topics, sentiment, style, and relationships — without being explicitly programmed. The result is a rich geometric space where semantic relationships become measurable distances.
For example, text-embedding-3-small produces 1536-dimensional vectors, while text-embedding-3-large produces 3072-dimensional vectors.
How Neural Networks Learn Embeddings
Embedding models are trained on massive text corpora to predict missing words (masked language modeling) or to distinguish semantically similar from dissimilar pairs. During training, the network adjusts millions of weights until similar texts naturally cluster together in the learned vector space.
This is why embeddings are called dense representations — every dimension carries information, unlike sparse one-hot encodings where most values are zero.
Cosine Similarity: Measuring Closeness
Cosine similarity measures the angle between two vectors, returning a value between -1 and 1. A score of 1 means the vectors point in exactly the same direction (identical meaning), 0 means orthogonal (unrelated), and -1 means opposite directions.
It is preferred over Euclidean distance for text because it ignores vector magnitude and focuses purely on direction, making it robust to differences in text length.
import numpy as np
def cosine_similarity(vec_a, vec_b):
dot_product = np.dot(vec_a, vec_b)
norm_a = np.linalg.norm(vec_a)
norm_b = np.linalg.norm(vec_b)
return dot_product / (norm_a * norm_b)
# Example with tiny 3D vectors
v1 = np.array([0.8, 0.2, 0.5])
v2 = np.array([0.9, 0.1, 0.4])
print(cosine_similarity(v1, v2)) # Close to 1.0Why Semantic Similarity Matters
Traditional keyword search breaks when users paraphrase: searching for automobile misses documents about cars. Vector embeddings solve this because semantically equivalent texts map to nearby vectors regardless of the exact words used.
This enables semantic search, where you find the most relevant result based on meaning rather than word overlap — a fundamental capability needed for RAG systems.
The Embedding Vector Visualized
Imagine plotting sentences in a 2D map (a simplified analogy). Sentences about machine learning cluster in one region, sentences about cooking cluster in another, and sentences about sports form a third cluster. Vector embeddings create this map in thousands of dimensions.
Famous relationships like king - man + woman ≈ queen are real arithmetic in this space — vector operations encode semantic analogies.
Generating a Simple Embedding
The OpenAI embeddings API accepts text and returns a list of floats. A single API call can embed one sentence or an entire paragraph. The returned vector has a fixed dimension regardless of input length.
Important: longer inputs do not produce larger vectors — they still produce the same fixed-size vector, but very long texts may lose fine-grained detail because the model compresses everything into that fixed space.
from openai import OpenAI
client = OpenAI() # uses OPENAI_API_KEY from environment
response = client.embeddings.create(
model='text-embedding-3-small',
input='Vector embeddings capture semantic meaning.'
)
embedding = response.data[0].embedding
print(f'Dimensions: {len(embedding)}') # 1536
print(f'First 5 values: {embedding[:5]}')Embedding Multiple Texts at Once
You can embed multiple strings in a single API call by passing a list to the input parameter. This is much more efficient than making one request per text because it reduces network round-trips and lets the API batch the computation.
The response contains one embedding object per input, in the same order, so you can zip them together with your original texts.
from openai import OpenAI
client = OpenAI()
texts = [
'Python is a programming language.',
'Snakes are reptiles.',
'Machine learning requires data.'
]
response = client.embeddings.create(
model='text-embedding-3-small',
input=texts
)
for text, result in zip(texts, response.data):
print(f'{text[:30]}... -> {len(result.embedding)}D vector')Dot Product vs Cosine Similarity
If your embeddings are L2-normalized (unit vectors with magnitude 1), the dot product equals cosine similarity. OpenAI's embedding models return normalized vectors, so you can use either metric — dot product is slightly faster to compute.
However, when combining embeddings from different models or sources that may not be normalized, always compute cosine similarity explicitly to avoid misleading results.
import numpy as np
def normalize(vec):
return vec / np.linalg.norm(vec)
v1 = normalize(np.array([3.0, 4.0, 0.0]))
v2 = normalize(np.array([4.0, 3.0, 0.0]))
# For unit vectors: dot product == cosine similarity
dot = np.dot(v1, v2)
print(f'Dot product: {dot:.4f}') # same as cosine simEmbeddings vs One-Hot Encoding
One-hot encoding represents each word as a vector with exactly one 1 and all other values 0. A vocabulary of 50,000 words produces 50,000-dimensional vectors that are almost entirely zeros — very inefficient.
Dense embeddings use 768-3072 dimensions but every dimension carries real information. The space is 10-20x smaller yet captures far more nuance, including synonyms, analogies, and compositional meaning that one-hot encoding completely misses.
Common Use Cases for Embeddings
Beyond semantic search, embeddings power many practical applications:
- Semantic search — find documents by meaning, not keywords
- Recommendation systems — suggest items similar to ones the user liked
- Clustering — group documents by topic automatically
- Classification — feed embeddings to a linear classifier
- Deduplication — detect near-duplicate content
All RAG systems depend on embeddings to match user queries with relevant document chunks.
Quick Check
Test your understanding of AI Engineering concepts from this lesson.
Lesson Recap
In this lesson you learned: vector embeddings encode semantic meaning as dense numerical arrays, cosine similarity measures semantic closeness by comparing vector directions, and OpenAI's embedding API converts text to fixed-size vectors in a single call. Next up we explore how to generate and compare embeddings using the OpenAI models in practice.
常见问题解答
「什么是向量嵌入？」课时是免费的吗？
是的 — 「什么是向量嵌入？」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 AI Engineering Academy 课程的其余内容，请升级到 CoddyKit PRO。 AI Engineering Academy 课程共包含 4 节课。
「什么是向量嵌入？」这节课中我会学到什么？
了解神经网络如何将文本映射到高维空间中的稠密向量，为什么语义相似的文本会产生相近的向量，以及余弦相似度所衡量的内容。 你通过在浏览器中直接运行的动手代码来练习 AI Engineering Academy，全天候 AI 导师会在你学习这节课的过程中回答你的问题。
学习 AI Engineering Academy 需要有经验吗？
无需任何先前经验。CoddyKit 上的 AI Engineering Academy 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。 这是第 1 节课，共 4 节。
「什么是向量嵌入？」课时需要多长时间？
大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。
我能在这节 AI Engineering Academy 课中编写并运行代码吗？
能。每节 AI Engineering Academy 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。