0Pricing
Learn AI with Python · Lesson

Word2Vec: Skip-gram and CBOW

Word2Vec theory, gensim implementation, vector arithmetic (king - man + woman = queen).

Word2Vec: Skip-gram and CBOW is a free Learn AI with Python lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Learn AI with Python learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

From Words to Vectors

Machine learning needs numbers, but words are symbols. Word embeddings map each word to a dense vector so that similar words sit close together in vector space.

The Distributional Hypothesis

Word2Vec rests on the idea that words appearing in similar contexts have similar meanings. By learning to predict context, the model captures meaning in the vectors.

Two Architectures

Word2Vec has two training schemes:

  • CBOW predicts the target word from its surrounding context
  • Skip-gram predicts the surrounding context from the target word

CBOW vs Skip-gram

CBOW is faster and works well on frequent words. Skip-gram is slower but handles rare words and small datasets better. Skip-gram is often the default choice for quality.

Training with gensim

The gensim library makes Word2Vec easy. You pass tokenized sentences (lists of word lists) and set the embedding configuration.

from gensim.models import Word2Vec

sentences = [["the", "cat", "sat"], ["the", "dog", "ran"]]
model = Word2Vec(sentences=sentences, vector_size=100, window=5, min_count=1)

Key Parameters

The main knobs are:

  • vector_size embedding dimension (e.g. 100-300)
  • window context width around each word
  • min_count ignore words rarer than this
from gensim.models import Word2Vec

model = Word2Vec(
    sentences,
    vector_size=200,
    window=5,
    min_count=5,
)

Choosing Skip-gram with sg=1

The sg parameter selects the architecture: sg=0 uses CBOW (default), sg=1 uses skip-gram.

from gensim.models import Word2Vec

model = Word2Vec(sentences, vector_size=100, window=5, min_count=1, sg=1)
# sg=1 -> skip-gram

Accessing Word Vectors

Trained vectors live in model.wv. Index it by a word to get that word vector.

vec = model.wv["cat"]
print(vec.shape)   # (vector_size,)
print("dog" in model.wv)

Finding Similar Words

wv.most_similar returns the words whose vectors are closest, a quick way to inspect what the embedding learned.

print(model.wv.most_similar("cat", topn=5))
# returns list of (word, similarity) pairs

Word Arithmetic

The famous property: vector math captures relationships. king - man + woman lands near queen, showing the embedding encodes analogies.

result = model.wv.most_similar(
    positive=["king", "woman"],
    negative=["man"],
    topn=1,
)
print(result)   # often [("queen", 0.7...)]

Using Embeddings Downstream

To represent a sentence, average its word vectors, then feed that into any classifier. Pretrained Word2Vec embeddings also give a strong starting point when your data is small.

import numpy as np

def sentence_vector(words, model):
    vecs = [model.wv[w] for w in words if w in model.wv]
    return np.mean(vecs, axis=0) if vecs else np.zeros(model.vector_size)

Quick Check

Test your Word2Vec knowledge.

Recap

Recap: Word2Vec learns dense word vectors from context. CBOW predicts a word from context (fast); skip-gram (sg=1) predicts context from a word (better for rare words). In gensim, set vector_size, window, min_count, then use wv.most_similar and analogies like king - man + woman.

Frequently asked questions

Is the “Word2Vec: Skip-gram and CBOW” lesson free?

Yes — the full text of “Word2Vec: Skip-gram and CBOW” is free to read here on the web, and the Learn AI with Python course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Learn AI with Python course, upgrade to CoddyKit PRO.

What will I learn in “Word2Vec: Skip-gram and CBOW”?

Word2Vec theory, gensim implementation, vector arithmetic (king - man + woman = queen). You practise Learn AI with Python with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start Learn AI with Python?

No prior experience is required. Learn AI with Python on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Word2Vec: Skip-gram and CBOW” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this Learn AI with Python lesson?

Yes. Every Learn AI with Python lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. Word2Vec: Skip-gram and CBOW
  2. GloVe and FastText Embeddings
  3. Text Classification with BERT
  4. Semantic Similarity and Sentence Embeddings
← Back to Learn AI with Python