0PricingLogin▼
Deep Learning Academy · Lesson

nn.Embedding: Learnable Word Vectors

A lookup table trained by gradient descent.

Ids Alone Are Not Enough

Token ids are just labels. The id 5 is not greater than 2 in any meaningful way, so feeding raw ids to a net misleads it.

One-Hot Is Wasteful

One way to represent ids is a giant one-hot vector, but with thousands of words it is huge, sparse, and carries no similarity.

All lessons in this course

  1. Tokenize and Build a Vocabulary
  2. nn.Embedding: Learnable Word Vectors
  3. Why Embeddings Capture Meaning
  4. Train a Text Classifier
← Back to Deep Learning Academy