Masked Language Modeling
How BERT learns from raw text.
Learning Without Labels
BERT trains on plain text with no human labels at all. This clever trick is called self-supervised learning.
The Fill-in-the-Blank Game
BERT learns by hiding some words and guessing them back. That game is masked language modeling, or MLM for short.
All lessons in this course
- Why Context Changes Word Meaning
- Masked Language Modeling
- Embedding Sentences With BERT
- Picking the Right Pre-trained Model