Pourquoi le contexte modifie le sens d’un mot

The word bank can mean a river edge or a place for money. Its meaning depends entirely on the context around it.

word2vec and GloVe give each word one fixed vector. So bank gets a single static meaning, no matter the sentence it appears in.

These two sentences use bank in opposite ways, yet a static model hands them the very same vector for the word.

You instantly know which bank is meant because you read the surrounding words. Meaning lives in context, not in the word alone.

A contextual model gives bank a different vector in each sentence. The embedding now shifts to fit its context every time. ✨

BERT is a transformer that reads a whole sentence at once and produces context-aware vectors. It is built around attention over all the words.

BERT looks at words on the left and the right of each token together. That two-way view is what bidirectional means.

Feed both river and money sentences to BERT and bank comes out with two distinct vectors. The embedding tracks the sentence.

Words with many senses, called polysemy, used to confuse static models. Contextual embeddings finally separate those senses cleanly.

Search, translation, and question answering all improve when a model truly grasps which meaning a word carries right now.

Moving from one vector per word to one per usage was a turning point. This is the heart of modern NLP.

What makes contextual embeddings different from static ones?

Static vectors trap each word in one meaning. Contextual embeddings from BERT shift per sentence, capturing the right sense. ✅