Why Single Words Lose Meaning
How not bad differs from bad.
Words Standing Alone
So far you have treated text as a bag of single words. But meaning often lives in how words sit next to each other, not in any word alone.
The Bag-of-Words Blind Spot
A bag-of-words model throws away order. To it, the two sentences below are identical, even though they mean opposite things. 😮
a = sorted("this is not good".split())
b = sorted("good is not this".split())
print(a == b) # TrueAll lessons in this course
- Why Single Words Lose Meaning
- Bigrams and Trigrams Explained
- N-Gram Features in scikit-learn
- Choosing the Right N-Gram Range