Beyond Bag-of-Words
Length, readability, and metadata signals.
Beyond Bag-of-Words is a free NLP Academy lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the NLP Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
The Baseline Wall
Bag-of-words and TF-IDF get you a solid first model. But at some point the score stops climbing, and you need richer features to push past it.
What Counts Get Wrong
Word counts ignore everything about a document except which words appear. Tone, length, and structure all carry signal that pure counts throw away.
Document Length as a Clue
How long a text is can predict its label. Spam is often short, while detailed reviews run long, so length becomes a useful feature.
text = "Buy now! Limited offer!"
word_count = len(text.split())Punctuation Tells a Story
Lots of exclamation marks or question marks hint at emotion or urgency. Counting punctuation turns that hidden cue into a number.
excls = text.count("!")Capitalization Patterns
SHOUTING in all caps often signals spam or anger. The ratio of uppercase letters is a tiny but surprisingly powerful feature.
Readability Scores
How hard a text is to read can separate audiences and styles. A readability score boils sentence and word complexity into one handy number.
Metadata Is Free Signal
Author, timestamp, and source often sit right next to your text. This metadata can predict labels without reading a single word.
Lexical Diversity
Unique words divided by total words measures how varied a text is. Repetitive writing scores low, and that diversity ratio can help your model.
tokens = text.lower().split()
diversity = len(set(tokens)) / len(tokens)Features Are Just Numbers
Every idea here ends as a number you can hand to a model. A good feature simply turns intuition about text into measurable values.
Domain Knowledge Wins
The best features come from knowing your problem. If you understand what separates the classes, you can design a feature that captures it. 💡
Start Small, Then Add
Begin with bag-of-words, then layer in a few hand-built features. Measure each one so you keep only the additions that truly help.
Quick Check
Why go beyond plain bag-of-words features?
Recap
Counts miss tone, length, and structure. Hand-built features like length, punctuation, and readability turn those cues into numbers that lift your model. ✅
Frequently asked questions
Is the “Beyond Bag-of-Words” lesson free?
Yes — the full text of “Beyond Bag-of-Words” is free to read here on the web, and the NLP Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the NLP Academy course, upgrade to CoddyKit PRO.
What will I learn in “Beyond Bag-of-Words”?
Length, readability, and metadata signals. You practise NLP Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start NLP Academy?
No prior experience is required. NLP Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Beyond Bag-of-Words” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this NLP Academy lesson?
Yes. Every NLP Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.