Why Accuracy Can Lie
The trouble with a single number.
Why Accuracy Can Lie is a free NLP Academy lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the NLP Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Accuracy, Defined
Accuracy is just the share of predictions your model got right out of all predictions. Simple, but it can quietly mislead you. 🎯
The Single-Number Trap
One number feels reassuring, but accuracy hides which kinds of mistakes you make. A high score can mask a model that fails where it matters most.
Imbalanced Classes
The danger appears with imbalanced data, where one class is far more common than the other. Here accuracy stops telling the real story.
A Spam Example
Say 95% of emails are not spam. A lazy model that labels everything not spam scores 95% accuracy while catching zero spam.
The Lazy Baseline
That always-predict-the-majority trick is the majority-class baseline. If your model barely beats it, your accuracy is hollow.
Compute It in Python
scikit-learn gives accuracy directly, so you can compare a model against that lazy baseline in one line.
from sklearn.metrics import accuracy_score
acc = accuracy_score(y_true, y_pred)Errors Are Not Equal
Missing a fraud charge and flagging a safe one are very different costs. Accuracy treats every error as equally cheap, which it rarely is.
False Sense of Safety
A medical screen that misses real disease can score high accuracy yet be dangerous. The cost lives in the errors accuracy quietly averages away.
When Accuracy Is Fine
With balanced classes and equal error costs, accuracy is a perfectly honest, easy-to-read summary of performance.
Always Ask the Split
Before trusting any score, check the class balance. Knowing the class distribution tells you whether accuracy can be believed at all.
Look Beyond One Number
Accuracy is a starting point, not a verdict. Real evaluation needs metrics that reveal where and how your model fails.
Quick Check
Quick gut check on the spam example.
Recap
Accuracy counts correct predictions, but on imbalanced data it hides failures. Always check class balance and error costs first. ✅
Frequently asked questions
Is the “Why Accuracy Can Lie” lesson free?
Yes — the full text of “Why Accuracy Can Lie” is free to read here on the web, and the NLP Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the NLP Academy course, upgrade to CoddyKit PRO.
What will I learn in “Why Accuracy Can Lie”?
The trouble with a single number. You practise NLP Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start NLP Academy?
No prior experience is required. NLP Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Why Accuracy Can Lie” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this NLP Academy lesson?
Yes. Every NLP Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Why Accuracy Can Lie
- Precision, Recall, and F1
- Reading the Confusion Matrix
- Cross-Validation Done Right