ROC, AUC, and Thresholds
Judging ranking quality across cutoffs.
ROC, AUC, and Thresholds is a free Data Science Academy lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Data Science Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Models Output Probabilities
Most classifiers do not just say yes or no. They give a probability, like 0.73, and you decide where to draw the line. 🎚️
probs = model.predict_proba(X)[:, 1]The Decision Threshold
A threshold turns a probability into a label. Above it means positive, below means negative. The default 0.5 is just one choice.
y_pred = probs >= 0.5Moving the Line
Lower the threshold and you catch more positives but raise false alarms. Raise it and you are stricter but miss more. Each setting trades off.
Two Rates to Track
As the threshold slides, watch the true positive rate against the false positive rate. Together they describe the model at every cutoff.
The ROC Curve
The ROC curve plots true positive rate versus false positive rate across all thresholds, showing the full trade-off in one picture.
from sklearn.metrics import roc_curve
fpr, tpr, thr = roc_curve(y_true, probs)Reading the Shape
A curve that hugs the top-left corner is excellent: high catch rate with few false alarms. A diagonal line means random guessing.
Area Under the Curve
AUC squeezes the whole ROC curve into one number: the area beneath it. It summarizes performance across every threshold at once.
from sklearn.metrics import roc_auc_score
roc_auc_score(y_true, probs)What AUC Means
AUC of 1.0 is perfect, 0.5 is coin-flip random. Read it as the chance the model ranks a true positive above a true negative.
Threshold-Free Comparison
Because AUC ignores any single cutoff, it is great for comparing models. The winner ranks positives ahead of negatives more often.
Then Pick a Threshold
AUC picks the model, but you still set the threshold for real use, tuning it to favor recall or precision as the problem demands.
When to Prefer PR Curves
On very imbalanced data, ROC can look rosy. A precision-recall curve often tells the harder truth about rare-class performance.
Quick Check
Let's confirm what AUC really tells you about a classifier.
Recap
The threshold turns probabilities into labels, the ROC curve shows every trade-off, and AUC scores the whole curve at once. 🎯
Frequently asked questions
Is the “ROC, AUC, and Thresholds” lesson free?
Yes — the full text of “ROC, AUC, and Thresholds” is free to read here on the web, and the Data Science Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Data Science Academy course, upgrade to CoddyKit PRO.
What will I learn in “ROC, AUC, and Thresholds”?
Judging ranking quality across cutoffs. You practise Data Science Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Data Science Academy?
No prior experience is required. Data Science Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “ROC, AUC, and Thresholds” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Data Science Academy lesson?
Yes. Every Data Science Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Regression Metrics: MAE, MSE, R2
- The Confusion Matrix Decoded
- Precision, Recall, and F1
- ROC, AUC, and Thresholds