ROC Curves and AUC-ROC
Learners will plot the ROC curve by varying the decision threshold, compute AUC, and interpret it as the probability that a random positive outranks a random negative.
Limitations of Fixed-Threshold Metrics
Precision, recall, and F1-score all depend on a fixed decision threshold (typically 0.5 for binary classifiers). But the optimal threshold varies by application: a spam filter might use 0.9 (strict, to avoid false positives), while a fraud detector might use 0.1 (lenient, to avoid missing fraud). Evaluating a model at only one threshold gives an incomplete picture of its capability. The ROC curve (Receiver Operating Characteristic) solves this by showing performance across all possible thresholds simultaneously, letting you understand the full range of precision-recall trade-offs the model can achieve.
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.metrics import precision_score, recall_score
X, y = load_breast_cancer(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=42)
clf = LogisticRegression(max_iter=10000)
clf.fit(X_tr, y_tr)
proba = clf.predict_proba(X_te)[:, 1]
for t in [0.1, 0.3, 0.5, 0.7, 0.9]:
y_pred = (proba >= t).astype(int)
print(f'thresh={t}: prec={precision_score(y_te,y_pred,zero_division=0):.3f}, '
f'rec={recall_score(y_te,y_pred,zero_division=0):.3f}')True Positive Rate and False Positive Rate
The ROC curve is built from two metrics computed at each threshold: True Positive Rate (TPR) (same as recall/sensitivity): TPR = TP / (TP + FN) — what fraction of actual positives did we catch? False Positive Rate (FPR): FPR = FP / (FP + TN) — what fraction of actual negatives did we incorrectly flag? As the threshold decreases, both TPR and FPR increase. The ROC curve plots TPR vs FPR as the threshold varies from 1 (predict nothing positive) to 0 (predict everything positive). A perfect classifier reaches (0, 1) — zero false alarms, all true positives found.
import numpy as np
from sklearn.metrics import confusion_matrix
def tpr_fpr(y_true, y_pred):
TN, FP, FN, TP = confusion_matrix(y_true, y_pred).ravel()
tpr = TP / (TP + FN) # Recall = sensitivity
fpr = FP / (FP + TN) # False positive rate = 1 - specificity
return tpr, fpr
# Example predictions at different thresholds
y_true = np.array([1,1,1,0,0,0,0,0,0,0])
for thresh in [0.3, 0.5, 0.7]:
# Simulated probabilities
proba = np.array([0.9, 0.8, 0.4, 0.6, 0.5, 0.3, 0.2, 0.1, 0.05, 0.01])
y_pred = (proba >= thresh).astype(int)
tpr, fpr = tpr_fpr(y_true, y_pred)
print(f'thresh={thresh}: TPR={tpr:.2f}, FPR={fpr:.2f}')