0Pricing
Machine Learning Academy · Ders

ROC Eğrileri ve AUC-ROC

Karar eşiğini değiştirerek ROC eğrisini çizin, AUC'yi hesaplayın ve bunu rastgele bir pozitifin rastgele bir negatiften daha yüksek sıralanma olasılığı olarak yorumlayın.

ROC Eğrileri ve AUC-ROC, CoddyKit'te ücretsiz bir Machine Learning Academy dersidir. Bu, 4 dersinin 2. dersidir. Aşağıdan dersin tamamını ücretsiz okuyabilir, sonra tarayıcıda yerleşik kod editörü ve 7/24 yapay zeka koçu ile uygulamalı olarak pratik yapabilirsin. Bu, Machine Learning Academy öğrenme yolunun bir parçasıdır ve ilerlemeniz web ve CoddyKit uygulaması arasında senkronize olur. Machine Learning Academy kursu toplamda 4 dersten oluşur.

Bu dersin bazı bölümleri henüz çevrilmemiş olup İngilizce olarak gösterilmektedir.

Limitations of Fixed-Threshold Metrics

Precision, recall, and F1-score all depend on a fixed decision threshold (typically 0.5 for binary classifiers). But the optimal threshold varies by application: a spam filter might use 0.9 (strict, to avoid false positives), while a fraud detector might use 0.1 (lenient, to avoid missing fraud). Evaluating a model at only one threshold gives an incomplete picture of its capability. The ROC curve (Receiver Operating Characteristic) solves this by showing performance across all possible thresholds simultaneously, letting you understand the full range of precision-recall trade-offs the model can achieve.

from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.metrics import precision_score, recall_score

X, y = load_breast_cancer(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=42)

clf = LogisticRegression(max_iter=10000)
clf.fit(X_tr, y_tr)
proba = clf.predict_proba(X_te)[:, 1]

for t in [0.1, 0.3, 0.5, 0.7, 0.9]:
    y_pred = (proba >= t).astype(int)
    print(f'thresh={t}: prec={precision_score(y_te,y_pred,zero_division=0):.3f}, '
          f'rec={recall_score(y_te,y_pred,zero_division=0):.3f}')

True Positive Rate and False Positive Rate

The ROC curve is built from two metrics computed at each threshold: True Positive Rate (TPR) (same as recall/sensitivity): TPR = TP / (TP + FN) — what fraction of actual positives did we catch? False Positive Rate (FPR): FPR = FP / (FP + TN) — what fraction of actual negatives did we incorrectly flag? As the threshold decreases, both TPR and FPR increase. The ROC curve plots TPR vs FPR as the threshold varies from 1 (predict nothing positive) to 0 (predict everything positive). A perfect classifier reaches (0, 1) — zero false alarms, all true positives found.

import numpy as np
from sklearn.metrics import confusion_matrix

def tpr_fpr(y_true, y_pred):
    TN, FP, FN, TP = confusion_matrix(y_true, y_pred).ravel()
    tpr = TP / (TP + FN)   # Recall = sensitivity
    fpr = FP / (FP + TN)   # False positive rate = 1 - specificity
    return tpr, fpr

# Example predictions at different thresholds
y_true = np.array([1,1,1,0,0,0,0,0,0,0])

for thresh in [0.3, 0.5, 0.7]:
    # Simulated probabilities
    proba = np.array([0.9, 0.8, 0.4, 0.6, 0.5, 0.3, 0.2, 0.1, 0.05, 0.01])
    y_pred = (proba >= thresh).astype(int)
    tpr, fpr = tpr_fpr(y_true, y_pred)
    print(f'thresh={thresh}: TPR={tpr:.2f}, FPR={fpr:.2f}')

Plotting the ROC Curve

Scikit-learn's roc_curve() computes the (FPR, TPR) pairs across all thresholds in a single call. RocCurveDisplay creates a publication-ready plot. The ROC curve for a random classifier is the diagonal line from (0,0) to (1,1) — it achieves no better than chance at any threshold. A good classifier bows toward the upper-left corner, achieving high TPR with low FPR. The further the curve is from the diagonal, the better the classifier. The area under this curve (AUC) summarises performance in a single number.

from sklearn.metrics import roc_curve, RocCurveDisplay
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt

X, y = load_breast_cancer(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=42)

clf = LogisticRegression(max_iter=10000)
clf.fit(X_tr, y_tr)

RocCurveDisplay.from_estimator(clf, X_te, y_te)
plt.plot([0,1],[0,1],'k--', label='Random classifier')
plt.title('ROC Curve')
plt.legend()
plt.show()

AUC-ROC: Area Under the Curve

AUC-ROC (Area Under the ROC Curve) summarises the ROC curve into a single number between 0 and 1. AUC = 0.5: no better than random (diagonal line). AUC = 1.0: perfect classifier (no false positives at any recall level). AUC = 0.0: perfect inverse classifier — every prediction is wrong (useful if you flip predictions). An AUC of 0.8 means that 80% of the time, a randomly selected positive sample receives a higher predicted probability than a randomly selected negative sample. AUC is threshold-independent — it evaluates the ranking quality of the probability scores, not a specific prediction decision.

from sklearn.metrics import roc_auc_score
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

X, y = load_breast_cancer(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=42)

clf = LogisticRegression(max_iter=10000)
clf.fit(X_tr, y_tr)
proba = clf.predict_proba(X_te)[:, 1]

auc = roc_auc_score(y_te, proba)
print(f'AUC-ROC: {auc:.4f}')

# Interpretation: AUC = P(score(pos) > score(neg))
import numpy as np
pos_scores = proba[y_te == 1]
neg_scores = proba[y_te == 0]
mc_auc = np.mean([p > n for p in pos_scores for n in neg_scores])
print(f'Monte Carlo AUC: {mc_auc:.4f}')  # Should match roc_auc_score

Interpreting AUC Values in Practice

Guidelines for interpreting AUC: 0.5 = random classifier, useless; 0.6-0.7 = poor, but better than chance; 0.7-0.8 = fair — acceptable for some applications; 0.8-0.9 = good — solid production model; 0.9-1.0 = excellent — likely near state-of-the-art for the problem. Beware of AUC inflation: temporal data leakage (using future information), label leakage, or highly imbalanced classes can artificially inflate AUC. Always sanity-check with calibration curves and by inspecting predictions on specific examples.

# AUC interpretation guide
auc_guide = [
    (0.50, 0.60, 'Random to poor'),
    (0.60, 0.70, 'Poor but predictive'),
    (0.70, 0.80, 'Fair'),
    (0.80, 0.90, 'Good'),
    (0.90, 1.00, 'Excellent'),
]

for low, high, label in auc_guide:
    print(f'AUC [{low:.2f}, {high:.2f}): {label}')

# Note: domain matters -- AUC 0.75 might be excellent
# for predicting disease 10 years in advance but poor
# for predicting email spam from clear signals

Comparing Multiple Models on the ROC Curve

Overlaying multiple ROC curves on the same plot enables direct model comparison. Each model's curve and AUC value appear together. A model is better if its ROC curve is uniformly above another's at every FPR. If curves cross, neither model dominates — one is better at low false positive rates, the other at high false positive rates, and the business context determines which is preferable. AUC comparison is a quick summary, but always check curves at the operating point you actually intend to use.

from sklearn.metrics import roc_curve, roc_auc_score
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt

models = [
    ('Logistic Reg', LogisticRegression(max_iter=1000)),
    ('Decision Tree', DecisionTreeClassifier(max_depth=5)),
    ('KNN', KNeighborsClassifier(n_neighbors=5)),
]

for name, model in models:
    model.fit(StandardScaler().fit_transform(X_tr) if name!='Decision Tree' else X_tr, y_tr)
    proba = model.predict_proba(X_te)[:, 1]
    fpr, tpr, _ = roc_curve(y_te, proba)
    auc = roc_auc_score(y_te, proba)
    plt.plot(fpr, tpr, label=f'{name} (AUC={auc:.3f})')

plt.plot([0,1],[0,1],'k--')
plt.legend()
plt.show()

The Precision-Recall Curve: Better for Imbalanced Data

For highly imbalanced datasets, the ROC curve can be misleadingly optimistic. When there are very few positives (e.g., 1% fraud rate), even a model with many false positives achieves a low FPR because the denominator (TN+FP) is huge. The Precision-Recall curve plots precision vs recall at different thresholds and is not affected by class imbalance. The area under the PR curve (Average Precision, AP) is a better summary metric for imbalanced problems. Use ROC for balanced classes; use Precision-Recall curve for imbalanced classes.

from sklearn.metrics import PrecisionRecallDisplay, average_precision_score
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt

X, y = load_breast_cancer(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=42)

clf = LogisticRegression(max_iter=10000)
clf.fit(X_tr, y_tr)
proba = clf.predict_proba(X_te)[:, 1]

ap = average_precision_score(y_te, proba)
print(f'Average Precision (AP): {ap:.4f}')

PrecisionRecallDisplay.from_estimator(clf, X_te, y_te)
plt.title(f'Precision-Recall Curve (AP={ap:.3f})')
plt.show()

AUC in Cross-Validation

To get a reliable AUC estimate that is not biased by a single train-test split, use cross_val_score with scoring='roc_auc'. This computes AUC on each fold's validation set and returns all fold scores. The mean is the best estimate; the standard deviation shows stability. High mean AUC with low std indicates a robust model. High AUC with high std suggests the model is sensitive to which samples appear in each fold — consider whether the data has natural groupings (e.g., multiple samples per patient) that should be kept together in folds.

from sklearn.model_selection import cross_val_score
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline

X, y = load_breast_cancer(return_X_y=True)

pipe = Pipeline([
    ('sc', StandardScaler()),
    ('clf', LogisticRegression(max_iter=1000))
])

aucs = cross_val_score(pipe, X, y, cv=10, scoring='roc_auc')
print(f'AUC per fold: {aucs.round(3)}')
print(f'Mean AUC: {aucs.mean():.4f}')
print(f'Std AUC:  {aucs.std():.4f}')

Setting an Operating Point on the ROC Curve

After evaluating a model with the ROC curve, you must choose a specific operating point (a threshold) for production use. Three strategies: (1) Youden's J statistic: choose the threshold that maximises (TPR - FPR) — the point farthest above the diagonal. (2) Business constraint: set maximum allowable FPR (e.g., flag at most 5% of legitimate transactions as fraud) and find the corresponding threshold. (3) Cost optimisation: if you know the cost of FP and FN, choose the threshold that minimises total cost. Document the chosen threshold along with the model for reproducible deployment.

from sklearn.metrics import roc_curve
import numpy as np

fpr, tpr, thresholds = roc_curve(y_te, proba)

# Strategy 1: Youden's J (max TPR - FPR)
youden_j = tpr - fpr
best_idx = np.argmax(youden_j)
best_threshold = thresholds[best_idx]
print(f'Youden threshold: {best_threshold:.3f}')
print(f'TPR={tpr[best_idx]:.3f}, FPR={fpr[best_idx]:.3f}')

# Strategy 2: Max TPR with FPR <= 0.05
allowed_fpr = 0.05
mask = fpr <= allowed_fpr
idx = np.where(mask)[0][-1]  # Highest TPR satisfying FPR constraint
print(f'\nAt FPR<={allowed_fpr}: TPR={tpr[idx]:.3f}, threshold={thresholds[idx]:.3f}')

Multi-Class ROC-AUC

For multi-class classification, ROC-AUC is extended by computing AUC for each class in a One-vs-Rest (OvR) setup: treat class k as positive and all others as negative, compute AUC, then average across classes. Scikit-learn's roc_auc_score supports this with multi_class='ovr' and average='macro' or average='weighted'. This requires probability estimates from predict_proba(), so the classifier must support probability output. Macro AUC treats all classes equally; weighted AUC accounts for class frequency.

from sklearn.metrics import roc_auc_score
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

X, y = load_iris(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=42)

clf = RandomForestClassifier(n_estimators=100, random_state=42)
clf.fit(X_tr, y_tr)
proba_all = clf.predict_proba(X_te)  # Shape: (n_samples, n_classes)

auc_macro    = roc_auc_score(y_te, proba_all, multi_class='ovr', average='macro')
auc_weighted = roc_auc_score(y_te, proba_all, multi_class='ovr', average='weighted')

print(f'Macro AUC (OvR):    {auc_macro:.4f}')
print(f'Weighted AUC (OvR): {auc_weighted:.4f}')

AUC for Imbalanced Datasets: A Warning

AUC-ROC can be misleadingly optimistic for severely imbalanced datasets. When only 0.1% of transactions are fraud, even a model that captures 80% of fraud cases while flagging 5% of legitimate ones achieves an AUC near 0.9 — but in absolute terms, flagging 5% of millions of legitimate transactions creates enormous operational cost. In such cases, AUC-PR (area under the Precision-Recall curve) is more informative because it focuses only on the positive class performance and is not inflated by the large number of true negatives. For extreme imbalance (<1% positive rate), always report AUC-PR alongside AUC-ROC.

from sklearn.metrics import roc_auc_score, average_precision_score
from sklearn.linear_model import LogisticRegression
import numpy as np

np.random.seed(42)
# Extremely imbalanced: 1% positive rate
n_neg, n_pos = 9900, 100
X = np.vstack([np.random.randn(n_neg, 5),
               np.random.randn(n_pos, 5) + 1.5])
y = np.array([0]*n_neg + [1]*n_pos)

clf = LogisticRegression(class_weight='balanced', max_iter=1000)
clf.fit(X, y)
proba = clf.predict_proba(X)[:, 1]

auc_roc = roc_auc_score(y, proba)
auc_pr  = average_precision_score(y, proba)

print(f'AUC-ROC: {auc_roc:.4f} (looks great!)')
print(f'AUC-PR:  {auc_pr:.4f} (more honest about minority class)')

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: the ROC curve plots TPR vs FPR across all thresholds and shows the full range of precision-recall trade-offs, AUC-ROC summarises ranking quality in one number where 0.5 = random and 1.0 = perfect, and for imbalanced classes use the Precision-Recall curve and Average Precision instead. Next up we explore regression metrics including MAE, MSE, RMSE, and R-squared.

Sıkça Sorulan Sorular

“ROC Eğrileri ve AUC-ROC” dersi ücretsiz mi?

Evet — “ROC Eğrileri ve AUC-ROC” dersin tüm metni burada web'de ücretsiz olarak okunabilir. Etkileşimli olarak pratik yapmak (yerleşik kod editörü ve 7/24 yapay zeka koçu) ve Machine Learning Academy kursunun geri kalanını açmak için CoddyKit PRO'ya yükselt. Machine Learning Academy kursu toplamda 4 dersten oluşur.

“ROC Eğrileri ve AUC-ROC” dersinde ne öğreneceğim?

Karar eşiğini değiştirerek ROC eğrisini çizin, AUC'yi hesaplayın ve bunu rastgele bir pozitifin rastgele bir negatiften daha yüksek sıralanma olasılığı olarak yorumlayın. Machine Learning Academy ile uygulamalı kodu tarayıcıda doğrudan çalıştırarak pratik yaparsın ve 7/24 yapay zeka koçu dersi çalışırken sorularını yanıtlar.

Machine Learning Academy öğrenmeye başlamak için deneyim gerekli mi?

Önceden deneyim gerekmez. CoddyKit'te Machine Learning Academy, başlangıçtan ileri seviyeye kadar yapılandırıldığı için buradan başlayabilir veya başından başlayıp kendi hızında ilerleme yapabilirsin. Bu, 4 dersinin 2. dersidir.

“ROC Eğrileri ve AUC-ROC” dersi ne kadar sürer?

Çoğu CoddyKit dersi yaklaşık 5–10 dakika sürer. Her biri kısa ve etkileşimli olduğu için sabit ilerleme yaparsın ve web ile uygulama arasında tam olarak bıraktığın yerden devam edebilirsin.

Bu Machine Learning Academy dersinde kod yazıp çalıştırabilir miyim?

Evet. Her Machine Learning Academy dersi yerleşik bir kod editörü içerir, bu sayede tarayıcıda gerçek kod yazıp çalıştırabilir ve anlık yapay zeka geri bildirimi alırsın — yerel kurulum gerekli değildir.

Bu kursun tüm dersleri

  1. Sınıflandırma Metrikleri: Doğruluk, Kesinlik, Duyarlılık ve F1
  2. ROC Eğrileri ve AUC-ROC
  3. Regresyon Metrikleri: MAE, MSE, RMSE ve R-Kare
  4. İş Probleminiz için Doğru Metriği Seçme
← Machine Learning Academy Sayfasına Dön