Machine Learning Academy · Ders

Aşırı Uydurmayı Önlemek için Ağaç Derinliğini Denetleme

Farklı max_depth değerleriyle ağaçlar eğitin, aşırı ve yetersiz uydurma ödünleşimini gözlemleyin ve derinliği doğrulama puanıyla seçin.

Aşırı Uydurmayı Önlemek için Ağaç Derinliğini Denetleme, CoddyKit'te ücretsiz bir Machine Learning Academy dersidir. Bu, 4 dersinin 3. dersidir. Aşağıdan dersin tamamını ücretsiz okuyabilir, sonra tarayıcıda yerleşik kod editörü ve 7/24 yapay zeka koçu ile uygulamalı olarak pratik yapabilirsin. Bu, Machine Learning Academy öğrenme yolunun bir parçasıdır ve ilerlemeniz web ve CoddyKit uygulaması arasında senkronize olur. Machine Learning Academy kursu toplamda 4 dersten oluşur.

Bu dersin bazı bölümleri henüz çevrilmemiş olup İngilizce olarak gösterilmektedir.

How Depth Leads to Overfitting

An unconstrained decision tree will grow until every training sample has its own leaf — achieving 100% training accuracy by memorising all data points, including noise. This is the extreme case of overfitting: the tree learns the idiosyncrasies of training data rather than general patterns. On any new data, such a tree performs poorly because its rules are too specific. Controlling tree depth is the primary regularisation mechanism for decision trees, analogous to choosing alpha in regularised linear models or choosing k in KNN.

from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

X, y = load_breast_cancer(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, random_state=42)

# Unlimited depth: memorises training data
tree = DecisionTreeClassifier(random_state=42)  # no max_depth
tree.fit(X_tr, y_tr)
print('Unlimited depth tree:')
print('  Tree depth:', tree.get_depth())
print('  Train accuracy:', tree.score(X_tr, y_tr).round(3))  # 1.000
print('  Test  accuracy:', tree.score(X_te, y_te).round(3))  # < 1.000

The max_depth Parameter

max_depth limits how many levels the tree can grow. With max_depth=1, the tree makes exactly one decision (a 'stump'). With max_depth=3, the tree can make up to three sequential questions. Shallower trees generalise better but may underfit; deeper trees fit training data better but risk overfitting. The right max_depth is a hyperparameter found through cross-validation. A useful heuristic: start around max_depth=3-5 and tune from there using a validation curve.

from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

X, y = load_breast_cancer(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, random_state=42)

for depth in [1, 2, 3, 5, 10, None]:
    tree = DecisionTreeClassifier(max_depth=depth, random_state=42)
    tree.fit(X_tr, y_tr)
    print(f'max_depth={str(depth):4}: train={tree.score(X_tr,y_tr):.3f}, '
          f'test={tree.score(X_te,y_te):.3f}')

Validation Curve for max_depth

Plotting training and validation accuracy vs max_depth reveals the classic bias-variance pattern. At depth 1, both training and validation accuracy are low (underfitting — high bias). As depth increases, training accuracy rises quickly to 100% while validation accuracy peaks and then decreases (overfitting — high variance). The optimal depth is where the validation curve peaks — before the train-test gap widens. Use cross-validation rather than a single validation split for a more reliable estimate of the peak location.

from sklearn.model_selection import validation_curve
from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import load_breast_cancer
import numpy as np
import matplotlib.pyplot as plt

X, y = load_breast_cancer(return_X_y=True)

train_sc, val_sc = validation_curve(
    DecisionTreeClassifier(random_state=42),
    X, y, param_name='max_depth',
    param_range=range(1, 16), cv=10
)

plt.plot(range(1,16), train_sc.mean(axis=1), label='Train')
plt.plot(range(1,16), val_sc.mean(axis=1), label='Validation')
plt.xlabel('max_depth'); plt.ylabel('Accuracy')
plt.title('Bias-Variance via max_depth')
plt.legend(); plt.show()

best_depth = np.argmax(val_sc.mean(axis=1)) + 1
print('Best depth:', best_depth)

min_samples_split: Minimum Samples to Split a Node

min_samples_split prevents splitting a node if it contains fewer than a specified number of samples. The default is 2 (any node with 2 or more samples can be split). Increasing this value forces the tree to wait for more evidence before making a decision, which prevents highly specific splits on tiny groups that are likely noise. Setting min_samples_split=20 means no node with fewer than 20 samples will be split further. This is especially useful for datasets with many samples and many rare subgroups.

from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import cross_val_score
from sklearn.datasets import load_breast_cancer

X, y = load_breast_cancer(return_X_y=True)

for min_split in [2, 5, 10, 20, 50]:
    tree = DecisionTreeClassifier(
        min_samples_split=min_split,
        random_state=42
    )
    score = cross_val_score(tree, X, y, cv=10).mean()
    tree.fit(X, y)
    print(f'min_samples_split={min_split:3}: depth={tree.get_depth()}, CV acc={score:.3f}')

min_samples_leaf: Minimum Samples in a Leaf

min_samples_leaf requires that after any split, each resulting child node must have at least this many samples. If a candidate split would create a child with too few samples, that split is rejected. This is a stronger constraint than min_samples_split because it guarantees a minimum number of samples in every leaf. Larger min_samples_leaf produces smaller, less deep trees with smoother decision boundaries. For probability estimation, this parameter is critical: leaf nodes with very few samples produce unreliable probability estimates.

from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import cross_val_score
from sklearn.datasets import load_breast_cancer

X, y = load_breast_cancer(return_X_y=True)

for min_leaf in [1, 5, 10, 20, 50]:
    tree = DecisionTreeClassifier(
        min_samples_leaf=min_leaf,
        random_state=42
    )
    score = cross_val_score(tree, X, y, cv=10).mean()
    tree.fit(X, y)
    print(f'min_samples_leaf={min_leaf:3}: leaves={tree.get_n_leaves():4}, CV acc={score:.3f}')

max_leaf_nodes: Limiting Total Leaves

Instead of controlling depth, you can directly limit the maximum number of leaf nodes. With max_leaf_nodes=10, the tree grows until it has exactly 10 leaves, choosing the split that provides the greatest impurity reduction at each step (best-first growth, not depth-first). This produces more balanced trees than depth-limited growth, as splits are allocated where they are most informative rather than being forced to equal depth. Best-first growth can sometimes give better performance than depth-limited growth on the same number of leaves.

from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import cross_val_score
from sklearn.datasets import load_breast_cancer

X, y = load_breast_cancer(return_X_y=True)

for n_leaves in [2, 4, 8, 16, 32, 64]:
    tree = DecisionTreeClassifier(
        max_leaf_nodes=n_leaves,
        random_state=42
    )
    score = cross_val_score(tree, X, y, cv=10).mean()
    tree.fit(X, y)
    print(f'max_leaf_nodes={n_leaves:3}: depth={tree.get_depth()}, '
          f'actual leaves={tree.get_n_leaves()}, CV acc={score:.3f}')

Cost-Complexity Pruning with ccp_alpha

Post-pruning reduces a fully-grown tree after training by removing splits that contribute little predictive value. Scikit-learn implements cost-complexity pruning via the ccp_alpha parameter. Larger ccp_alpha prunes more aggressively, resulting in smaller trees. You can find the optimal alpha by computing the pruning path with cost_complexity_pruning_path(), which returns the alpha values at which each subtree becomes optimal. Cross-validate over these alphas to find the one that maximises validation accuracy.

from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import cross_val_score, train_test_split
from sklearn.datasets import load_breast_cancer
import numpy as np

X, y = load_breast_cancer(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, random_state=42)

# Get pruning path
tree = DecisionTreeClassifier(random_state=42)
path = tree.cost_complexity_pruning_path(X_tr, y_tr)
ccp_alphas = path.ccp_alphas

# Cross-validate each alpha
best_alpha, best_score = 0, 0
for alpha in ccp_alphas[::5]:  # sample every 5th
    t = DecisionTreeClassifier(ccp_alpha=alpha, random_state=42)
    score = cross_val_score(t, X_tr, y_tr, cv=5).mean()
    if score > best_score:
        best_score, best_alpha = score, alpha

print(f'Best ccp_alpha: {best_alpha:.5f}, CV acc: {best_score:.3f}')

Visualising Overfitting vs Good Fit

Plotting a shallow tree vs a deep tree side-by-side on a 2D dataset makes the overfitting effect visually obvious. A shallow tree (depth=2) draws a few rectangular decision regions that capture the main structure of the data. A deep tree (depth=20) creates hundreds of tiny rectangular regions that perfectly trace the training points, including noise. Good generalisation comes from finding the depth where test accuracy is highest — this is always below the fully-grown tree depth, and finding it is the goal of every depth-tuning experiment.

from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import make_moons
import numpy as np

X, y = make_moons(n_samples=300, noise=0.3, random_state=42)

for depth in [2, 5, 20]:
    tree = DecisionTreeClassifier(max_depth=depth, random_state=42)
    tree.fit(X, y)
    train_acc = tree.score(X, y)
    print(f'depth={depth:2}: train={train_acc:.3f}, '
          f'leaves={tree.get_n_leaves()}')

# depth=2:  train~0.85 (underfit), few leaves
# depth=5:  train~0.92 (good fit), balanced leaves
# depth=20: train=1.00 (overfit), many tiny leaves

GridSearchCV for Tree Depth Hyperparameters

Use GridSearchCV to search over multiple regularisation parameters simultaneously. Searching over combinations of max_depth, min_samples_split, and min_samples_leaf finds the best-regularised tree in one step. This is more systematic than tuning parameters one at a time, as they can interact: a shallow tree may need larger min_samples_leaf than a deep tree. Always perform this search within a cross-validation loop to get an unbiased estimate of the optimal configuration's true performance.

from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.datasets import load_breast_cancer

X, y = load_breast_cancer(return_X_y=True)

param_grid = {
    'max_depth': [3, 5, 7, 10, None],
    'min_samples_split': [2, 10, 20],
    'min_samples_leaf': [1, 5, 10]
}

grid = GridSearchCV(
    DecisionTreeClassifier(random_state=42),
    param_grid, cv=10, scoring='accuracy', n_jobs=-1
)
grid.fit(X, y)

print('Best params:', grid.best_params_)
print('Best CV accuracy:', grid.best_score_.round(3))

Learning Curves: Diagnosing Over- and Underfitting

Learning curves plot model performance vs the number of training samples. For a well-tuned tree: both training and validation accuracy converge to a high value as training size increases. For an overfit deep tree: training accuracy stays high but validation accuracy remains low even with many samples. For an underfit shallow tree: both curves plateau at a mediocre accuracy regardless of data size. If adding more data improves validation accuracy, the model is underfitting. If the gap between train and val accuracy is large, the model is overfitting and needs more regularisation.

from sklearn.model_selection import learning_curve
from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import load_breast_cancer
import numpy as np

X, y = load_breast_cancer(return_X_y=True)

for depth in [3, None]:
    tree = DecisionTreeClassifier(max_depth=depth, random_state=42)
    sizes, tr_sc, val_sc = learning_curve(
        tree, X, y, cv=5,
        train_sizes=np.linspace(0.1, 1.0, 10)
    )
    print(f'max_depth={depth}: final train={tr_sc[:,-1].mean():.3f}, '
          f'final val={val_sc[:,-1].mean():.3f}')

Best Practices for Tree Depth Control

A summary of depth-control best practices: (1) Always start with a shallow tree (depth 3-5) and add depth only if cross-validation improves. (2) Use min_samples_leaf in addition to max_depth — they control overfitting at different granularities. (3) For imbalanced datasets, use stratified CV when tuning. (4) If the dataset has many features, also consider max_features to introduce randomness. (5) Remember that if you plan to use an ensemble (Random Forest or Gradient Boosting), single-tree depth tuning is less critical — the ensemble will compensate.

from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import cross_val_score
from sklearn.datasets import load_breast_cancer

X, y = load_breast_cancer(return_X_y=True)

# Practical starting configuration
tree = DecisionTreeClassifier(
    max_depth=5,             # Start shallow
    min_samples_split=20,    # Need 20+ samples to split
    min_samples_leaf=10,     # Each leaf must have 10+ samples
    class_weight='balanced', # Handle class imbalance
    random_state=42
)

score = cross_val_score(tree, X, y, cv=10)
print(f'CV accuracy: {score.mean():.3f} (+/- {score.std():.3f})')

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: how unconstrained trees overfit by memorising training noise, the main regularisation parameters — max_depth, min_samples_split, min_samples_leaf, and max_leaf_nodes — and how to select them via GridSearchCV, and cost-complexity pruning as a post-training alternative. Next up we explore how to visualise and interpret decision trees to communicate model decisions.

Sıkça Sorulan Sorular

“Aşırı Uydurmayı Önlemek için Ağaç Derinliğini Denetleme” dersi ücretsiz mi?

Evet — “Aşırı Uydurmayı Önlemek için Ağaç Derinliğini Denetleme” dersin tüm metni burada web'de ücretsiz olarak okunabilir. Etkileşimli olarak pratik yapmak (yerleşik kod editörü ve 7/24 yapay zeka koçu) ve Machine Learning Academy kursunun geri kalanını açmak için CoddyKit PRO'ya yükselt. Machine Learning Academy kursu toplamda 4 dersten oluşur.

“Aşırı Uydurmayı Önlemek için Ağaç Derinliğini Denetleme” dersinde ne öğreneceğim?

Farklı max_depth değerleriyle ağaçlar eğitin, aşırı ve yetersiz uydurma ödünleşimini gözlemleyin ve derinliği doğrulama puanıyla seçin. Machine Learning Academy ile uygulamalı kodu tarayıcıda doğrudan çalıştırarak pratik yaparsın ve 7/24 yapay zeka koçu dersi çalışırken sorularını yanıtlar.

Machine Learning Academy öğrenmeye başlamak için deneyim gerekli mi?

Önceden deneyim gerekmez. CoddyKit'te Machine Learning Academy, başlangıçtan ileri seviyeye kadar yapılandırıldığı için buradan başlayabilir veya başından başlayıp kendi hızında ilerleme yapabilirsin. Bu, 4 dersinin 3. dersidir.

“Aşırı Uydurmayı Önlemek için Ağaç Derinliğini Denetleme” dersi ne kadar sürer?

Çoğu CoddyKit dersi yaklaşık 5–10 dakika sürer. Her biri kısa ve etkileşimli olduğu için sabit ilerleme yaparsın ve web ile uygulama arasında tam olarak bıraktığın yerden devam edebilirsin.

Bu Machine Learning Academy dersinde kod yazıp çalıştırabilir miyim?

Evet. Her Machine Learning Academy dersi yerleşik bir kod editörü içerir, bu sayede tarayıcıda gerçek kod yazıp çalıştırabilir ve anlık yapay zeka geri bildirimi alırsın — yerel kurulum gerekli değildir.

Bu kursun tüm dersleri

Ağaç Oluşturma: Bölmeler, Düğümler ve Yapraklar
Gini Safsızlığı ve Bilgi Kazancı
Aşırı Uydurmayı Önlemek için Ağaç Derinliğini Denetleme
Karar Ağaçlarını Görselleştirme ve Yorumlama

← Machine Learning Academy Sayfasına Dön