Tree-Based Models: Decision Trees and Random Forests
Build and tune decision tree and ensemble forest models.
Decision Tree Concepts
A decision tree splits data into subsets by asking a sequence of binary questions. Each internal node is a feature threshold; each leaf is a prediction.
from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import load_iris
X, y = load_iris(return_X_y=True)
model = DecisionTreeClassifier(max_depth=3, random_state=0).fit(X, y)
print("Depth:", model.get_depth())
print("Leaves:", model.get_n_leaves())DecisionTreeClassifier
Key hyperparameters: max_depth, min_samples_split, min_samples_leaf. Deeper trees overfit; shallower trees underfit.
from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
X, y = make_classification(random_state=0)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=0)
model = DecisionTreeClassifier(max_depth=5, min_samples_leaf=5)
model.fit(X_tr, y_tr)
print("Test acc:", model.score(X_te, y_te))All lessons in this course
- The scikit-learn API: fit, transform, predict
- Linear Models: Regression and Classification
- Tree-Based Models: Decision Trees and Random Forests
- Model Evaluation and Cross-Validation