랜덤 forest와 배깅

단일 트리의 문제

Bagging (부트스트랩 집계)

from sklearn.ensemble import BaggingClassifier
from sklearn.tree import DecisionTreeClassifier

bag = BaggingClassifier(
    estimator=DecisionTreeClassifier(),
    n_estimators=100,
    random_state=0,
)
bag.fit(Xtr, ytr)

Bagging에서 랜덤 포레스트로

RandomForestClassifier 기초

from sklearn.ensemble import RandomForestClassifier

rf = RandomForestClassifier(n_estimators=200, random_state=0)
rf.fit(Xtr, ytr)
print("Accuracy:", rf.score(Xte, yte))

n_jobs를 사용한 병렬 학습

from sklearn.ensemble import RandomForestClassifier

rf = RandomForestClassifier(
    n_estimators=500,
    n_jobs=-1,        # use all cores
    random_state=0,
)
rf.fit(Xtr, ytr)

OOB 점수

from sklearn.ensemble import RandomForestClassifier

rf = RandomForestClassifier(
    n_estimators=300,
    oob_score=True,
    random_state=0,
)
rf.fit(Xtr, ytr)
print("OOB score:", rf.oob_score_)

포레스트에서 트리 깊이 제어하기

from sklearn.ensemble import RandomForestClassifier

rf = RandomForestClassifier(
    n_estimators=300,
    max_depth=12,
    max_features="sqrt",
    random_state=0,
)

내장 특성 중요도

from sklearn.ensemble import RandomForestClassifier
import numpy as np

rf = RandomForestClassifier(n_estimators=200, random_state=0).fit(Xtr, ytr)
order = np.argsort(rf.feature_importances_)[::-1]
for i in order[:5]:
    print(i, rf.feature_importances_[i])

순열 중요도

from sklearn.inspection import permutation_importance

result = permutation_importance(
    rf, Xte, yte, n_repeats=10, random_state=0, n_jobs=-1
)
print(result.importances_mean)

순열 결과 해석하기

import numpy as np

means = result.importances_mean
stds = result.importances_std
for i in np.argsort(means)[::-1]:
    print(f"feature {i}: {means[i]:.3f} +/- {stds[i]:.3f}")

랜덤 포레스트를 사용할 때

빠른 확인

복습