Random Forests with ranger
Train random forest models, tune mtry and ntrees, and assess OOB error.
What Is a Random Forest?
A random forest builds many decision trees on bootstrapped samples of the data, then averages their predictions (regression) or takes a majority vote (classification). Two sources of randomness give the ensemble its name: bootstrap sampling of rows, and random feature selection at each split.
library(ranger)
# Minimal random forest: 500 trees, auto mtry
rf <- ranger(
medv ~ .,
data = MASS::Boston,
num.trees = 500
)
print(rf)The mtry Parameter
mtry is the number of features randomly considered at each split. Using fewer features than the total decorrelates the trees, reducing variance. The rule of thumb is mtry = sqrt(p) for classification and mtry = p/3 for regression, where p is the number of predictors.
p <- ncol(MASS::Boston) - 1 # number of predictors
rf_class <- ranger(
Species ~ ., data = iris,
num.trees = 500,
mtry = floor(sqrt(4)) # sqrt(p) for classification
)
rf_reg <- ranger(
medv ~ ., data = MASS::Boston,
num.trees = 500,
mtry = floor(p / 3) # p/3 for regression
)
cat('Classification OOB Error:', rf_class$prediction.error)
cat('Regression OOB RMSE:', sqrt(rf_reg$prediction.error))