0Pricing
R Academy · Lesson

Gradient Boosting with xgboost

Configure xgboost parameters, early stopping, and learning rate schedules.

What Is Gradient Boosting?

Gradient boosting builds an ensemble sequentially. Each new tree corrects the residual errors of the previous ensemble by fitting to the negative gradient of the loss function. Unlike random forests (parallel trees), boosting builds trees one at a time, each learning from the mistakes of the last.

library(xgboost)

# XGBoost expects data in a special matrix format
# We'll build a simple example step by step
cat('XGBoost version:', packageVersion('xgboost'))

# Key concept: each tree reduces the ensemble error
# Final prediction = sum of all tree predictions

xgb.DMatrix()

xgb.DMatrix(data, label) is XGBoost's optimised internal data format. It stores the feature matrix and label vector together, enabling fast memory-efficient computation. Always convert your data to DMatrix before training.

library(xgboost)
library(MASS)

X_train <- as.matrix(Boston[1:400, -14])  # features
y_train <- Boston[1:400, 14]              # medv (target)

X_test  <- as.matrix(Boston[401:506, -14])
y_test  <- Boston[401:506, 14]

dtrain <- xgb.DMatrix(data = X_train, label = y_train)
dtest  <- xgb.DMatrix(data = X_test,  label = y_test)

cat('DMatrix rows:', nrow(dtrain))

All lessons in this course

  1. Decision Trees: Foundation of Ensembles
  2. Random Forests with ranger
  3. Gradient Boosting with xgboost
  4. Feature Importance and Model Interpretation
← Back to R Academy