Resampling and Cross-Validation with rsample
Evaluate models with k-fold CV, bootstrap, and nested resampling.
Why Resample?
A single train/test split gives a noisy estimate of model performance — you got lucky or unlucky with which observations ended up in the test set. Resampling repeats the process multiple times to get a stable, reliable estimate of how your model generalises to new data.
library(rsample)
# Single split — performance estimate depends heavily
# on which 20% ended up as test data
split <- initial_split(mtcars, prop = 0.8)
train <- training(split)
test <- testing(split)
cat('Train:', nrow(train), '| Test:', nrow(test))initial_split()
initial_split(data, prop, strata) creates a single random split into training and test sets. Use strata to stratify by a column (e.g. the outcome variable) to ensure class balance is maintained in both partitions.
library(rsample)
# Stratified split by outcome variable
split <- initial_split(ames, prop = 0.8, strata = Sale_Price)
train <- training(split)
test <- testing(split)
cat('Train rows:', nrow(train))
cat('Test rows:', nrow(test))All lessons in this course
- Feature Engineering with recipes
- Model Specifications with parsnip
- Workflows: Combining Recipes and Models
- Resampling and Cross-Validation with rsample