K-Fold Cross-Validation: Splitting Without Leaking
Learners will implement 5-fold CV with cross_val_score, understand why the test fold is never used during training, and interpret the mean and standard deviation of CV scores.
Why a Single Train-Test Split Is Risky
When you evaluate a model on a single held-out test set, the result depends heavily on which examples happened to end up in the test set. With a small dataset, 80 test examples might be unusually easy or hard, making your accuracy estimate misleading. K-Fold Cross-Validation solves this by averaging accuracy across multiple test folds, giving a more reliable estimate of how the model will perform on new data. It also uses the data more efficiently — every example is used for both training and testing across different folds.
How K-Fold CV Works Step by Step
K-Fold CV splits the dataset into k equally sized parts called folds. In each of k rounds, one fold is used as the test fold and the remaining k-1 folds form the training set. The model is trained from scratch on the training folds and evaluated on the test fold. This repeats until every fold has served as the test fold exactly once. The final performance estimate is the mean (and standard deviation) of the k scores. Common values: k=5 or k=10.
All lessons in this course
- K-Fold Cross-Validation: Splitting Without Leaking
- Stratified and Time-Series Cross-Validation
- Grid Search vs Random Search
- Nested Cross-Validation: Selecting and Evaluating Simultaneously