R Academy · Lesson

Training, Validation, and Preventing Overfitting

Monitor val_loss, apply Dropout, and use callbacks for early stopping.

Lesson 4 of 413 steps

Training, Validation, and Preventing Overfitting is a free R Academy lesson on CoddyKit. This is lesson 4 of 4. Any 3 lessons of this learning path are free to read in full — after that, CoddyKit PRO unlocks every lesson, plus hands-on practice with a built-in code editor and a 24/7 AI tutor. It is part of the R Academy learning path, and your progress syncs across the web and the CoddyKit app. The R Academy course includes 4 lessons in total.

Training, Validation, Test Split

Deep learning requires three data partitions: training (model learns parameters), validation (monitor generalisation during training, tune hyperparameters), and test (final unbiased evaluation). The validation_split argument in fit() creates the validation set automatically.

# validation_split = 0.2 reserves last 20% as validation
history <- model |> fit(
  x_train, y_train,
  epochs           = 50,
  batch_size       = 32,
  validation_split = 0.2,
  verbose          = 0
)

# history contains train and val metrics per epoch
names(history$metrics)

The Training History Object

The object returned by fit() contains a $metrics list with one entry per logged metric per epoch. Call plot(history) to visualise training and validation curves side by side. Diverging curves (train improves, val plateaus) signal overfitting.

# Training loss and accuracy over epochs
head(history$metrics$loss)
head(history$metrics$val_loss)

# Plot training curves
plot(history)

# Or create a custom ggplot
library(ggplot2)
df <- data.frame(
  epoch    = seq_along(history$metrics$loss),
  train    = history$metrics$loss,
  val      = history$metrics$val_loss
)
ggplot(df, aes(epoch)) +
  geom_line(aes(y = train, colour = 'Train')) +
  geom_line(aes(y = val,   colour = 'Validation')) +
  labs(y = 'Loss', title = 'Training Curves')

Detecting Overfitting

Overfitting occurs when training loss continues to decrease but validation loss starts to increase. The model memorises training data rather than learning generalisable patterns. Signs: large gap between train and val accuracy, val loss has a visible minimum then increases.

# Overfitting is visible in the history
# Example of overfitting signatures:
cat('Epoch 5  — Train loss: 0.12, Val loss: 0.18\n')
cat('Epoch 10 — Train loss: 0.06, Val loss: 0.21\n')
cat('Epoch 20 — Train loss: 0.02, Val loss: 0.31\n')

# The model should have stopped at epoch 5!
# Strategies: early stopping, dropout, regularisation, more data

callback_early_stopping()

callback_early_stopping(monitor, patience, restore_best_weights) halts training when the monitored metric stops improving. patience is the number of epochs to wait after last improvement. Set restore_best_weights = TRUE to automatically roll back to the best epoch's weights.

early_stop <- callback_early_stopping(
  monitor              = 'val_loss',
  patience             = 10,        # wait 10 epochs
  restore_best_weights = TRUE       # rollback to best
)

history <- model |> fit(
  x_train, y_train,
  epochs           = 200,
  batch_size       = 32,
  validation_split = 0.2,
  callbacks        = list(early_stop),
  verbose          = 0
)

cat('Stopped at epoch:', length(history$metrics$loss))

callback_reduce_lr_on_plateau()

When training stalls, reducing the learning rate often restarts progress. callback_reduce_lr_on_plateau(monitor, factor, patience) multiplies the current learning rate by factor when the monitored metric has not improved for patience epochs.

reduce_lr <- callback_reduce_lr_on_plateau(
  monitor  = 'val_loss',
  factor   = 0.5,    # halve the learning rate
  patience = 5,      # after 5 stagnant epochs
  min_lr   = 1e-6    # floor for learning rate
)

history <- model |> fit(
  x_train, y_train,
  epochs           = 100,
  batch_size       = 64,
  validation_split = 0.2,
  callbacks        = list(early_stop, reduce_lr),
  verbose          = 1
)

callback_model_checkpoint()

callback_model_checkpoint(filepath, save_best_only) saves the model weights to disk at the end of each epoch (or only when performance improves with save_best_only = TRUE). This protects against training crashes and lets you load the best model even if training continues past the optimal point.

checkpoint <- callback_model_checkpoint(
  filepath       = '/tmp/best_model.h5',
  monitor        = 'val_accuracy',
  save_best_only = TRUE,
  mode           = 'max',      # higher accuracy = better
  verbose        = 1
)

history <- model |> fit(
  x_train, y_train,
  epochs           = 100,
  batch_size       = 32,
  validation_split = 0.2,
  callbacks        = list(early_stop, reduce_lr, checkpoint)
)

# Reload best model
best_model <- load_model_hdf5('/tmp/best_model.h5')

validation_data vs validation_split

validation_split takes the last N% of your training data. If your data is ordered (e.g. time series), this is biased. Use validation_data = list(x_val, y_val) instead to supply a pre-built validation set from a random stratified split.

# Manually create a random validation split
set.seed(42)
val_idx  <- sample(nrow(x_train), size = 0.2 * nrow(x_train))
x_val    <- x_train[val_idx, ]
y_val    <- y_train[val_idx, ]
x_tr     <- x_train[-val_idx, ]
y_tr     <- y_train[-val_idx, ]

history <- model |> fit(
  x_tr, y_tr,
  epochs         = 50,
  batch_size     = 32,
  validation_data = list(x_val, y_val),  # explicit val set
  callbacks      = list(early_stop)
)

Batch Size Effect

Batch size is a key training hyperparameter. Smaller batches introduce more noise into gradient estimates (acts as regularisation), helping generalisation. Larger batches are faster but can converge to sharper, less generalisable minima. Typical values: 32, 64, 128. Try 32 first.

# Compare training with different batch sizes
for (bs in c(32, 128, 512)) {
  set_weights(model, init_weights)  # reset
  h <- model |> fit(
    x_train, y_train,
    epochs           = 20,
    batch_size       = bs,
    validation_split = 0.2,
    verbose          = 0
  )
  cat('Batch:', bs, '| Val Acc:',
      tail(h$metrics$val_accuracy, 1), '\n')
}

Learning Rate Warmup

Starting with a very small learning rate and gradually increasing it over the first few epochs (warmup) can stabilise training, especially for large models or small datasets. A custom LearningRateScheduler callback enables this pattern.

# Custom learning rate schedule with warmup
lr_schedule <- function(epoch, lr) {
  if (epoch < 5) {
    return(lr * (epoch + 1) / 5)  # warmup
  } else if (epoch < 30) {
    return(lr)                     # constant
  } else {
    return(lr * 0.95)              # decay
  }
}

lr_callback <- callback_learning_rate_scheduler(lr_schedule)

history <- model |> fit(
  x_train, y_train,
  epochs     = 50,
  callbacks  = list(lr_callback, early_stop),
  validation_split = 0.2
)

Regularisation Summary

Multiple regularisation techniques should be combined for robust deep learning:

Dropout: randomly zero neurons during training.
Weight decay (L2): penalise large weights in the loss.
Early stopping: stop before the model overfits.
Data augmentation: artificially increase training set size.
Batch normalisation: stabilise activations and reduce covariate shift.

model <- keras_model_sequential(input_shape = c(784)) |>
  layer_dense(512, use_bias = FALSE,
              kernel_regularizer = regularizer_l2(1e-4)) |>
  layer_batch_normalization() |>
  layer_activation('relu') |>
  layer_dropout(0.4) |>
  layer_dense(256, kernel_regularizer = regularizer_l2(1e-4)) |>
  layer_batch_normalization() |>
  layer_activation('relu') |>
  layer_dropout(0.3) |>
  layer_dense(10, activation = 'softmax')

model |> compile('adam', 'categorical_crossentropy', 'accuracy')

Plotting Training Curves

Always visualise training and validation metrics side by side. A well-trained model should show both curves converging and staying close. If they diverge, add regularisation or reduce model capacity. If both curves plateau at high loss, the model is underfitting.

# Detailed training curve plot
df <- data.frame(
  epoch = seq_along(history$metrics$loss),
  train_loss = history$metrics$loss,
  val_loss   = history$metrics$val_loss,
  train_acc  = history$metrics$accuracy,
  val_acc    = history$metrics$val_accuracy
)

par(mfrow = c(1, 2))
plot(df$epoch, df$train_loss, type = 'l', col = 'blue',
     xlab = 'Epoch', ylab = 'Loss', main = 'Loss')
lines(df$epoch, df$val_loss, col = 'red')
legend('topright', c('Train', 'Val'), col = c('blue','red'), lty=1)

plot(df$epoch, df$train_acc, type = 'l', col = 'blue',
     xlab = 'Epoch', ylab = 'Accuracy', main = 'Accuracy')
lines(df$epoch, df$val_acc, col = 'red')

Quick Check

What does setting restore_best_weights = TRUE in callback_early_stopping() do?

Answer it in the interactive lesson

Training and Overfitting Recap

Key takeaways from Training, Validation, and Preventing Overfitting:

Use validation_split or validation_data to monitor generalisation during training.
plot(history) visualises training curves — diverging curves signal overfitting.
callback_early_stopping(patience, restore_best_weights=TRUE) stops training at the optimal epoch.
callback_reduce_lr_on_plateau() reduces learning rate when progress stalls.
callback_model_checkpoint(save_best_only=TRUE) saves the best model to disk.
Combine dropout, L2 regularisation, batch normalisation, and data augmentation for robust training.

# Best practice training setup
callbacks <- list(
  callback_early_stopping(
    monitor = 'val_loss', patience = 15,
    restore_best_weights = TRUE
  ),
  callback_reduce_lr_on_plateau(
    monitor = 'val_loss', factor = 0.5, patience = 5
  ),
  callback_model_checkpoint(
    '/tmp/best.h5', monitor = 'val_accuracy',
    save_best_only = TRUE
  )
)

model |> fit(
  x_train, y_train,
  epochs = 200, batch_size = 64,
  validation_split = 0.2,
  callbacks = callbacks, verbose = 0
)

Free to start

Learn R with an AI tutor — free

Write and run real code in your browser, get instant help from a 24/7 AI tutor, and pick up where you left off on the web or in the app.

Courses: 43
Lessons: 159

Start learning free View the full course

Frequently Asked Questions

Is the “Training, Validation, and Preventing Overfitting” lesson free?

Yes — any 3 lessons of the R Academy learning path, including “Training, Validation, and Preventing Overfitting”, are free to read in full here on the web. After that, CoddyKit PRO unlocks every lesson, plus interactive practice with a built-in code editor and a 24/7 AI tutor. The R Academy course includes 4 lessons in total.

What will I learn in “Training, Validation, and Preventing Overfitting”?

Monitor val_loss, apply Dropout, and use callbacks for early stopping. You practise R Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start R Academy?

No prior experience is required. R Academy on CoddyKit is structured for beginners through advanced learners, so you can start here or from the beginning and move at your own pace. This is lesson 4 of 4.

How long does the “Training, Validation, and Preventing Overfitting” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this R Academy lesson?

Yes. Every R Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

Setting Up Keras and TensorFlow in R
Building Sequential Models
Convolutional Neural Networks Basics
Training, Validation, and Preventing Overfitting

← Back to R Academy