0Pricing
R Academy · Lesson

Debugging and Load Balancing Parallel Code

Handle errors in parallel workers and balance uneven workloads effectively.

Why Parallel Debugging Is Hard

Debugging parallel code is challenging because workers run in separate processes: print() statements are invisible to the master, interactive debuggers like browser() don't work inside workers, and errors get serialised and re-thrown in the master, losing their original stack trace.

library(parallel)

# Problem: cat() inside worker is invisible in the master
cl <- makeCluster(2)
clusterExport(cl, c())

result <- parLapply(cl, 1:4, function(i) {
  # This cat() goes to the worker's stdout, NOT the master
  cat('Worker processing', i, '\n')  # you will NOT see this
  i^2
})

cat('Master received:', unlist(result), '\n')
stopCluster(cl)

Error Propagation in parLapply

When a worker throws an error, parLapply() wraps it and re-throws it in the master. The error message is preserved but the remote stack trace is not. Always test your function sequentially first before parallelising.

library(parallel)
cl <- makeCluster(2)

# Step 1: test sequentially first!
my_fn <- function(x) {
  if (x == 3) stop('bad value: 3')
  sqrt(x)
}

# Sequential test reveals the bug before parallelising
tryCatch(
  lapply(1:5, my_fn),
  error = function(e) cat('Sequential error caught:', e$message, '\n')
)

# Fix: guard the bad case
my_fn_safe <- function(x) {
  if (x <= 0) return(NA_real_)
  sqrt(x)
}
clusterExport(cl, 'my_fn_safe')
cat(unlist(parLapply(cl, 1:5, my_fn_safe)), '\n')

stopCluster(cl)

All lessons in this course

  1. parallel Package and detectCores()
  2. The future Framework
  3. furrr: Parallel purrr Operations
  4. Debugging and Load Balancing Parallel Code
← Back to R Academy