0Pricing
R Academy · Lesson

parallel Package and detectCores()

Launch forked or socket clusters and distribute work across CPU cores.

parallel Package and detectCores() is a free R Academy lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the R Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Why Parallel Computing?

Modern computers have multiple CPU cores. By default, R runs on a single core, leaving the rest idle. The parallel package (built into R) lets you harness all cores to speed up repetitive computations.

# Check how many cores your machine has
library(parallel)

total_cores <- detectCores()
logical_cores <- detectCores(logical = TRUE)
physical_cores <- detectCores(logical = FALSE)

cat('Total logical cores:', total_cores, '
')
cat('Physical cores:', physical_cores, '
')

makeCluster and stopCluster

makeCluster(n) spawns n worker processes. Always call stopCluster(cl) when done to free resources. A common convention is to use detectCores() - 1 to leave one core for the OS.

library(parallel)

# Spawn workers (leave 1 core for system)
n_cores <- detectCores() - 1
cl <- makeCluster(n_cores)

cat('Cluster created with', n_cores, 'workers\n')

# Always clean up!
stopCluster(cl)
cat('Cluster stopped.\n')

clusterExport: Sharing Variables

Worker processes have their own memory space and cannot see your global environment. Use clusterExport(cl, varlist) to copy named objects from the master to all workers.

library(parallel)
cl <- makeCluster(2)

# Define a variable and a function in the master
base_value <- 100
add_base <- function(x) x + base_value

# Export them to workers
clusterExport(cl, varlist = c('base_value', 'add_base'))

# Now workers can use them
result <- parLapply(cl, 1:4, function(x) add_base(x))
cat(unlist(result), '\n')  # 101 102 103 104

stopCluster(cl)

clusterEvalQ: Running Setup Code

clusterEvalQ(cl, expr) evaluates an expression on every worker — useful for loading packages or sourcing helper files across all nodes before the main computation begins.

library(parallel)
cl <- makeCluster(2)

# Load a package on every worker
clusterEvalQ(cl, {
  library(stats)
  set.seed(42)  # set seed per worker
})

# Each worker can now use stats functions
result <- parLapply(cl, 1:4, function(n) rnorm(n, mean = 0, sd = 1))
result[[1]]  # one random normal value

stopCluster(cl)

parLapply: Parallel lapply

parLapply(cl, X, FUN) is the parallel equivalent of lapply(). It distributes elements of X across workers and collects results as a list. It works on all platforms (Windows, macOS, Linux).

library(parallel)
cl <- makeCluster(2)

# Simulate slow computation: sleep 0.1s per item
clusterEvalQ(cl, Sys.sleep)

slow_square <- function(x) {
  Sys.sleep(0.05)
  x^2
}

clusterExport(cl, 'slow_square')

system.time(
  result <- parLapply(cl, 1:8, slow_square)
)
cat(unlist(result), '\n')  # 1 4 9 16 25 36 49 64

stopCluster(cl)

parSapply: Simplified Results

parSapply(cl, X, FUN) is the parallel version of sapply(). It automatically simplifies the result to a vector or matrix when possible, making it convenient when your function returns a scalar.

library(parallel)
cl <- makeCluster(2)

# parSapply returns a simplified vector
squares <- parSapply(cl, 1:10, function(x) x^2)
cat(squares, '\n')  # 1 4 9 16 25 36 49 64 81 100

# Returns a matrix if FUN returns a vector of same length
stats_result <- parSapply(cl, 1:4, function(n) {
  x <- rnorm(100)
  c(mean = mean(x), sd = sd(x))
})
print(stats_result)  # 2x4 matrix

stopCluster(cl)

mclapply: Fork-Based Parallelism

mclapply() uses forking — it copies the current R process instead of spawning new ones, making it faster to start and giving workers implicit access to the parent environment. However, it only works on Unix/macOS, not Windows.

library(parallel)

# mclapply: Unix/macOS only
# Workers inherit the parent environment automatically
base_value <- 42

if (.Platform$OS.type != 'windows') {
  result <- mclapply(
    1:8,
    function(x) x * base_value,  # base_value visible without export
    mc.cores = 4
  )
  cat(unlist(result), '\n')
} else {
  cat('mclapply not supported on Windows. Use parLapply instead.\n')
}

Benchmarking Sequential vs Parallel

Parallelism has overhead: cluster startup, data serialisation, and inter-process communication all take time. It pays off only when the per-element computation is expensive enough to outweigh that overhead.

library(parallel)
cl <- makeCluster(2)

# Task: compute 100 iterations of matrix multiply
heavy_task <- function(n) {
  m <- matrix(rnorm(200), nrow = 100)
  sum(m %*% t(m))
}
clusterExport(cl, 'heavy_task')

seq_time <- system.time(lapply(1:20, heavy_task))[['elapsed']]
par_time <- system.time(parLapply(cl, 1:20, heavy_task))[['elapsed']]

cat('Sequential:', round(seq_time, 3), 's\n')
cat('Parallel:  ', round(par_time, 3), 's\n')
cat('Speedup:   ', round(seq_time / par_time, 2), 'x\n')

stopCluster(cl)

Seeding RNG in Parallel

Random number generation in parallel is tricky — each worker needs an independent, reproducible stream. Use clusterSetRNGStream(cl, seed) with the L'Ecuyer-CMRG generator to get reproducible parallel randomness.

library(parallel)
cl <- makeCluster(2)

# Set reproducible RNG streams across workers
clusterSetRNGStream(cl, iseed = 123)

# Each worker uses its own independent random stream
results1 <- parSapply(cl, 1:6, function(i) rnorm(1))

# Reset and repeat — same results
clusterSetRNGStream(cl, iseed = 123)
results2 <- parSapply(cl, 1:6, function(i) rnorm(1))

cat('Run 1:', round(results1, 4), '\n')
cat('Run 2:', round(results2, 4), '\n')
cat('Identical:', identical(results1, results2), '\n')

stopCluster(cl)

Error Handling in Clusters

If a worker throws an error, parLapply() re-throws it in the master process. Wrap calls in tryCatch() inside the function, or use tryCatch() around the entire parLapply() call to handle failures gracefully.

library(parallel)
cl <- makeCluster(2)

# Wrap risky code inside the worker function
safe_log <- function(x) {
  tryCatch(
    log(x),
    warning = function(w) NA_real_,
    error   = function(e) NA_real_
  )
}
clusterExport(cl, 'safe_log')

# -1 produces NaN warning, 'a' produces an error
input <- list(4, 9, -1, 'a', 16)
result <- parLapply(cl, input, safe_log)
cat(unlist(result), '\n')  # 1.386 2.197 NaN NA 2.773

stopCluster(cl)

Practical Parallel Pattern

Here is a complete, idiomatic parallel workflow: detect cores, create cluster, export dependencies, run computation, collect results, and always stop the cluster — even if an error occurs — using on.exit().

library(parallel)

run_parallel <- function(data, fn, n_workers = detectCores() - 1) {
  cl <- makeCluster(n_workers)
  on.exit(stopCluster(cl))  # guaranteed cleanup

  clusterExport(cl, 'fn', envir = environment())

  result <- parLapply(cl, data, fn)
  result
}

# Example: bootstrap mean estimation
samples <- lapply(1:100, function(i) rnorm(50, mean = 5, sd = 2))
means <- run_parallel(samples, mean)

cat('Grand mean:', round(mean(unlist(means)), 3), '\n')
cat('95% CI: [',
    round(quantile(unlist(means), 0.025), 3), ',',
    round(quantile(unlist(means), 0.975), 3), ']\n')

Quick Check

Which function should you use to guarantee that stopCluster(cl) is called even if an error occurs inside your parallel code?

Recap: parallel Package

Key takeaways:

  • detectCores() reports available cores; use detectCores() - 1 for the cluster size
  • makeCluster(n) / stopCluster(cl) manage worker lifecycles
  • clusterExport() copies objects; clusterEvalQ() runs setup code on workers
  • parLapply() / parSapply() are cross-platform parallel iterators
  • mclapply() is Unix-only but faster to start due to forking
  • Use on.exit(stopCluster(cl)) for guaranteed cleanup
  • Parallelism pays off only for computationally heavy tasks
# Minimal reproducible parallel pattern
library(parallel)

cl <- makeCluster(max(1, detectCores() - 1))
on.exit(stopCluster(cl))

clusterSetRNGStream(cl, iseed = 42)
result <- parSapply(cl, 1:8, function(x) x^2 + rnorm(1, 0, 0.1))
cat(round(result, 2), '\n')

Frequently asked questions

Is the “parallel Package and detectCores()” lesson free?

Yes — the full text of “parallel Package and detectCores()” is free to read here on the web, and the R Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the R Academy course, upgrade to CoddyKit PRO.

What will I learn in “parallel Package and detectCores()”?

Launch forked or socket clusters and distribute work across CPU cores. You practise R Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start R Academy?

No prior experience is required. R Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “parallel Package and detectCores()” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this R Academy lesson?

Yes. Every R Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. parallel Package and detectCores()
  2. The future Framework
  3. furrr: Parallel purrr Operations
  4. Debugging and Load Balancing Parallel Code
← Back to R Academy