parallel Package and detectCores()
Launch forked or socket clusters and distribute work across CPU cores.
parallel Package and detectCores() is a free R Academy lesson on CoddyKit. This is lesson 1 of 4. Any 3 lessons of this learning path are free to read in full — after that, CoddyKit PRO unlocks every lesson, plus hands-on practice with a built-in code editor and a 24/7 AI tutor. It is part of the R Academy learning path, and your progress syncs across the web and the CoddyKit app. The R Academy course includes 4 lessons in total.
Why Parallel Computing?
Modern computers have multiple CPU cores. By default, R runs on a single core, leaving the rest idle. The parallel package (built into R) lets you harness all cores to speed up repetitive computations.
# Check how many cores your machine has
library(parallel)
total_cores <- detectCores()
logical_cores <- detectCores(logical = TRUE)
physical_cores <- detectCores(logical = FALSE)
cat('Total logical cores:', total_cores, '
')
cat('Physical cores:', physical_cores, '
')makeCluster and stopCluster
makeCluster(n) spawns n worker processes. Always call stopCluster(cl) when done to free resources. A common convention is to use detectCores() - 1 to leave one core for the OS.
library(parallel)
# Spawn workers (leave 1 core for system)
n_cores <- detectCores() - 1
cl <- makeCluster(n_cores)
cat('Cluster created with', n_cores, 'workers\n')
# Always clean up!
stopCluster(cl)
cat('Cluster stopped.\n')clusterExport: Sharing Variables
Worker processes have their own memory space and cannot see your global environment. Use clusterExport(cl, varlist) to copy named objects from the master to all workers.
library(parallel)
cl <- makeCluster(2)
# Define a variable and a function in the master
base_value <- 100
add_base <- function(x) x + base_value
# Export them to workers
clusterExport(cl, varlist = c('base_value', 'add_base'))
# Now workers can use them
result <- parLapply(cl, 1:4, function(x) add_base(x))
cat(unlist(result), '\n') # 101 102 103 104
stopCluster(cl)clusterEvalQ: Running Setup Code
clusterEvalQ(cl, expr) evaluates an expression on every worker — useful for loading packages or sourcing helper files across all nodes before the main computation begins.
library(parallel)
cl <- makeCluster(2)
# Load a package on every worker
clusterEvalQ(cl, {
library(stats)
set.seed(42) # set seed per worker
})
# Each worker can now use stats functions
result <- parLapply(cl, 1:4, function(n) rnorm(n, mean = 0, sd = 1))
result[[1]] # one random normal value
stopCluster(cl)parLapply: Parallel lapply
parLapply(cl, X, FUN) is the parallel equivalent of lapply(). It distributes elements of X across workers and collects results as a list. It works on all platforms (Windows, macOS, Linux).
library(parallel)
cl <- makeCluster(2)
# Simulate slow computation: sleep 0.1s per item
clusterEvalQ(cl, Sys.sleep)
slow_square <- function(x) {
Sys.sleep(0.05)
x^2
}
clusterExport(cl, 'slow_square')
system.time(
result <- parLapply(cl, 1:8, slow_square)
)
cat(unlist(result), '\n') # 1 4 9 16 25 36 49 64
stopCluster(cl)parSapply: Simplified Results
parSapply(cl, X, FUN) is the parallel version of sapply(). It automatically simplifies the result to a vector or matrix when possible, making it convenient when your function returns a scalar.
library(parallel)
cl <- makeCluster(2)
# parSapply returns a simplified vector
squares <- parSapply(cl, 1:10, function(x) x^2)
cat(squares, '\n') # 1 4 9 16 25 36 49 64 81 100
# Returns a matrix if FUN returns a vector of same length
stats_result <- parSapply(cl, 1:4, function(n) {
x <- rnorm(100)
c(mean = mean(x), sd = sd(x))
})
print(stats_result) # 2x4 matrix
stopCluster(cl)mclapply: Fork-Based Parallelism
mclapply() uses forking — it copies the current R process instead of spawning new ones, making it faster to start and giving workers implicit access to the parent environment. However, it only works on Unix/macOS, not Windows.
library(parallel)
# mclapply: Unix/macOS only
# Workers inherit the parent environment automatically
base_value <- 42
if (.Platform$OS.type != 'windows') {
result <- mclapply(
1:8,
function(x) x * base_value, # base_value visible without export
mc.cores = 4
)
cat(unlist(result), '\n')
} else {
cat('mclapply not supported on Windows. Use parLapply instead.\n')
}Benchmarking Sequential vs Parallel
Parallelism has overhead: cluster startup, data serialisation, and inter-process communication all take time. It pays off only when the per-element computation is expensive enough to outweigh that overhead.
library(parallel)
cl <- makeCluster(2)
# Task: compute 100 iterations of matrix multiply
heavy_task <- function(n) {
m <- matrix(rnorm(200), nrow = 100)
sum(m %*% t(m))
}
clusterExport(cl, 'heavy_task')
seq_time <- system.time(lapply(1:20, heavy_task))[['elapsed']]
par_time <- system.time(parLapply(cl, 1:20, heavy_task))[['elapsed']]
cat('Sequential:', round(seq_time, 3), 's\n')
cat('Parallel: ', round(par_time, 3), 's\n')
cat('Speedup: ', round(seq_time / par_time, 2), 'x\n')
stopCluster(cl)Seeding RNG in Parallel
Random number generation in parallel is tricky — each worker needs an independent, reproducible stream. Use clusterSetRNGStream(cl, seed) with the L'Ecuyer-CMRG generator to get reproducible parallel randomness.
library(parallel)
cl <- makeCluster(2)
# Set reproducible RNG streams across workers
clusterSetRNGStream(cl, iseed = 123)
# Each worker uses its own independent random stream
results1 <- parSapply(cl, 1:6, function(i) rnorm(1))
# Reset and repeat — same results
clusterSetRNGStream(cl, iseed = 123)
results2 <- parSapply(cl, 1:6, function(i) rnorm(1))
cat('Run 1:', round(results1, 4), '\n')
cat('Run 2:', round(results2, 4), '\n')
cat('Identical:', identical(results1, results2), '\n')
stopCluster(cl)Error Handling in Clusters
If a worker throws an error, parLapply() re-throws it in the master process. Wrap calls in tryCatch() inside the function, or use tryCatch() around the entire parLapply() call to handle failures gracefully.
library(parallel)
cl <- makeCluster(2)
# Wrap risky code inside the worker function
safe_log <- function(x) {
tryCatch(
log(x),
warning = function(w) NA_real_,
error = function(e) NA_real_
)
}
clusterExport(cl, 'safe_log')
# -1 produces NaN warning, 'a' produces an error
input <- list(4, 9, -1, 'a', 16)
result <- parLapply(cl, input, safe_log)
cat(unlist(result), '\n') # 1.386 2.197 NaN NA 2.773
stopCluster(cl)Practical Parallel Pattern
Here is a complete, idiomatic parallel workflow: detect cores, create cluster, export dependencies, run computation, collect results, and always stop the cluster — even if an error occurs — using on.exit().
library(parallel)
run_parallel <- function(data, fn, n_workers = detectCores() - 1) {
cl <- makeCluster(n_workers)
on.exit(stopCluster(cl)) # guaranteed cleanup
clusterExport(cl, 'fn', envir = environment())
result <- parLapply(cl, data, fn)
result
}
# Example: bootstrap mean estimation
samples <- lapply(1:100, function(i) rnorm(50, mean = 5, sd = 2))
means <- run_parallel(samples, mean)
cat('Grand mean:', round(mean(unlist(means)), 3), '\n')
cat('95% CI: [',
round(quantile(unlist(means), 0.025), 3), ',',
round(quantile(unlist(means), 0.975), 3), ']\n')Quick Check
Which function should you use to guarantee that stopCluster(cl) is called even if an error occurs inside your parallel code?
Recap: parallel Package
Key takeaways:
detectCores()reports available cores; usedetectCores() - 1for the cluster sizemakeCluster(n)/stopCluster(cl)manage worker lifecyclesclusterExport()copies objects;clusterEvalQ()runs setup code on workersparLapply()/parSapply()are cross-platform parallel iteratorsmclapply()is Unix-only but faster to start due to forking- Use
on.exit(stopCluster(cl))for guaranteed cleanup - Parallelism pays off only for computationally heavy tasks
# Minimal reproducible parallel pattern
library(parallel)
cl <- makeCluster(max(1, detectCores() - 1))
on.exit(stopCluster(cl))
clusterSetRNGStream(cl, iseed = 42)
result <- parSapply(cl, 1:8, function(x) x^2 + rnorm(1, 0, 0.1))
cat(round(result, 2), '\n')
Learn R with an AI tutor — free
Write and run real code in your browser, get instant help from a 24/7 AI tutor, and pick up where you left off on the web or in the app.
- Courses
- 43
- Lessons
- 159
Frequently Asked Questions
Is the “parallel Package and detectCores()” lesson free?
Yes — any 3 lessons of the R Academy learning path, including “parallel Package and detectCores()”, are free to read in full here on the web. After that, CoddyKit PRO unlocks every lesson, plus interactive practice with a built-in code editor and a 24/7 AI tutor. The R Academy course includes 4 lessons in total.
What will I learn in “parallel Package and detectCores()”?
Launch forked or socket clusters and distribute work across CPU cores. You practise R Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start R Academy?
No prior experience is required. R Academy on CoddyKit is structured for beginners through advanced learners, so you can start here or from the beginning and move at your own pace. This is lesson 1 of 4.
How long does the “parallel Package and detectCores()” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this R Academy lesson?
Yes. Every R Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- parallel Package and detectCores()
- The future Framework
- furrr: Parallel purrr Operations
- Debugging and Load Balancing Parallel Code