R Academy · Lesson

Benchmarking with microbenchmark

Compare multiple implementations statistically with microbenchmark().

Lesson 4 of 413 steps

Benchmarking with microbenchmark is a free R Academy lesson on CoddyKit. This is lesson 4 of 4. Any 3 lessons of this learning path are free to read in full — after that, CoddyKit PRO unlocks every lesson, plus hands-on practice with a built-in code editor and a 24/7 AI tutor. It is part of the R Academy learning path, and your progress syncs across the web and the CoddyKit app. The R Academy course includes 4 lessons in total.

Why microbenchmark?

system.time() has millisecond resolution and is unreliable for fast operations. The microbenchmark package runs expressions hundreds of times, handles warm-up, and reports nanosecond-resolution statistics — making it the right tool for comparing similar implementations.

Basic microbenchmark Usage

Pass named expressions to microbenchmark(). Each argument name becomes the label in the output. The times argument controls how many times each expression is evaluated.

# library(microbenchmark)
# x <- 1:10000
#
# microbenchmark(
#   loop = {
#     s <- 0
#     for (v in x) s <- s + v
#   },
#   vectorized = sum(x),
#   times = 200L
# )

Choosing the times Argument

More repetitions give more stable estimates but take longer. General guidelines:

Fast expressions (microseconds): times = 1000L or more
Moderate (milliseconds): times = 100L
Slow (seconds): times = 10L or fewer

The default is times = 100L, which is a good starting point.

# library(microbenchmark)
# microbenchmark(
#   fast_op = sqrt(2),
#   times = 10000L   # many reps for a nanosecond-scale operation
# )
# microbenchmark(
#   slow_op = sort(rnorm(1e6)),
#   times = 10L      # fewer reps for second-scale operation
# )

Specifying the unit Argument

Use the unit argument to display results in a convenient scale:

'ns' — nanoseconds (for very fast operations)
'us' — microseconds
'ms' — milliseconds
's' — seconds
'relative' — ratio to the fastest expression

# library(microbenchmark)
# x <- runif(1000)
#
# microbenchmark(
#   sapply_sqrt = sapply(x, sqrt),
#   vectorized  = sqrt(x),
#   times = 500L,
#   unit = 'us'   # display in microseconds
# )

Interpreting the Summary Output

microbenchmark prints a summary table with these columns:

min — fastest single run
lq / mean / median / uq — lower quartile, mean, median, upper quartile
max — slowest single run
neval — number of evaluations

Use median as the primary comparison metric — it is robust to occasional GC pauses that inflate max and mean.

# Example summary output (unit: microseconds):
#
#         expr    min     lq   mean  median    uq    max neval
#         loop 1203.1 1245.3 1301.7  1262.4 1310.1 2100.8   100
#   vectorized    2.1    2.3    2.9     2.4    2.6   18.3   100
#
# => vectorized is ~525x faster at median
cat('Always compare medians, not means, for microbenchmark results
')

summary() on a microbenchmark Object

Calling summary(mb) on a stored microbenchmark result returns a data frame you can inspect programmatically. You can also change the unit in the summary call.

# library(microbenchmark)
# x <- rnorm(5000)
# mb <- microbenchmark(
#   vapply_abs  = vapply(x, abs, numeric(1)),
#   base_abs    = abs(x),
#   times = 200L
# )
# s <- summary(mb, unit = 'ms')
# print(s[, c('expr', 'min', 'median', 'max')])

autoplot() for Visual Comparison

autoplot(mb) uses ggplot2 to draw a violin or box plot of timing distributions across expressions. This makes it easy to see not just median differences but also variability and overlap between alternatives.

# library(microbenchmark)
# library(ggplot2)
#
# x <- 1:50000
# mb <- microbenchmark(
#   loop   = { s <- 0; for (v in x) s <- s + v },
#   vapply = vapply(x, identity, numeric(1)),
#   vec    = sum(x),
#   times  = 100L
# )
# autoplot(mb)  # opens ggplot2 violin chart

Comparing Loop vs vapply vs sapply

A classic benchmark: applying a function element-wise using a for loop, sapply(), or vapply(). vapply() is faster than sapply() because it pre-allocates the result vector. Both are slower than fully vectorized code.

# library(microbenchmark)
# x <- runif(5000, 1, 100)
#
# mb <- microbenchmark(
#   for_loop = {
#     r <- numeric(length(x))
#     for (i in seq_along(x)) r[i] <- log(x[i])
#   },
#   sapply_log  = sapply(x, log),
#   vapply_log  = vapply(x, log, numeric(1)),
#   vec_log     = log(x),
#   times = 200L, unit = 'us'
# )
# print(mb)

Checking for Correctness First

Before benchmarking, verify that all expressions return identical results. A faster but incorrect implementation is useless. Use identical() or all.equal() to compare outputs.

# x <- runif(1000)
# r1 <- sapply(x, sqrt)
# r2 <- sqrt(x)
# r3 <- vapply(x, sqrt, numeric(1))
#
# stopifnot(isTRUE(all.equal(r1, r2)))
# stopifnot(isTRUE(all.equal(r1, r3)))
# cat('All three produce identical results -- safe to benchmark
')
cat('Always verify correctness before comparing speed
')

Benchmarking with setup Argument

Use the setup argument to run code once before the timed expressions. This avoids including data-creation time in the benchmark when the data creation is not what you are measuring.

# library(microbenchmark)
#
# microbenchmark(
#   sort_base  = sort(x),
#   sort_order = x[order(x)],
#   setup = { x <- rnorm(10000) },
#   times = 100L
# )
# Each iteration refreshes x via setup, then times sort_base and sort_order

Reporting Benchmarks in Analysis

When sharing benchmark results, always report:

The R version and platform
Package versions
The times value used
The data size benchmarked

Timings are not portable across machines — report ratios, not absolute numbers, when comparing implementations.

cat('R version    :', R.version$version.string, '
')
cat('Platform     :', R.version$platform, '
')
cat('Logical cores:', parallel::detectCores(), '
')

Quick Check: microbenchmark Metric

Which summary statistic from microbenchmark output is the most reliable for comparing two implementations?

Answer it in the interactive lesson

microbenchmark Recap

microbenchmark is the standard tool for rigorous micro-benchmarking in R:

Pass named expressions and set times to control repetitions
Use unit = 'us' or 'ms' for readable output
Compare medians — they are robust to GC outliers
Use autoplot() to visualize timing distributions
Verify correctness with all.equal() before benchmarking

Free to start

Learn R with an AI tutor — free

Write and run real code in your browser, get instant help from a 24/7 AI tutor, and pick up where you left off on the web or in the app.

Courses: 43
Lessons: 159

Start learning free View the full course

Frequently Asked Questions

Is the “Benchmarking with microbenchmark” lesson free?

Yes — any 3 lessons of the R Academy learning path, including “Benchmarking with microbenchmark”, are free to read in full here on the web. After that, CoddyKit PRO unlocks every lesson, plus interactive practice with a built-in code editor and a 24/7 AI tutor. The R Academy course includes 4 lessons in total.

What will I learn in “Benchmarking with microbenchmark”?

Compare multiple implementations statistically with microbenchmark(). You practise R Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start R Academy?

No prior experience is required. R Academy on CoddyKit is structured for beginners through advanced learners, so you can start here or from the beginning and move at your own pace. This is lesson 4 of 4.

How long does the “Benchmarking with microbenchmark” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this R Academy lesson?

Yes. Every R Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to R Academy