0Pricing
R Academy · Lesson

Benchmarking with microbenchmark

Compare multiple implementations statistically with microbenchmark().

Benchmarking with microbenchmark is a free R Academy lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the R Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Why microbenchmark?

system.time() has millisecond resolution and is unreliable for fast operations. The microbenchmark package runs expressions hundreds of times, handles warm-up, and reports nanosecond-resolution statistics — making it the right tool for comparing similar implementations.

Basic microbenchmark Usage

Pass named expressions to microbenchmark(). Each argument name becomes the label in the output. The times argument controls how many times each expression is evaluated.

# library(microbenchmark)
# x <- 1:10000
#
# microbenchmark(
#   loop = {
#     s <- 0
#     for (v in x) s <- s + v
#   },
#   vectorized = sum(x),
#   times = 200L
# )

Choosing the times Argument

More repetitions give more stable estimates but take longer. General guidelines:

  • Fast expressions (microseconds): times = 1000L or more
  • Moderate (milliseconds): times = 100L
  • Slow (seconds): times = 10L or fewer

The default is times = 100L, which is a good starting point.

# library(microbenchmark)
# microbenchmark(
#   fast_op = sqrt(2),
#   times = 10000L   # many reps for a nanosecond-scale operation
# )
# microbenchmark(
#   slow_op = sort(rnorm(1e6)),
#   times = 10L      # fewer reps for second-scale operation
# )

Specifying the unit Argument

Use the unit argument to display results in a convenient scale:

  • 'ns' — nanoseconds (for very fast operations)
  • 'us' — microseconds
  • 'ms' — milliseconds
  • 's' — seconds
  • 'relative' — ratio to the fastest expression
# library(microbenchmark)
# x <- runif(1000)
#
# microbenchmark(
#   sapply_sqrt = sapply(x, sqrt),
#   vectorized  = sqrt(x),
#   times = 500L,
#   unit = 'us'   # display in microseconds
# )

Interpreting the Summary Output

microbenchmark prints a summary table with these columns:

  • min — fastest single run
  • lq / mean / median / uq — lower quartile, mean, median, upper quartile
  • max — slowest single run
  • neval — number of evaluations

Use median as the primary comparison metric — it is robust to occasional GC pauses that inflate max and mean.

# Example summary output (unit: microseconds):
#
#         expr    min     lq   mean  median    uq    max neval
#         loop 1203.1 1245.3 1301.7  1262.4 1310.1 2100.8   100
#   vectorized    2.1    2.3    2.9     2.4    2.6   18.3   100
#
# => vectorized is ~525x faster at median
cat('Always compare medians, not means, for microbenchmark results
')

summary() on a microbenchmark Object

Calling summary(mb) on a stored microbenchmark result returns a data frame you can inspect programmatically. You can also change the unit in the summary call.

# library(microbenchmark)
# x <- rnorm(5000)
# mb <- microbenchmark(
#   vapply_abs  = vapply(x, abs, numeric(1)),
#   base_abs    = abs(x),
#   times = 200L
# )
# s <- summary(mb, unit = 'ms')
# print(s[, c('expr', 'min', 'median', 'max')])

autoplot() for Visual Comparison

autoplot(mb) uses ggplot2 to draw a violin or box plot of timing distributions across expressions. This makes it easy to see not just median differences but also variability and overlap between alternatives.

# library(microbenchmark)
# library(ggplot2)
#
# x <- 1:50000
# mb <- microbenchmark(
#   loop   = { s <- 0; for (v in x) s <- s + v },
#   vapply = vapply(x, identity, numeric(1)),
#   vec    = sum(x),
#   times  = 100L
# )
# autoplot(mb)  # opens ggplot2 violin chart

Comparing Loop vs vapply vs sapply

A classic benchmark: applying a function element-wise using a for loop, sapply(), or vapply(). vapply() is faster than sapply() because it pre-allocates the result vector. Both are slower than fully vectorized code.

# library(microbenchmark)
# x <- runif(5000, 1, 100)
#
# mb <- microbenchmark(
#   for_loop = {
#     r <- numeric(length(x))
#     for (i in seq_along(x)) r[i] <- log(x[i])
#   },
#   sapply_log  = sapply(x, log),
#   vapply_log  = vapply(x, log, numeric(1)),
#   vec_log     = log(x),
#   times = 200L, unit = 'us'
# )
# print(mb)

Checking for Correctness First

Before benchmarking, verify that all expressions return identical results. A faster but incorrect implementation is useless. Use identical() or all.equal() to compare outputs.

# x <- runif(1000)
# r1 <- sapply(x, sqrt)
# r2 <- sqrt(x)
# r3 <- vapply(x, sqrt, numeric(1))
#
# stopifnot(isTRUE(all.equal(r1, r2)))
# stopifnot(isTRUE(all.equal(r1, r3)))
# cat('All three produce identical results -- safe to benchmark
')
cat('Always verify correctness before comparing speed
')

Benchmarking with setup Argument

Use the setup argument to run code once before the timed expressions. This avoids including data-creation time in the benchmark when the data creation is not what you are measuring.

# library(microbenchmark)
#
# microbenchmark(
#   sort_base  = sort(x),
#   sort_order = x[order(x)],
#   setup = { x <- rnorm(10000) },
#   times = 100L
# )
# Each iteration refreshes x via setup, then times sort_base and sort_order

Reporting Benchmarks in Analysis

When sharing benchmark results, always report:

  • The R version and platform
  • Package versions
  • The times value used
  • The data size benchmarked

Timings are not portable across machines — report ratios, not absolute numbers, when comparing implementations.

cat('R version    :', R.version$version.string, '
')
cat('Platform     :', R.version$platform, '
')
cat('Logical cores:', parallel::detectCores(), '
')

Quick Check: microbenchmark Metric

Which summary statistic from microbenchmark output is the most reliable for comparing two implementations?

microbenchmark Recap

microbenchmark is the standard tool for rigorous micro-benchmarking in R:

  • Pass named expressions and set times to control repetitions
  • Use unit = 'us' or 'ms' for readable output
  • Compare medians — they are robust to GC outliers
  • Use autoplot() to visualize timing distributions
  • Verify correctness with all.equal() before benchmarking

Frequently asked questions

Is the “Benchmarking with microbenchmark” lesson free?

Yes — the full text of “Benchmarking with microbenchmark” is free to read here on the web, and the R Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the R Academy course, upgrade to CoddyKit PRO.

What will I learn in “Benchmarking with microbenchmark”?

Compare multiple implementations statistically with microbenchmark(). You practise R Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start R Academy?

No prior experience is required. R Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Benchmarking with microbenchmark” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this R Academy lesson?

Yes. Every R Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. system.time() and proc.time()
  2. Profiling Code with Rprof and profvis
  3. Vectorization for Speed
  4. Benchmarking with microbenchmark
← Back to R Academy