C# Academy · Lesson

Benchmarking with BenchmarkDotNet

Write accurate micro-benchmarks with BenchmarkDotNet, analyze throughput and allocations, and avoid common pitfalls.

Benchmarking with BenchmarkDotNet is a free C# Academy lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the C# Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Why BenchmarkDotNet?

Micro-benchmarking is surprisingly hard: JIT warm-up, GC pauses, CPU caches, and OS scheduling all introduce noise. BenchmarkDotNet handles all of this automatically and produces statistically reliable results.

Installing and First Benchmark

Add the NuGet package, create a class with [Benchmark] methods, and run with BenchmarkRunner.Run. Always run benchmarks in Release mode.

// dotnet add package BenchmarkDotNet

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

[MemoryDiagnoser]           // report allocations
[RankColumn]                // rank methods by speed
public class StringBenchmarks
{
    [Benchmark(Baseline = true)]
    public string Concatenation()
    {
        var s = "";
        for (int i = 0; i < 100; i++) s += i;
        return s;
    }

    [Benchmark]
    public string StringBuilder()
    {
        var sb = new System.Text.StringBuilder();
        for (int i = 0; i < 100; i++) sb.Append(i);
        return sb.ToString();
    }
}

// Program.cs (must run as Release):
BenchmarkRunner.Run<StringBenchmarks>();

Reading the Results Table

BenchmarkDotNet prints a results table with mean time, error, standard deviation, ratio, and memory allocation. Here's how to interpret each column.

// Example output:
// | Method        |     Mean |   Error |  StdDev | Ratio | Alloc |
// |-------------- |---------:|--------:|--------:|------:|------:|
// | Concatenation | 4,210 ns |  81 ns  |  76 ns  |  1.00 | 6.4KB |
// | StringBuilder |   312 ns |   4 ns  |   4 ns  |  0.07 | 1.6KB |

// Mean:  average time per operation
// Error: half of 99.9% confidence interval
// StdDev: standard deviation — high value = noisy environment
// Ratio: relative to Baseline=true method (1.00)
// Alloc: managed heap allocated per operation

// Rule of thumb:
// Error should be < 5% of Mean for reliable results
// Run on an isolated machine (no background tasks)

Parameterized Benchmarks

Use [Params] to run a benchmark across multiple input sizes. BenchmarkDotNet runs every combination and generates a separate row per parameter value.

[MemoryDiagnoser]
public class SearchBenchmarks
{
    [Params(10, 100, 1000, 10000)]
    public int N;

    private int[] _data = Array.Empty<int>();

    [GlobalSetup]
    public void Setup()
        => _data = Enumerable.Range(0, N).ToArray();

    [Benchmark]
    public bool LinearSearch()
        => Array.IndexOf(_data, N - 1) >= 0;

    [Benchmark]
    public bool BinarySearch()
        => Array.BinarySearch(_data, N - 1) >= 0;
}

Setup and Cleanup

Use lifecycle attributes to run one-time setup or per-iteration setup/cleanup. This keeps setup cost out of the measured benchmark time.

[MemoryDiagnoser]
public class DbQueryBenchmarks
{
    private NpgsqlConnection _conn = null!;
    private NpgsqlCommand _cmd = null!;

    [GlobalSetup]              // runs once before all benchmarks
    public void Setup()
    {
        _conn = new NpgsqlConnection("...");
        _conn.Open();
        _cmd = new NpgsqlCommand("SELECT id FROM orders LIMIT 100", _conn);
    }

    [IterationSetup]           // runs before EACH benchmark iteration
    public void IterationSetup() { /* per-iteration prep */ }

    [Benchmark]
    public int QueryOrders() => (int)_cmd.ExecuteScalar()!;

    [GlobalCleanup]            // runs once after all benchmarks
    public void Cleanup() { _cmd.Dispose(); _conn.Dispose(); }
}

Memory Diagnostics

[MemoryDiagnoser] reports managed heap allocations per operation. Pair it with [GcForce] options to control GC behavior during benchmarks.

[MemoryDiagnoser]  // adds Alloc column to results
public class AllocationBenchmarks
{
    [Benchmark]
    public int[] AllocateArray() => new int[1000];

    [Benchmark]
    public Span<int> UseStackalloc()
    {
        Span<int> span = stackalloc int[1000];
        return span; // WARN: for illustration; don't return stackalloc
    }

    [Benchmark]
    public int[] ArrayPoolRent()
    {
        var arr = System.Buffers.ArrayPool<int>.Shared.Rent(1000);
        System.Buffers.ArrayPool<int>.Shared.Return(arr);
        return arr;
    }
}

Diagnosers: ETW and Disassembly

BenchmarkDotNet supports multiple diagnosers. The DisassemblyDiagnoser shows the actual native assembly output, perfect for verifying JIT optimizations.

using BenchmarkDotNet.Diagnosers;

[DisassemblyDiagnoser(          // show generated ASM
    maxDepth: 2,
    printSource: true,
    exportGithubMarkdown: true)]
[MemoryDiagnoser]
public class VectorizationBenchmarks
{
    private float[] _data = new float[1024];

    [Benchmark]
    public float SumScalar()
    {
        float sum = 0;
        foreach (var v in _data) sum += v;
        return sum;
    }

    [Benchmark]
    public float SumVectorized()
        => System.Numerics.Vector<float>.Count > 1
            ? VectorSum(_data)
            : SumScalar();
}

Job Configurations

Jobs control benchmark execution environment — number of warmup iterations, invocations per iteration, .NET runtime version, and JIT settings. Use predefined or custom jobs.

using BenchmarkDotNet.Jobs;

// Compare JIT vs Native AOT performance:
[SimpleJob(RuntimeMoniker.Net90)]    // .NET 9 JIT
[SimpleJob(RuntimeMoniker.NativeAot90)] // .NET 9 AOT
[MemoryDiagnoser]
public class RuntimeComparison
{
    [Benchmark]
    public string SerializeJson()
    {
        var obj = new { Name = "Alice", Age = 30 };
        return System.Text.Json.JsonSerializer.Serialize(obj);
    }
}

// Custom job for more warmup:
[Config(typeof(MyConfig))]
public class PreciseBenchmarks
{
    private class MyConfig : ManualConfig
    {
        public MyConfig()
            => AddJob(Job.Default.WithWarmupCount(5).WithIterationCount(20));
    }
}

Exporting Results

BenchmarkDotNet exports results in multiple formats — Markdown (for GitHub PRs), CSV (for spreadsheets), JSON, and HTML charts. Export options are configured in the job config.

using BenchmarkDotNet.Exporters;
using BenchmarkDotNet.Exporters.Csv;

[Config(typeof(ExportConfig))]
public class MyBenchmarks { /* ... */ }

public class ExportConfig : ManualConfig
{
    public ExportConfig()
    {
        AddExporter(MarkdownExporter.GitHub);
        AddExporter(CsvExporter.Default);
        AddExporter(HtmlExporter.Default);
        // Results in: BenchmarkDotNet.Artifacts/results/
    }
}

// CLI usage to run specific benchmarks:
// dotnet run -c Release -- --filter "*StringBenchmarks*"
// dotnet run -c Release -- --list all  (list all benchmarks)

Real-World: Before/After Optimization

Using BenchmarkDotNet to validate a performance optimization — comparing naive vs pooled vs Span-based approaches.

[MemoryDiagnoser]
[RankColumn]
public class ParsingBenchmarks
{
    private const string Input = "2025-06-15T08:30:00Z";

    [Benchmark(Baseline = true)]
    public DateTime ParseViaString()
        => DateTime.Parse(Input);

    [Benchmark]
    public DateTime ParseExact()
        => DateTime.ParseExact(Input,
            "yyyy-MM-ddTHH:mm:ssZ",
            System.Globalization.CultureInfo.InvariantCulture);

    [Benchmark]
    public bool TryParse()
        => DateTime.TryParseExact(Input.AsSpan(),
            "yyyy-MM-ddTHH:mm:ssZ",
            System.Globalization.CultureInfo.InvariantCulture,
            System.Globalization.DateTimeStyles.None,
            out _);
}

Quick Check

Why must BenchmarkDotNet benchmarks be run with dotnet run -c Release?

Recap: Benchmarking with BenchmarkDotNet

Key takeaways:

Install via NuGet; mark methods with [Benchmark]; run with dotnet run -c Release
[MemoryDiagnoser]: adds allocation column; [RankColumn]: ranks by speed
[Params]: run across multiple input sizes to find algorithmic complexity
[GlobalSetup] / [GlobalCleanup]: one-time setup outside measured time
[DisassemblyDiagnoser]: inspect generated native ASM to verify JIT optimizations
Use Jobs to compare .NET 9 JIT vs Native AOT vs different iteration counts

Frequently asked questions

Is the “Benchmarking with BenchmarkDotNet” lesson free?

Yes — the full text of “Benchmarking with BenchmarkDotNet” is free to read here on the web, and the C# Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the C# Academy course, upgrade to CoddyKit PRO.

What will I learn in “Benchmarking with BenchmarkDotNet”?

Write accurate micro-benchmarks with BenchmarkDotNet, analyze throughput and allocations, and avoid common pitfalls. You practise C# Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start C# Academy?

No prior experience is required. C# Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Benchmarking with BenchmarkDotNet” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this C# Academy lesson?

Yes. Every C# Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to C# Academy