MarkdownReporter

MarkdownReporter writes results to a .md file as a formatted table. It is part of the core NBenchmark package with no additional dependencies.

Markdown output is a good choice for committing results to source control, attaching to pull requests, or including in documentation.

Setup

using NBenchmark.Reporters;

// Default - writes to the current directory with auto-naming
.WithReporter(new MarkdownReporter())

// Explicit directory
.WithReporter(new MarkdownReporter("results/"))

// Explicit directory and filename
.WithReporter(new MarkdownReporter("results/", "benchmarks.md"))

Constructor

MarkdownReporter(string outputDirectory = ".", string? fileName = null)

• outputDirectory - The directory to write the file to. Created automatically if it does not exist. Must be under the current working directory.
• fileName - When null (the default), the reporter generates a timestamped filename to avoid overwriting previous runs. When specified, the exact filename is used (no counter or timestamp is appended).

Auto-naming

When fileName is not provided, the reporter generates a filename that includes the UTC timestamp and a per-process counter:

benchmark-results-20260606-034000-001.md

The counter increments each time ReportAsync is called within the same process, so multiple suite runs produce separate files instead of overwriting each other.

Explicit filename

Pass a fileName when you want a stable output path (e.g. for CI scripts that expect a known filename):

new MarkdownReporter("results/", "BENCHMARKS.md")

When an explicit fileName is provided, subsequent calls to ReportAsync overwrite the same file.

Output format

## Benchmark Results

> **2026-06-06 03:40:00 UTC** · 40 warmup · 190 measured · realistic profile

### Comparison

| Benchmark | Median | Mean | Ops/s | Ratio | Scale | Sig | Magnitude | Alloc/op |
|---|---:|---:|---:|---:|---:|---:|---:|---:|
| Compute | 300.0 ns | 275.3 ns | 3.64 Mops/s | **0.75x** | `############` | ✓ | large | - |
| **Baseline** _(baseline)_ | 400.0 ns | 375.8 ns | 2.66 Mops/s | _baseline_ | `################` | - | - | - |

> The real `MarkdownReporter` emits Unicode block characters (`█`) in the `Scale` column. They are replaced with `#` above so the example stays aligned in source form.

### Precision & Tail Latency

| Benchmark | Error (±CI) | StdDev | CV | P95 | P99 |
|---|---:|---:|---:|---:|---:|
| Compute | ±16.2 ns (5.89%) | 85.9 ns | 31.22% | 500.0 ns | 500.0 ns |
| Baseline | ±21.6 ns (5.75%) | 114.3 ns | 30.43% | 500.0 ns | 900.0 ns |

Percentile columns (P95, P99, etc.) are dynamic -- they appear only when the corresponding percentiles are configured via `MeasurementOptions.ReportedPercentiles` or the `--percentiles` CLI flag. With the default set of `[0.50, 0.95, 0.99, 0.999, 1.0]`, columns P95 and P99 are emitted in the tail-latency table (P50 is already shown as Median; Max appears as a separate stat).

---

### Interpretation

**Omnibus**: not run (fewer than 3 comparable groups).

- Significance: Mann-Whitney U (p < 0.05)
- Outliers: IQR fence (1.5×)
- Effect metric: Cliff's δ (Romano neg/small/med/large labels)

When three or more benchmarks are compared, the Sig column shows the post-hoc pairwise verdict (candidate versus baseline, Holm-Bonferroni corrected) and the Interpretation section includes an omnibus line summarising the Kruskal-Wallis verdict across all groups:

**Omnibus (Kruskal-Wallis)** across 3 groups: H(2) = 7.20, p = 0.027 → significant

Columns

Column	Description
Benchmark	Benchmark name.
Median	Median timing.
Mean	Arithmetic mean.
Ops/s	Mean operations per second (1e9 / Mean when timing is in nanoseconds). - for errored or dry-run results.
Ratio	Speed relative to the baseline.
Scale	Visual bar scaled to the slowest successful benchmark.
Sig	✓ = significant, ✗ = not significant, - = not applicable.
Magnitude	Strategy-defined qualitative effect label. With the built-in Mann-Whitney tests this is Cliff's delta classified by Romano (2006): neg (abs(δ) < 0.147), small (< 0.33), med (< 0.474), large (≥ 0.474). - for the baseline or when significance is not tested. See Cliff's delta.
Alloc/op	Mean bytes allocated per iteration, or - if not measured.

Notes

• Results are sorted by median (fastest first).
• For parameterized benchmarks, one column per parameter appears after Benchmark (which shows the base method name). When a single method is swept across parameter values, the Ratio column reports each point's scaling factor relative to the fastest point (the reference, shown as baseline), while Sig and Magnitude stay -. When a parameter group holds competing benchmarks, Sig and Magnitude carry the usual within-group significance and effect.
• Errored benchmarks are listed with a - in the Error, Ratio, and Sig columns. The Median, Mean, and StdDev columns show 0.0 ns. Percentile columns show empty cells.
• The output directory is created automatically if it does not exist.
• The report order is: Comparison -> Precision & Tail Latency -> (optional) Distribution Details -> Interpretation -> (optional) Warnings.
• In Standard mode (--detail standard or WithDetail(ReportDetail.Standard)), the full multi-section output is shown: comparison table, Precision & Tail Latency, and Interpretation.
• In Advanced mode (--detail advanced or WithDetail(ReportDetail.Advanced)), a per-benchmark details section is appended after the table showing quartiles, fences, CI, margin percent, CV, skewness, kurtosis, MAD, and allocation breakdown, followed by an auto-tuned: … line summarising the adaptive loop's decisions (resolved samples × ops-per-sample, warmup length, achieved CI half-width).

Using with Benchmark (Single mode)

var result = Benchmark.Run(() => MyMethod());
await result.ToMarkdownAsync("results/");
await result.ToMarkdownAsync("results/", "benchmarks.md");

CLI usage (BenchmarkHarness)

dotnet run -- --reporter markdown
dotnet run -- --reporter markdown --output ./results

When --output is specified, files are written inside that directory.