Statistics

This section explains how NBenchmark collects and analyses measurements. The Key Concepts page covers the practical side. For a practical guide to interpreting the output you see on screen, see Reading Your Results. These pages are for readers who want the full mathematical picture.

In this section

• Measurement - the measurement loop, timer resolution, and warmup sequence.
• Allocation Measurement - how per-iteration heap allocation is sampled.
• Outlier Trimming - IQR fence, MAD, fixed-quota modes, custom detectors, and the bimodal-distribution warning.
• Descriptive Statistics - mean, median, percentiles, standard deviation, confidence intervals, CV, distribution shape (skewness, kurtosis, MAD), and the complete BenchmarkResult field reference.
• Significance Testing - the Mann-Whitney U test for two groups and the Kruskal-Wallis omnibus test (with post-hoc pairwise Mann-Whitney U and Holm-Bonferroni correction) for three or more: why non-parametric, the algorithms, p-value interpretation, Cliff's delta effect size and Magnitude column, the MinimumPracticalEffect practical-significance gate, and custom tests.
• Diagnostics - runtime counters for GC collection counts, heap state, exceptions, and CPU time.
• Validation & Accuracy - how the numerical implementations are verified against SciPy and NumPy.