StatReason

Exact Binomial Test

Test whether an observed proportion differs from a benchmark rate, using the exact binomial distribution at every step rather than the normal approximation the One-Proportion z-Test relies on.

What this answers

This test answers the same question as the One-Proportion z-Test, "is my observed proportion different from a fixed benchmark?", but without ever approximating the binomial distribution with a normal curve. That makes it the right choice with small samples or a benchmark near 0 or 1, exactly where the normal approximation breaks down.

How it is calculated

The p-value is computed directly from the binomial distribution's exact probability mass function under the benchmark rate p0, with no approximation step. For a two-sided test, this calculator sums the probability of every possible outcome count that is no more likely than the observed outcome, the standard "small-p-value" exact method. The accompanying confidence interval is the Clopper-Pearson interval, built from the same exact binomial relationship, guaranteed to have at least the stated coverage rather than only approximately.

Worked example

For 3 successes out of 15 trials against a benchmark of .1, the observed proportion is .2, higher than the benchmark, though not dramatically so given the modest sample size. The exact two-sided p-value and Clopper-Pearson interval both properly reflect the discreteness of the binomial distribution at this small n, something a normal approximation would only approximate.

Assumption audit

Calculated from your data: the observed proportion, the exact p-value from the full binomial distribution, and the exact Clopper-Pearson interval bounds.
Evidence to review: whether your sample is small or your benchmark is near 0 or 1; those are exactly the conditions where this exact test and the normal-approximation z-test can disagree meaningfully.
You must verify: that each trial is an independent observation and that the benchmark p0 was fixed in advance, not chosen after seeing your data.

What this result does not mean

A significant exact test means the observed count is unlikely under the benchmark rate at your threshold, not that the true rate is dramatically different; check the width of the Clopper-Pearson interval to judge the estimate's practical precision. The exact method does not make small-sample results more powerful, only more honestly calibrated.

Limitations

The Clopper-Pearson interval is known to be conservative (wider than strictly necessary) in exchange for its guaranteed coverage; this is a deliberate, documented tradeoff, not an error. With large samples and expected counts safely above 10, the One-Proportion z-Test gives a nearly identical answer with simpler arithmetic.