StatReason

Two-Proportion z-Test

Compare two independent proportions, such as conversion rates from two groups, to see whether the observed difference is likely to reflect a real difference.

What this answers

This test answers "are these two independently observed proportions likely to reflect the same underlying rate, or is the difference between them larger than chance alone would produce?" A common use is comparing a conversion rate, response rate, or event rate between two independent groups.

How it is calculated

The z statistic uses a pooled standard error: both groups' successes and trials are combined into one pooled proportion under the null hypothesis that the two groups share the same true rate, then the observed difference is divided by the standard error implied by that pooled proportion. The confidence interval for the difference, however, uses the unpooled Newcombe-Wilson method (combining each group's own Wilson interval), which is more accurate than a simple pooled-variance interval, especially with smaller samples or proportions near 0% or 100%.

Worked example

For Group A with 60 successes out of 100 trials and Group B with 50 out of 100: the pooled proportion is 55%, giving a z statistic of about 1.42 and a two-sided p-value of about .155. At the conventional .05 threshold, this particular difference (60% versus 50%) is not statistically significant, illustrating that a 10-point difference is not automatically meaningful without accounting for sample size.

Assumption audit

Calculated from your data: each group's observed proportion, the pooled proportion under the null, and whether either group's counts are sparse enough to warrant caution (very few successes or very few trials).
Evidence to review: whether both groups' expected success and failure counts are reasonably large. With very small or very lopsided counts, the normal approximation behind this z-test becomes less reliable, and an exact test may be a better fit.
You must verify: that the two groups were sampled independently of each other and that assignment to Group A or B was not influenced by the outcome you are measuring.

What this result does not mean

A significant result means the data provide evidence the two rates differ at your chosen threshold, not that whatever distinguishes Group A from Group B caused that difference. A nonsignificant result means the comparison was inconclusive at that threshold, not proof the two true rates are identical.

Limitations

This test assumes independence both within and between groups. With very small trial counts or proportions extremely close to 0 or 1, consider an exact test rather than this normal-approximation-based z-test.