Introduction
Introduction to the statistical methods Statsig uses to analyze experiments, including hypothesis testing, confidence intervals, and variance reduction.
How the Statsig Stats Engine works
This section provides a detailed view of the statistical methodology that powers Pulse. Delivering these results involves a series of calculations to identify the following:- Metric deltas: The difference in metrics between the test and control groups.
- Confidence intervals: The gray, red, and green bars in Pulse. They represent the uncertainty in the metric deltas and depend on the standard error of the metrics.
- p-Values: A statistical measurement common in hypothesis testing. Represents the probability of observing a metric delta at least as extreme as the one measured, under the assumption that the treatment has no impact on that metric.
Was this helpful?