Introduction
Introduction to the statistical methods Statsig uses to analyze experiments, including hypothesis testing, confidence intervals, and variance reduction.
How the Statsig Stats Engine works
The Statsig Stats Engine powers the experiment results you read in Pulse. It runs a series of calculations to identify the following:- Metric deltas: The difference in metrics between the test and control groups.
- Confidence intervals: The gray, red, and green bars in Pulse. They represent the uncertainty in the metric deltas and depend on the standard error of the metrics.
- p-Values: A statistical measurement common in hypothesis testing. Represents the probability of observing a metric delta at least as extreme as the one measured, under the assumption that the treatment has no impact on that metric.
Was this helpful?