Introduction

Statsig Stats Engine

This section provides detailed view of the statistical methodology that powers Pulse. Delivering these results involves a series of calculations to identify the following:

• Metric deltas: The difference in metrics between the test and control groups.
• Confidence intervals: The gray, red, and green bars in Pulse. They represent the uncertainty in the metric deltas and depend on the standard error of the metrics.
• p-Values: A statistical measurement common in hypothesis testing. Represents the probability of observing a metric delta at least a extreme as the one we measured, under the assumption that the treatment has no impact on that metric.