Mean Metrics

Use cases

Mean metrics calculate the mean value of a numeric column in your metric source. You most often use them on event-level data, for example to measure "Average Time to Load" or "Average Purchase Value".

Calculation

At the unit level, mean metrics SUM their value column, and COUNT records where the value column is non-null.

At the group level, Statsig calculates the mean as the SUM of the unit-level sums, and the SUM of the unit-level counts.

The SQL looks like the following:

-- Unit Level
SELECT
  source_data.unit_id,
  exposure_data.group_id,
  SUM(source_data.value_column) as value,
  COUNT(source_data.value_column) as records
FROM source_data
JOIN exposure_data
ON
  -- Only include users who saw the experiment
  source_data.unit_id = exposure_data.unit_id
  -- Only include data from after the user saw the experiment
  -- In this case exposure_data is already deduped to the "first exposure"
  AND source_data.timestamp >= exposure_data.timestamp
WHERE value_column IS NOT NULL
GROUP BY
  source_data.unit_id,
  exposure_data.group_id;

-- Group Level
SELECT
  group_id,
  SUM(value)/SUM(records) as mean
FROM unit_data
GROUP BY group_id;

Methodology notes

Options

• Metric Breakdowns
  • You can configure Metadata Columns to group results by, getting easy access to dimensional views in pulse results.
• Winsorization
  • Specify a lower and/or upper percentile bound to winsorize at. Statsig clamps all values below the lower threshold, or above the upper threshold, to that threshold to reduce the outsized impact of outliers on your analysis.
• Cohort Windows
  • You can specify a window for data collection after a unit's exposure. For example, a 0-1 day cohort window counts only actions from days 0 and 1 after a unit is exposed to an experiment.
    • Select Only include units with a completed window to remove units from pulse analysis for this metric until the cohort window has completed.
• Baked Metrics
  • Baked Metrics let you specify how long a metric needs to mature. Metric maturation is common for chargebacks or cancellations. Statsig delays loading the data until the window has elapsed, and only calculates pulse results for that metric after a unit's metric has matured.