Ratio Metrics

Use cases

Ratio metrics provide more nuanced understanding: revenue went up and purchase volume increased, but did revenue per purchase increase in a meaningful way?

Ratio metrics also normalize metrics. For example, if you rent out devices like scooters, you might run a scooter-level experiment but want to measure average revenue per distinct rider. You can create a metric of SUM(revenue)/COUNT_DISTINCT(rider_id) to calculate this normalized metric and reduce the influence of scooters in popular areas that get many riders.

Calculation

At the unit level, ratio metrics calculate the unit-level aggregation of both component metrics.

At the group level, the mean is the total group value of the first metric divided by the total group value of the second metric.

The SQL for this calculation:

-- Denominator (Checkouts)
SELECT
  source_data.unit_id,
  exposure_data.group_id,
  COUNT(1) as denominator
FROM source_data
JOIN exposure_data
ON
  -- Only include users who saw the experiment
  source_data.unit_id = exposure_data.unit_id
  -- Only include data from after the user saw the experiment
  -- In this case exposure_data is already deduped to the "first exposure"
  AND source_data.timestamp >= exposure_data.timestamp
GROUP BY
  source_data.unit_id,
  exposure_data.group_id;

-- Numerator (Revenue)
SELECT
  unit_id,
  group_id,
  SUM(revenue) as numerator
FROM source_data
GROUP BY unit_id, group_id;

-- Group Level
SELECT
  group_id,
  SUM(numerator)/SUM(denominator) as mean
FROM denominator
-- full outer join depending on settings
LEFT JOIN numerator
USING (group_id)
GROUP BY group_id;

Methodology notes

Ratio metrics require adjustment because the numerator and denominator may covary at the unit level. Statsig uses the delta method to estimate this adjustment.

By default, Statsig treats ratio metrics as a conversion rate (unordered), counting numerator events only for units that also performed the denominator event. In Statsig Warehouse Native, you can configure this behavior through Advanced settings at the metric level.

Options

• Treat as conversion rate (unordered): includes numerator events only if the unit also performed the denominator event, regardless of order. Uncheck for a simple ratio, which counts numerator events for all units even if they never performed the denominator event.
• Cohort Windows (Numerator and Denominator): specify a window for data collection after a unit's exposure. For example, a 0-1 day cohort window counts only actions from days 0 and 1 after exposure. Select Only include units with a completed window to exclude units from pulse analysis until their cohort window has completed.
• Winsorization: specify a lower and/or upper percentile bound. You can set thresholds independently for the numerator and denominator. All values below the lower threshold or above the upper threshold are clamped to reduce the impact of outliers.
• Baked Metrics: specify how long a metric needs to mature. This is common for chargebacks or cancellations. Statsig delays loading the data until the window has elapsed, and calculates pulse results only after a unit's metric has matured.