To create custom metrics, navigate to Metrics from the left-hand navigation panel, then to the Metrics Catalog tab. Tap on the Create button.
Statsig supports four types of custom metrics:
|Event Count||Total count of events filtered by the value and metadata properties of an event type||Add to Cart event filtered by category type|
|User Count||Number of unique users that trigger events filtered by the value and metadata of an event type||Active Users based on their views of a product category|
|Aggregation||Sum or Average of the value or metadata property of an event type||Total Revenue|
|Ratio||Rates (e.g. cart conversion rate, purchase rate), Normalized Values (e.g. sessions per user, items per cart)||Cart Conversion Rate, Sessions per User|
Statsig computes custom metrics on a per day basis for your Metrics dashboard, and rolled up for the duration of the experiment in your Pulse Results delivered with your Feature Gates and Experiments. After you create a custom metric, it will not populate until the next day (and will not backfill to previous days). Statsig will only calculate it moving forward from the creation date.
1. Event Count Metrics
Here's an example of setting up a custom event metric to count the number of add_to_cart events filtered by a metadata property called value, which carries the price of the item added to the cart. As this example specifies the ID Type as userID, Statsig will compute this metric as part of the test group that the corresponding user is assigned to in an experiment.
If you select the ID Type as stableID, Statsig will compute this metric in the test group that the corresponding device is assigned to in an experiment. When you select more than ID Type, Statsig will compute this metric for each type of ID Type that you specify.
2. User Count Metrics
The example below creates a metric to count the number of unique users who viewed a product in the books category and that was priced under $10.
Two time window options are available for user count metrics
- Daily Participation Rate: This works just like the default event_dau metrics. It counts the total number of days that a user has the selected event, divided by the number of days the user is in the experiment. The result is a metric value between 0 and 1, which represents the probably of a user having the event on a daily basis. It works best for events that are expected to occur repeatedly for a given user.
- One Time Event: This checks if a user has the selected event at least once during their time in the experiment. The result is a binary metric with value 0 or 1 for each user. This is ideal for events that are only expected once per user, such as sign up events.
3. Aggregation Metrics
The example below shows a Total Revenue metric that sums the value associated with all purchase events.
4. Ratio Metrics
The example below shows the creation of a Cart Conversion Rate metric. Here we use the unique users who triggered the purchase event as the numerator and the unique users who triggered the add to cart event in the denominator. Note that when calculating the numerator, we filter to only include users who also had the denominator event in the same day. So in the case of this metric, a user who only has purchase event on a given day without an add to cart on that same day will not count towards the numerator.
This pattern also applies to click through rates (CTR) in any part of a step-wise product journey (aka funnels). Statsig recommends using unique users in both the numerator and denominator for defining these kinds of metrics. As an example, when a user reloads a page multiple times but clicks only once, this corresponds to a 100% CTR (1 out of 1). Similarly, a user who loads a page once but clicks multiple times on a button should only count as 1 out of 1. This also solves for cases where users see an important button such as "Sign-up" multiple times a day, and we would still consider it a success if they click just once.
The example below shows creating a metric for Items per Cart. You can track the number of unique items added to a cart if you log an add_to_cart event for each item. For the numerator, select total event count. For the denominator, select unique users. As this metric is computed daily and only for users with a non-zero denominator, this metric can generate ratios such as 1/1, 2/1, 2/1, and 5/1 for individual users. When aggregated, this translates to 10/4 = 2.5 items per cart on average per day.
A Word of Caution
In experimentation, ratio metrics are a frequent source of misleading information. It's possible to see an increase in click through rate alongside a net decrease in total clicks (the opposite may also happen). This situation can occur if the number of unique users viewing a button (denominator) decreases. As a best practice, Statsig recommends tracking the numerator and denominator as independent metrics when monitoring ratio indicator. Ratio metrics are often subject to statistical noise and can be tricky to use for obtaining a statistically significant result.
You can create a custom funnel metric, from either the Custom Metrics Creation wizard in the Metrics Catalog or via the Charts tab.
Components of Funnel Metrics
Funnel metrics have a few components:
- Lineage: Surfaces the events used to generate the funnel
- Metric Value: Metric value represents the overall funnel conversion rate, or the percentage of users who complete a funnel (trigger the end event) relative to all users who start the funnel (trigger the starting event)
- Conversion rate between stages: This set of metrics track the percentage of users who triggered an event N relative to all users that triggered event N-1 in the funnel
After funnels are created and populated, you can view your funnel metric much like any other metric in Pulse. Additionally, you can expand the funnel metric to view Pulse performance at each step in the funnel.
In the example below, the Square variant shows a lift in the overall funnel conversion rate. Expanding the metrics to examine the entire funnel reveals two key insights:
- Both the Square and Circle variants show a lift in top-of-funnel DAU (Land Page View Start DAU). However, only the Square variant shows statistically significant increase in end-of-funnel DAU (Purchase Event End DAU).
- The overall funnel conversion rate improvement for Square is primarily due to the higher conversion from Checkout Event to Purchase Event stages in the funnel.