Custom "Explore" Queries

Custom queries run additional experiment analyses on your existing data beyond what is in your main Results tab. Use them to gain deeper insights from experiments and feature rollouts, debug interesting results, or scope results to specific sub-groups. Custom queries let you filter or group metrics by event or user dimensions, or filter to a specific set of users to see how an experiment or launch affected those users.

Dimension loading timing for precomputed user dimensions

When viewing results for precomputed user dimensions (which are configured and run on a schedule), Statsig loads these dimensions through separate asynchronous explore queries. This means:

• The main experiment results appear first.
• Precomputed dimensions continue loading in the background and are available within a few minutes.
• This timing gap is most noticeable immediately after the first reload of the day.
• If you see "No dimensions available for this time range" for precomputed dimensions, wait a few minutes and refresh to check whether dimensions have finished loading.

Running a custom query

To run a custom query, go to the Explore tab within your experiment.

Custom Query fields:

• Metric(s): The metric(s) you want to analyze. You can select a single metric, a few metrics, or a Metric Tag. Adding a Tag will include all the metrics within that Tag in your Custom Query. There are three "default" metric selections included as shortcuts:
  • "Scorecard Metrics", all metrics included in your experiment setup's Primary and Secondary Metrics sections
  • "Primary Metrics"
  • "Secondary Metrics"
• Metric Filter: With metrics selected, you can filter metrics by either Event or User dimensions using the "Add Filter" dropdown. For example, to view experiment results for Canadian users only, filter to "Country = CA".

• Group By: You can group your custom query results by either an Event or User dimension. Whereas custom query filters can be applied at the per-metric level, the Group By action is at the query level (so all included metrics have whatever Group By you select applied to them).
• Time Range for Metric Data: The date range you're running your analysis on. By default this is the "Full date range" of your experiment data.
• (Advanced) ID List Segment filters: You can choose an ID-list based Segment, and your results will only be calculated for users in that segment. This is useful when you did not log an important user dimension to filter on, or when you only need results for a sub-population defined in your own data warehouse.
  • This option can easily lead to erroneous and biased results. The segment must be defined based on the user's status before they were exposed to the experiment or feature gate.
  • You can also choose to exclude a certain ID list segment. For example, exclude users who have been retroactively identified as bad actors from your lifts analysis.
• (Advanced) Filter by Exposure Date: You can also filter the results by Exposure Date for more flexibility. You can include or exclude a date range, or in WHN, include/exclude users based on when they were first exposed to the experiment.
  • This is useful when your metrics have a novelty effect, delayed impact, or when you need to filter results to specific users. Use it cautiously because it can lead to biased results.

Viewing a custom query in Explore

These queries take a few minutes to run. After the query completes, Statsig sends an email notification and the results appear in the Query History section of the Explore interface. All historical queries across your team are stored there. You can assign a display name to a query inline for easier future identification.

Scheduling a custom query

To get a daily refresh of a custom query, schedule it directly from the Explore tab. Create the custom query, then select the "..." menu, then Schedule. The custom query runs daily and appears in the Scheduled tab of your Metric Lifts.

Reviewing custom query results

Custom query results resemble the main Results tab because the statistical methods are the same. The key difference is that a custom query result is a snapshot in time. After a query runs, Statsig saves the results and doesn't update them when more metric data is collected. To update results, run a new custom query or schedule custom queries to run on a regular cadence.

Sequential testing and custom queries

Because custom queries are snapshots in time, sequential testing adjustments apply to that specific analysis only. Running additional custom queries with more or less data (for example, more days or more unique users) changes the adjustments accordingly. Custom queries that meet the configured minimum Days or Unique Exposures may have no sequential adjustments applied.