Experiment Process Overview

The order of steps for running a Statsig experiment, from sizing it with a power analysis through configuring, implementing, testing, launching, and reading results.

Running an experiment on Statsig takes four kinds of work: sizing it, configuring it in the console, implementing it in your code, and checking it before and after you start. This page gives the order and links to the page that owns each step. If you'd rather follow one continuous tutorial that includes SDK setup, go to Run your first experiment.

You don't need every step. A simple A/B test on an existing metric needs the create, configure, and implement steps; the planning and pre-launch checks matter more as the cost of a wrong result goes up.

Before you create the experiment

Step	What it covers
Run a power analysis	Estimate how much traffic and how many days the experiment needs to detect the effect you care about. Run it first, because its output sets the target duration you enter later.
Choose a feature gate or an experiment	Decide whether you need to measure variant lift at all. A plain rollout doesn't need an experiment.

Create and configure

Step	What it covers
Create the experiment	Name it, state the hypothesis, pick the randomization ID type, and choose whether it runs in its own layer. Covers user-level, device-level, custom ID, and isolated experiments.
Configure the scorecard	Record the hypothesis and choose the primary and secondary metrics that measure it.
Set allocation and targeting	Choose how many eligible users enter the experiment and which users qualify.
Define groups and parameters	Name the variants and set the values your code reads. Do this before you start: group structure is frozen once the experiment runs.
Review the analysis settings	Set the target duration from your power analysis and choose the significance level Statsig applies to results.

Implement in your code

Step	What it covers
Implement the experiment	Read the experiment's parameters with `getExperiment`, log the events your scorecard metrics use, and test each variant in a lower environment.
Read parameters, not group names	Why checking a group name in code is an anti-pattern, and what the diagnostics stream's rules mean.

Check before you start

Step	What it covers
Set overrides	Force yourself or specific users into each group to confirm every variant works during development.
Review the quality score	Catch setup gaps, such as a thin hypothesis, before the experiment starts collecting data.

After you start

Step	What it covers
Monitor the experiment	Confirm assignment and logging are healthy: checks arriving, unit IDs matching, exposures balanced, no crossover.
Read the results	Interpret exposures, the scorecard, and metric lifts once Statsig computes them.
End the experiment	Ship a group, stop assignments, conclude and defer, abandon, or reset.

Reference

Page	What it covers
Manage an experiment	The detail page's tabs and the actions menu: clone, rename, save as template, share, export, archive, and delete.
Layers	How layers let several experiments share a user population without overlapping.
Holdouts	How to keep a population out of all experiments to measure cumulative impact.

Was this helpful?