Create an Experiment

User-level experiments

To create a user-level experiment, follow these steps:

1. Log into the Statsig console at https://console.statsig.com/
2. Navigate to Experiments in the left-hand navigation panel
3. Click on the Create button
4. Enter the name and description for your experiment as shown in the figure below
5. By default, your experiment runs in its own Layer. A Layer allows you to manage multiple experiments and feature flags together. To add this experiment to an existing Layer, select Add Layer under Advanced in the experiment creation modal. To create a new Layer, select Create New Layer.
6. Click Create

Configure your scorecard

When running an experiment, it’s common to test a specific hypothesis using a set of key metrics. The Scorecard feature makes this easy by letting you enter your hypothesis and select both primary and secondary metrics.

• Primary Metrics are those you expect to be directly impacted by the experiment.
• Secondary Metrics are important to monitor to ensure there are no unintended side effects, but they aren’t the primary focus of your experiment.

Configure allocation and targeting

The Allocation and Targeting section is where most of your experiment configuration happens.

Allocation

For Allocation, enter the percentage of users you want to assign to this experiment. You can allocate up to 100% of eligible users. It’s good practice to start with a smaller percentage, verify the experiment’s stability, and then increase the allocation.

You can increase the allocation of your experiment anywhere from 0% to 100% at any time after experiment start; however, you can't decrease allocation without resetting your experiment, as this would cause biases in group allocation and pollute your metric results.

Targeting

To configure Targeting criteria, click to edit the Targeting section. You can either set new targeting criteria or use an existing Feature Gate. This limits the experiment to only the users who meet the defined conditions.

• For basic targeting criteria, creating it through Inline Targeting works well. (Select "Criteria: Everyone" to get started.)
• For more advanced targeting (for example, progressive rollouts) or if you want to maintain targeting criteria when you launch your experiment, reference an existing Feature Gate.

By default, no targeting criteria are set, so your experiment includes all allocated users within the defined Layer or exposed user base.

Configure your groups and parameters

When configuring Groups and Parameters, define your parameters first. These are the variables that control the behavior of the different experiment variants.

• Enter the values the experiment parameter takes for each variant. For more about the difference between Groups and Parameters, go to Groups vs. Parameters.

You can add additional groups by clicking the "+" next to the existing groups. The user allocation automatically adjusts as you add more groups.

You can also name, describe, and add variant images for each group under the Groups section. Only the parameters and values affect what users see; group names and descriptions aren't used in the experiment code.

Device-level and custom ID experiments

By default, experiments randomize users based on User ID. If you need to use a different ID type (for example, device-level), follow steps 1–4 from the "User-level Experiments" section, then:

1. Click the ID Type dropdown menu and choose the ID type you need.
2. Click Create

Afterward, continue with the same steps described above to finish configuring the experiment.

ID mapping capabilities

When running experiments, you may want to start with one ID type (like stableID for device-level targeting) but analyze results using events from another ID type (like userID for logged-in user metrics).

Warehouse Native: Supports ID mapping between different identifier types (for example, stableID to userID) through Entity Property Source configuration.

Cloud: Does not currently support mapping between different ID types. Experiments started with stableID only analyze events with stableID, and experiments started with userID only analyze events with userID.

For advanced ID mapping requirements, consider using Statsig Warehouse Native.

Isolated experiments

If you want to create an experiment that excludes users exposed to other experiments, follow steps 1–4 from the "User-level Experiments" section. Then:

1. Select Advanced options.
2. Select an existing Layer or create a new one.
3. Click Create.

Now, complete the rest of the experiment setup as described above.

Reusing experiment salts

For advanced use cases (for example, a series of related experiments that need to reuse the control and test buckets), Statsig exposes the ability to copy and set the salts used for deterministic hashing. This feature is intended for careful use and is available only to Project Administrators. It is available in the Overflow (...) menu in Experiments.

Significance level adjustments

By default, Experiment Results display with 95% confidence intervals and without Bonferroni correction. You can customize this during experiment setup or later when viewing results in Experiment Results.

• Bonferroni Correction: Apply this to reduce the risk of false positives in experiments with multiple test groups. The significance level (α) is divided by the number of test variants.
• Default Confidence Interval: Choose a lower confidence interval (for example, 80%) if you prefer faster results with higher tolerance for false positives, or use 95% for greater certainty.

Target duration

Target durations longer than 90 days: By default, Statsig computes Experiment Results for the first 90 days, though the experiment itself can run longer. Before setting a duration beyond 90 days, consider whether results past that period will still be relevant, and whether earlier data might already provide the insights you need.

After you set the target, you can track progress against the target duration/exposures in the experiment header. You also receive notifications through email and Slack (if integrated) when the target is reached.

Hypothesis Advisor

Writing good experiment hypotheses is key to a strong experimentation culture. Statsig gives instant feedback on experiment hypotheses, flagging what’s missing. Admins can set custom requirements, which Statsig uses to guide experimenters toward stronger, more complete hypotheses.

This Statsig AI feature is disabled by default. Enable it from Settings > Experiment > Project > Statsig AI. This is also where you configure any custom requirements you want Hypothesis Advisor to enforce (for example, "Strongly recommend that a validation plan be mentioned").