Implement an Experiment

Implementing an experiment means reading its parameters in your code with getExperiment, logging the events your scorecard metrics use, and testing each variant in a non-production environment before you start it. Statsig generates the assignment code and runs the analysis on exposures and events; your code supplies the exposures and events. Use this page after you create the experiment in the console. To check a feature gate rather than read experiment parameters, follow the feature gate SDK guide instead.

To deploy an experiment:

1. Pull the experiment configuration in your application.
2. Log the events you want in your experiment results.
3. Test the experiment in development or another lower environment.
4. Start the experiment.

Pull experiment configurations from Statsig

The following snippet experiments on a product demo flow, where you might want to improve conversion through the funnel to demo completion. For full implementation details, go to the SDK documentation for your language, or work through the guide for your first A/B test.

const user = { userID: loggedInUserID };
const demoConfiguration = statsig.getExperiment(user, "demo_experience");

// use parameters to control the experience
if (demoConfiguration.get("show_banner", false)) {
  showBanner();
}

const title = demoConfiguration.get("title", "Start Demo");
banner.setTitle(title);

You can also view a code snippet for your experiment by selecting the code snippet button on the experiment page and choosing your SDK.

Log events for your scorecard

To get results for the events and metrics you care about, instrument the experience with event logging. You can also set up an event integration or a data warehouse import to send events to Statsig's stats engine. With the Statsig SDKs, your code might look like this:

statsig.logEvent(user, "demo_started");
...
statsig.logEvent(user, "demo_completed");

A few events let you measure how users move through a funnel in your product and experiment on those flows to increase conversion.

Test in a lower environment

After you start an experiment, you can't edit the groups without restarting it, because Statsig is already allocating users to each group. Test each experiment in a lower environment before you start it. On the experiment setup page, click Test, then select Enable for Environments. These environments should match your SDK environment setup. Testing in a lower environment together with overrides lets you set your own experiment group and test each variant.

After you enable the experiment for a lower environment, the experiment status changes from Not Started to Testing.

In the results section, you can track cumulative exposures and metric results collected from lower environments. Statsig displays results in aggregate across all lower environments and doesn't distinguish between individual environments. Statsig retains metric data and exposure data even when you repeatedly disable and re-enable the experiment for the lower environment.

When you need to re-test the experiment end to end, resalt the experiment in lower environments.

Start the experiment

After your experiment has metrics, parameters, and a hypothesis, and you've tested it in a lower environment, click Start. Your experiment is immediately live in production.