Configure the Experiment Scorecard

An experiment's scorecard documents the hypothesis you're testing and the metrics that measure it. Configure it on the experiment's Setup tab after you create the experiment and before you start it. Statsig computes scorecard metrics daily, and they're eligible for advanced treatments such as CUPED and sequential testing.

The Setup page extends the information you entered when you created the experiment.

Scorecard fields

• Hypothesis: A rich-text field. Use its toolbar to apply headings, bold, italic, underline, strikethrough, links, text color, highlighting, bulleted and numbered lists, blockquotes, images, and code blocks.
• Primary Metrics: The metrics you expect the experiment to directly affect.
• Secondary Metrics: Metrics you monitor for unintended side effects. They aren't the primary focus of the experiment.

Keep the primary metric set small. For how a focused scorecard changes the way you read results, go to Best practices and avoiding false positives.

Write the hypothesis

The Hypothesis field includes a template that you can modify:

We believe that [change or feature] for [user segment] will [desired outcome] because [reason or insight]. We will measure success using [primary metric], and monitor [guardrail metrics] to ensure no negative impact.

Hypothesis Advisor

Hypothesis Advisor gives instant feedback on the hypothesis you enter and flags what's missing. Admins can set custom requirements that Statsig uses to guide experimenters toward more complete hypotheses.

This Statsig AI feature is off by default. Enable it from Settings > Experiment > Project > Statsig AI. On the same settings page, configure any custom requirements you want Hypothesis Advisor to enforce (for example, "Strongly recommend that a validation plan be mentioned").