Decision Framework
Use a structured decision framework in Statsig to evaluate experiment results, including success criteria, ship versus iterate logic, and reviewer sign-off.
A Decision Framework for experiment templates allows teams to standardize their interpretation of results and decision-making process. After configuration, it provides clear recommendations for which group to ship based on experimental outcomes. The framework highlights recommended actions based on metric movements, but doesn't enforce them.
Configure a decision framework
To configure a Decision Framework, find the experiment template you want under Settings → Templates, then navigate to the Decision Framework tab. Select one primary metric and one or more guardrail metrics to choose a recommended action based on metric performance.
You can configure three types of recommendations:
- Rollout Winning Group: An icon appears next to the winning group in the Make Decision button
- Discuss: A message appears on the experiment page recommending discussion before making a ship decision
- Do Not Roll Out: An icon appears next to the control group in the Make Decision button
Reviews for shipping negative results
Configure reviewers for when a shipping decision doesn't align with the Decision Framework recommendations. Toggle on "Require reviews when decision opposes recommended decision" and add reviewers in the dialog box.
After setup, Statsig requires a review in the following situations:
- Rollout Winning Group: Shipping a treatment group that isn't recommended
- Discuss: Shipping any treatment group
- Do Not Roll Out: No review is required
Was this helpful?