Decision Framework
Use a structured decision framework in Statsig to evaluate experiment results, including success criteria, ship versus iterate logic, and reviewer sign-off.
A decision framework standardizes how teams interpret experiment results and decide which group to ship. After you configure a decision framework, it recommends a ship action based on how your primary and guardrail metrics move. The decision framework highlights recommended actions but doesn't enforce them.
Configure a decision framework
To configure a Decision Framework, find the experiment template you want under Settings → Templates, then navigate to the Decision Framework tab. Select one primary metric and one or more guardrail metrics to choose a recommended action based on metric performance.
You can configure three types of recommendations:
- Rollout Winning Group: An icon appears next to the winning group in the Make Decision button.
- Discuss: A message appears on the experiment page recommending discussion before making a ship decision.
- Do Not Roll Out: An icon appears next to the control group in the Make Decision button.
Reviews for shipping negative results
Configure reviewers for when a shipping decision doesn't align with the Decision Framework recommendations. Toggle on "Require reviews when decision opposes recommended decision" and add reviewers in the dialog box.
After setup, Statsig requires a review in the following situations:
- Rollout Winning Group: Shipping a treatment group that isn't recommended
- Discuss: Shipping any treatment group
- Do Not Roll Out: No review required
Was this helpful?