Working with experiments
Similar to A/B tests, Experiments provides the following features to run and analyze experiments with ease:
- Automatically compute the sample size required to achieve statistical significance for a given experiment
- Configure allocations to define what proportion of eligible users are exposed to each variant in the experiment
- Review the Metrics Lifts and confidence intervals for all your business metrics
- Choose confidence intervals ranging from 80% to 99% depending on the confidence you need for your decision; a larger lift (say 20%) generally requires a lower confidence interval (about 80%), while a smaller lift (say 1%) generally requires a higher confidence interval (95%)
- Analyze Metrics Lifts for different segments by filtering for dimensions such as operating system and browser version
- Use the Statsig client or server SDK of your choice to implement the experiment in your application
Experiments offers additional features to accelerate and scale your experiments:
- Create multiple variants as part of the same experiment (A/B/n tests)
- Select the key metrics that you want to prioritize for your hypothesis
- Define layers to exclude the users participating in one experiment from other related experiments