---
title: Bayesian Experiments
description: "Learn about Bayesian A/B testing on Statsig, including informative priors and implementation details."
product: general
lang: en
token_estimate: 1007
---
# Bayesian Experiments

> For AI agents: a documentation index is available at [/llms.txt](/llms.txt). Append `.md` to any page URL for markdown, or send `Accept: text/markdown`.

Bayesian mode reports each metric as a chance to beat control and an expected loss instead of a p-value, and can combine an informative prior with the observed data. Use it when stakeholders need probability readouts, or when you have reliable priors from past experiments. Experiments use frequentist analysis by default.

Statsig offers three analysis methods, which you choose with the _Analytics Type_ setting under Advanced Settings on the experiment Setup tab. Use frequentist analysis with [sequential testing](https://docs.statsig.com/experiments/advanced-setup/sequential-testing) for standard p-values and confidence intervals that stay valid when you read results early. Use [SPRT](https://docs.statsig.com/experiments/advanced-setup/sprt) when you want unlimited peeking and the option to accept the null hypothesis as well as reject it. Use Bayesian mode for chance-to-beat and expected-loss readouts, with optional priors from past experiments.

## Turn on Bayesian mode

To switch an experiment to Bayesian mode, set _Analytics Type_ to **Bayesian** under Advanced Settings on the experiment Setup tab. You can't change the analytics type after the experiment starts.

![Bayesian experiment configuration interface](https://docs.statsig.com/images/experiments/advanced-setup/bayesian/c9c01a57-fe13-47a9-b734-20d6e8d715a4.png)

Pulse scorecard results and deep-dive analysis both reflect Bayesian statistics.

![Bayesian scorecard results showing credible intervals](https://docs.statsig.com/images/experiments/advanced-setup/bayesian/be912632-6200-4408-977c-92f48dfdd7bc.png)

![Bayesian deep dive analysis interface](https://docs.statsig.com/images/experiments/advanced-setup/bayesian/c9214142-d11f-48c8-92a4-53581bbc498c.png)

## Informative priors

In Bayesian mode, you can set a prior belief on the relative average treatment effect. Statsig combines the prior distribution with the observed data and displays the prior-adjusted results. To turn on priors, select **Use informative priors**.

![Informative priors configuration interface](https://docs.statsig.com/images/experiments/advanced-setup/bayesian/0aa0a52c-4f97-42af-82dd-4d26dd1de7c0.jpg)

## Deriving a prior from historical data

When you use informative priors, make sure your organization understands how the prior influences the experiment results, and base the prior on domain knowledge. Two common patterns:

- Use the $AVG(\text{average treatment effect})$ of past experiments with a similar setup and population as the prior mean, and the standard deviation of those effects, or a multiple of it, as the prior standard error.
- Use the $AVG(\text{observed standard error})$ of past experiments as the prior standard error.

## How Statsig computes the posterior

Denote $\mathcal{N}(ATE_{prior}, STE_{prior}^2)$ as the prior distribution, where $ATE_{prior}$ is the average treatment effect and $STE_{prior}$ is the standard error. Denote $\mathcal{N}(ATE_{observed}, STE_{observed}^2)$ as the observed distribution.

Statsig calculates the posterior distribution as:

$$
ATE_{post} =
\frac{
\frac{ATE_{prior}}{STE_{prior}^2} +
\frac{ATE_{observed}}{STE_{observed}^2}
}{
\frac{1}{STE_{prior}^2} +
\frac{1}{STE_{observed}^2}
}
$$

$$
STE_{post}^2 =
\frac{1}{
\frac{1}{STE_{prior}^2} +
\frac{1}{STE_{observed}^2}
}
$$

If you don't specify a prior, Statsig uses $\mathcal{N}(0, \infty)$ as the prior distribution $\mathcal{N}(ATE_{prior}, STE_{prior}^2)$.

## Bayesian statistics glossary

Bayesian A/B tests use different terminology from the frequentist framework.

| Term | Definition |
| --- | --- |
| Credible interval | The interval that contains the true parameter at the given probability. |
| Chance to beat | The probability that the test group is better than control. |
| Expected loss | The average potential risk if you ship the test variant. |