---
title: Best Practices and Avoiding False Positives
description: "Best practices for interpreting Statsig experiment results, including avoiding common biases, reading lift correctly, and trusting statistical significance."
product: general
last_updated: 2025-09-18
token_estimate: 517
---
# Best Practices and Avoiding False Positives

> For AI agents: a documentation index is available at [/llms.txt](/llms.txt). Append `.md` to any page URL for markdown, or send `Accept: text/markdown`.

Sound experiment readouts start from a written hypothesis, a small set of Scorecard metrics, and skepticism toward borderline p-values. Follow these practices when you read a Scorecard. For why a result can flip from significant to neutral as data accumulates, refer to the [Pulse FAQ](https://docs.statsig.com/experiments/interpreting-results/faq).

## Start with a hypothesis

Decide before you open the Results tab which metrics you expect the change to move, what else could have happened, and what would signal that something went wrong.

## Pick a small set of key metrics

Choose a few metrics directly tied to your hypothesis that establish whether the experiment worked. More than a handful of key metrics usually indicates an ill-defined hypothesis or unfocused experimentation.

## Avoid cherry-picking results

Don't select three metrics that look good while ignoring two that don't, and don't pick "good" or "bad" numbers with no connection to your hypothesis. A statistically significant result should have a plausible explanation, and a false positive is one plausible explanation.

## Look for consistent supporting effects

Multiple independent effects that fit one plausible story lend credibility to the observed effects, even when individual p-values are borderline.

## Expect false positives

Examining many metrics increases the false positive rate. At a 95% confidence interval (5% significance level), expect about one statistically significant metric in twenty from random chance alone, and more if you include borderline metrics (for example, p = 0.06). Be suspicious of statistically significant results with borderline p-values.

## Look beyond your hypothesis

Check for other effects, tradeoffs, and unexpected behaviors. These findings show how users interact with your product and often lead to follow-up experiments and new ideas.