Interaction Detection
This feature is only available to warehouse native customers now.
What is Interaction Detection
When you run overlapping experiments, it is possible for them to interfere with each other. Interaction Detection lets you pick two experiments and evaluate them for interaction. This helps you understand if people exposed to both experiments behave very differently from people who're exposed to either one of the experiments.
Should I worry about it?
Our general guidance is to run overlapping experiments. People seeing your landing page should experience multiple experiments at the same time. Our experience is echoed by all avid experimenters (link). Teams expecting to run conflicting experiments are typically aware of this and can avoid conflicts by making experiments mutually exclusive via Layers (also referred to as Universes).
How to use it
You can find it from Experiment -> Pulse Results -> Explore -> Interaction Effect Detection
Interaction Detection is available as a special custom query on an experiment. You can pick a second experiment to analyze (along with metrics) and we will show you a quick summary.
In each section:
- By Groups: gives you the unit counts based on the collective slice of group assignment
- Metric Summary Results: shows you the estimated intervals of difference in metric lift
- Overlapping Unique Users: offers an overview of the general traffic intersection of the two experiments
