Email AB Testing with Customer.io
Run email A/B tests with Statsig and Customer.io by sending exposure events from email sends to compare open, click, and conversion metrics.
Email campaigns are a critical tool for any marketing team. Finding the best-performing email template is an ideal use case for an A/B test. Statsig lets you run A/B tests on different parts of your email content. Because Statsig integrates with product analytics, you can run email experiments and measure business-level impact on product metrics.
This guide assumes you have an existing Statsig account. Go here to create a new free account if you don't already have one: https://statsig.com/signup
Step 1: Create an experiment
Create a new Experiment in the Statsig console. Enter a name and leave the remaining fields at their default values.
Step 2: Start the experiment
An experiment requires at least one parameter before it can start. Add a placeholder parameter now.
Save the experiment setup and select Start.
Copy the Experiment Name. You need it in the next step.
Step 3: Set up Exposure Webhooks
In your Customer.io campaign, create a Random Cohort Branch with a flow similar to the following:
In the Webhook actions, add a POST request similar to the following, with your API key and experiment name filled in:
Pass in any other custom IDs and user attributes inside the post body.
For each webhook, expose the correct group to attribute the branch to. The webhook shown above exposes the "Control" group.
In Statsig, you'll now have exposures for each of your experiment groups.
Holdouts
To use Statsig Holdouts with Customer.io, identify users who are part of a holdout using Customer.io's identify function: https://customer.io/docs/sdk/ios/identify/
When you call Customer.io's identify method, check a Statsig holdout gate and add an attribute to the user to mark them as being in a holdout.
In your campaign, create a True/False branch to check whether a user is in the holdout.
Was this helpful?