Skip to main content

Build your first Device-level Experiment

In cases where you're unable to establish the user's identity and are unable to leverage a user ID as the unit of randomization for your experiment, Statsig's device-level experiments enable you to design experiments using a Statsig-generated stable ID, saved on the user's device.

Setting up a device-level experiment is the same as creating a traditional user-level experiment except that you simply have to click the Use Device ID check box when you create the experiment and do not need to pass the user ID when initializing the client SDK in your application if you don't have it yet.

Step 1 - Create a new experiment in the Statsig console#

Log into the Statsig console at https://console.statsig.com/ and navigate to Experiments+ in the left-hand navigation panel.

Click on the Create button and enter the name and description for your experiment. Click the checkbox for Use Stable ID check box and click Create.

Screen Shot 2021-10-26 at 3 38 33 PM

By default, each experiment runs in its own layer. When you want to create an experiment that excludes any users exposed to other experiments, you will also want to click the checkbox for Add Layer. As this is our first experiment, enabling this isolation isn't relevant for us at this stage.

In the experiment Setup tab, under the Allocation panel enter the percentage of users that you want to allocate to this experiment. As this experiment will run in its own layer and you can enter a value up to 100%.

image

Staying in the experiment Setup tab, under the Variations panel, enter the variant names and click on Add Another Variant to enter more variants. You'll find that the split percentages between these variants automatically change to evenly distribute users between these variants.

image

Staying in the Variations panel, click on the pencil icon next the sample_parameter and enter the name of your experiment parameter and the type, and click Confirm

image

Enter the values that the experiment parameter will take for the control and treatment groups.

image

Enter the key metrics you want to track in this experiment

image

Don't forget to click on the Save button at the top right hand side of the page to complete your experiment setup.

Step 2 - Use your experiment configuration in the client-side of your application#

Once you have installed a client SDK of your choice (see Javascript example), you must initialize the SDK using your client SDK key as shown below. For device-level experiments, you do not have to include the user ID as you would with more traditional experiments if the user is not yet known/signed-in.

const user = {
ip: "192.168.1.101",
appVersion: "1.0.0",
custom: {
promoCode: "New30Off",
}
};
await statsig.initialize('client-sdk-key', user);

Step 3 (Optional) - Add user identifiers for other experiments involving signed-in users#

Once the user creates an account or logs into your application, you can also call the updateUser API with user ID and any other new user attributes as shown below.

user.userID = realUserID;
user.email = signUpEmail;
await statsig.updateUser(user);