Skip to main content

Build your first Device-level Experiment

In cases where you're unable to establish the user's identity and are unable to leverage a user ID as the unit of randomization for your experiment, Statsig's device-level experiments enable you to design experiments using a Statsig-generated stable ID, saved on the user's device in local storage.

Setting up a device-level experiment is the same as creating a traditional user-level experiment except that you simply have to set the ID type on the experiment to Stable ID when creating the experiment. You can pass your user object with or without a user ID when initializing the client SDK - the SDK will generate a stable ID for that user to use for evaluating these experiments.

Step 1 - Create a new experiment in the Statsig console​

Log into the Statsig console at https://console.statsig.com/ and navigate to Experiments+ in the left-hand navigation panel.

Click on the Create button and enter the name and description for your experiment. Click the checkbox for Use Stable ID check box and click Create.

Screen Shot 2022-06-26 at 10 43 49 AM

You can fill out the Experiment Scorecard with a hypothesis and primary/secondary metrics, just as you would with any other experiment.

Under the Groups card, add a parameter that you wish to experiment on. For now, let's just add a simple boolean "enabled":

Screen Shot 2022-06-26 at 10 49 05 AM

By default, the experiment will be allocated to 100% of your user base in the Allocation panel. Adjust this as necessary - you may wish to start with a smaller rollout percentage, and ramp up the allocation after proving out the new variant.

image

Finally, don't forget to click on the Save button at the top right hand side of the page to complete your experiment setup, and the Start button when you are ready to roll this out to production users!

Step 2 - Use your experiment configuration in the client-side of your application​

Once you have installed a client SDK of your choice (see Javascript example), you must initialize the SDK using your client SDK key as shown below. For device-level experiments, you do not have to include the user ID as you would with more traditional experiments if the user is not yet known/signed-in.

const user = {
    ip: "192.168.1.101",
    appVersion: "1.0.0",
    custom: {
        promoCode: "New30Off",
    }
};

await statsig.initialize('client-sdk-key', user);

Step 3 (Optional) - Add user identifiers for other experiments involving signed-in users​

Once the user creates an account or logs into your application, you can also call the updateUser API with user ID and any other new user attributes as shown below. This will make sure the user is in the right groups for your logged in/userID based experiments and feature gates, but will not affect any Stable ID based evaluations.

user.userID = realUserID;
user.email = signUpEmail;

await statsig.updateUser(user);