Get Started with Autotune AI

Autotune AI personalizes which variant each user sees based on their context features, automatically optimizing a single goal metric. Setting it up takes a few steps: create a contextual autotune in the Statsig console, choose the goal metric to optimize, define your variants, then fetch the assigned variant in your app code. Use it when you want to optimize per user instead of picking one winning variant for everyone.

Set up your Contextual Autotune

Configure your contextual autotune in the Statsig console. You can also configure it programmatically through the Statsig console API.

Create the contextual autotune

Log in to your Statsig console and navigate to Autotune under Experiments.

Select Create. Name your contextual autotune and, optionally, specify the goal so other users can understand the motivation behind it.

Set your autotune type to Contextual.

Configure optimization

If optimizing for discrete outcomes such as:

• User clicks
• Checkouts
• Actions

Choose Event Occurring as the outcome type. If optimizing for a continuous output such as:

• Revenue
• Latency

Choose Event Value and set the directionality. Also choose the field from your log or metric source (Warehouse Native) to use for the value.

For Warehouse Native customers, specify the metric source and any optional filters for your target event.

Statsig highly recommends wrapping contextual autotunes in an experiment, but doesn't require it. You can set this up before or after your contextual autotune. The experiment wraps autotune calls in code and measures the overall impact of using this contextual autotune in your project.

Training settings

The remaining settings have defaults, but you can tune them as needed:

• Exploration window: How long to serve traffic randomly to bootstrap bandit exploration.
• Attribution window: How long after a user sees your variant to count outcome events. If you set it to 1 hour, a user has 1 hour to take action after seeing the experience.
• Exploration rate: Controls how much the bandit favors exploration over exploitation. 0 uses only the best prediction without confidence intervals. 1 uses the 99.9% CI instead of the 95% CI for exploration.
• Long-term exploration allocation %: The percentage of traffic that Statsig always randomly assigns. Use higher values for contextual autotunes you plan to run for a long time, to help avoid model drift.
• Feature list: A list of features Statsig should use to train the model. This acts as a filter; if you don't set it, Statsig reads every custom attribute. The main use case is fetching the feature list through CAPI to understand which features a given contextual autotune requires for on-demand evaluation.

Set up your variants. Variants are configurations you fetch in code. For example, the configuration below sends a "red" value to your codebase, which you can pass to the color setting on a button.

Use the Contextual Autotune in code (Python example)

The following code assumes you have your server secret key. Before running Python, install the SDK:

pip install statsig-python-core

First, import and initialize Statsig:

from statsig_python_core import Statsig, StatsigUser

key = <your_key_here>
autotune_name = <autotune_name_here>

statsig = Statsig(key)
statsig.initialize().wait()

Then, create a user object and fetch your config:

user = StatsigUser('user_id', custom={'key1': 'value1', 'key2': 'value2'})
cfg = statsig.get_experiment(user, autotune_name)

Now you have your config and can apply it:

color = cfg.get_string("color", "default color")
print(f"Going to use {color} for my color now")

After you run this code, you can:

• Confirm that you fetched a value from one of your variants.
• Navigate to the diagnostics page of your autotune and see a log of the user ID along with the corresponding variant.

Your code is now serving personalized variants to your users.

Important considerations

Statsig requires a few hundred units to train a model and doesn't start training until those units' attribution window has elapsed. To test the functionality, you can simulate events to confirm things work as expected. Use logic like:

fetch_autotune_value()
if(user country == 'us'):
    log_click()

to conditionally send events and verify that the model picks up on the conditional behavior.