Advanced Bandit Usage

When and how to use Statsig multi-armed bandits to optimize for a single goal metric, including ideal use cases, limitations, and result interpretation.

Both contextual and non-contextual bandits are managed on Statsig's console, or through Statsig's console API for programmatic creation. Both bandit types use a common, streamlined API, so you can explore either use case without significant changes from using experiments.

Statsig's bandits provide the power of complex bandits through a simple interface. A key observation from development is that many such systems fail due to implementation issues.

No additional steps beyond a regular experiment check are required to use a bandit. For contextual multi-armed bandits, attach any context to your user object (usually at initialization) before making the experiment check. You can confirm this in the diagnostics tab of your bandit in the console.

Checking a bandit

Check bandits using your standard experiment call. For example, in React, configure a bandit's variant JSON like this:

plaintext

{
    "text": "<my_text>",
    ...
}

and accessing it in code using the same pattern as experiments:

plaintext

const banditText = useExperiment('contextual_bandit').config.get('text');

This is separated from a bandit's linked experiment call to reduce excessive logging; if you have a linked wrapping experiment, wrap the call after checking the experiment.

Statsig will automatically detect and differentiate categorical and numerical features based on cardinality.

Getting a list

Getting a ranked list of variants is a common requirement. Client-side filtering or other constraints may prevent showing a particular variant.

Statsig supports getting a list of scored outcomes, but requires manual exposure logging so that Statsig knows which variant you displayed to the user.

Outcomes

In Statsig Cloud, log an outcome event (as a flag, or with a value) to Statsig, and Statsig handles connecting exposures to outcome events within your configured time periods.

In Statsig Warehouse Native, you can choose which approach to take. Logging events with Statsig exports events to your warehouse in near-real-time for use in Bandits, or you can provide your own outcome dataset as a metric source.

Data latency matters: if your output dataset has more than 24 hours of lag, bandit exploration can slow down significantly.

Was this helpful?