Synthetic A/A Test

Run an A/A test in Statsig Warehouse Native to validate your experimentation setup and confirm metrics behave correctly before running real A/B tests.

To create a quick A/A test, randomly split existing users you already have events for. Create an assignment source using the script below and you'll be ready to analyze your A/A test in minutes and see Pulse scorecards populate.

Example script:

sql

SELECT
    user_id,
    timestamp,
    'AA_Test_1' AS experiment_name, --CAST('AA_Test_1' as varchar) AS experiment_name for Redshift warehouse
    CASE
        WHEN <random_logic> THEN 'Control'
        ELSE 'Test'
    END AS GroupAssignment
FROM <my_event/metrics_table>

Replace <random_logic> with the following based on your warehouse:

Bigquery: mod(abs(farm_fingerprint(cast(user_id as string))), 100) < 50
Redshift: mod(abs(farmFingerprint64(cast(user_id as varchar))), 100) < 50
Snowflake: mod(abs(hash(cast(user_id as string))), 100) < 50
Databricks: mod(abs(hash(cast(user_id as string))), 100) < 50
Athena: mod(abs(cast(conv(substr(md5(cast(user_id as varchar)), 1, 16), 16, 10) as bigint)), 100) < 50

Was this helpful?