Synthetic A/A Test

An A/A test randomly splits users you already have events for into two groups that receive the same experience. Run one to validate your experimentation setup and confirm your metrics and Pulse scorecards behave correctly before you run a real A/B test. Create an assignment source with the following script, then analyze the A/A test in minutes as Pulse scorecards populate.

Example script:

SELECT
    user_id,
    timestamp,
    'AA_Test_1' AS experiment_name, --CAST('AA_Test_1' as varchar) AS experiment_name for Redshift warehouse
    CASE
        WHEN <random_logic> THEN 'Control'
        ELSE 'Test'
    END AS GroupAssignment
FROM <my_event/metrics_table>

Replace <random_logic> with the following based on your warehouse:

• Bigquery: mod(abs(farm_fingerprint(cast(user_id as string))), 100) < 50
• Redshift: mod(abs(farmFingerprint64(cast(user_id as varchar))), 100) < 50
• Snowflake: mod(abs(hash(cast(user_id as string))), 100) < 50
• Databricks: mod(abs(hash(cast(user_id as string))), 100) < 50
• Athena: mod(abs(cast(conv(substr(md5(cast(user_id as varchar)), 1, 16), 16, 10) as bigint)), 100) < 50