Skip to main content

How to run an AB Test in Python

Running a simple A/B experiment in a web page with Python and the Statsig SDK involves a few distinct steps, including creating a basic web server, integrating with Statsig for A/B testing, and configuring the experiment via the Statsig console. Below is a step-by-step guide.

Step 1: Create a Basic Python Web Server​

First, you'll need a simple Python web server. Flask is a lightweight WSGI web application framework that's perfect for this purpose.

Step 1: Creating a Simple Python "Hello World" Application​

  1. Set Up Your Python Environment:

    • Ensure Python is installed on your system. You can download it from python.org.
    • Optionally, create a virtual environment for your project to manage dependencies:
      python -m venv venv
      source venv/bin/activate  # On Windows use `venv\Scripts\activate`

  2. Create a New Python File:

    • In your project directory, create a file named app.py.

  3. Write a Simple Python Program:

    • Open app.py in your favorite text editor or IDE and write a simple Python program:
      print("Hello, World!")

  4. Run Your Program:

    • Run your program from the command line or terminal:
      python app.py
    • You should see "Hello, World!" printed to the console.

Step 2: Integrate Statsig for A/B Testing​

  1. Install Statsig SDK:

    • Add the Statsig SDK to your project:
      pip install statsig

  2. Initialize Statsig in Your Application:

    • Modify app.py to include initialization of the Statsig SDK and to check a feature flag. You will need a server secret key from Statsig, which you can get from the Statsig dashboard.

      from statsig import statsig
      from statsig.statsig_user import StatsigUser

      # Initialize Statsig with your server secret key
      statsig.initialize('your_server_secret_key')
      user = StatsigUser(
        user_id = "user123", 
        email = "test@statsig.com",
      )  # Customize user info as needed
      experiment = statsig.get_experiment(user, 'your_experiment_name')
      text_to_show = experiment.get('text', 'Hello, World!')
      print(text_to_show)

      # Shutdown Statsig before your application exits
      statsig.shutdown()

    • Replace 'your_server_secret_key' with your actual server secret key from Statsig.

    • Replace 'your_experiment_name' with the name of your experiment.

Step 3: Creating the A/B Test in Statsig Console​

  1. Log Into Statsig Console:

    • Access the Statsig Console and sign in.

  2. Create an Experiment:

    • Navigate to the "Experiments" section and click "Create New".
    • Fill in the experiment details, including a unique key for 'your_experiment_name'.

  3. Configure Experiment Variants:

    • Define your variants, for example, variant_a and variant_b.
    • Add a parameter called 'text' for each variant and configure the text you wish to display, such as "This is Control Variant" and "This is Test Variant".

    image

  4. Launch Your Experiment:

    • After setting up your variants and their parameters, save your configurations and start the experiment.

By following these steps, you've set up a Python Flask application that utilizes Statsig for running an A/B test, dynamically serving different text content based on the experiment's variant. This setup enables data-driven decisions and iterative improvements based on user interactions.