A/B Testing on Shopify

Run A/B tests on a Shopify storefront with Statsig, including adding the SDK to your theme, defining variants, and tracking order conversion metrics.

Use cases & considerations

Shopify provides solutions for commerce businesses to build and manage all aspects of their online storefront, including product catalogue, inventory, site content, marketing, and user experience.

For experimenting with the more static aspects of the store experience (static landing pages, visual aspects), use Statsig Sidecar to both build your test treatments and to assign users to experiments when they land on your site, all without writing any code.To experiment on the more dynamic aspects of your online store (such as product catalogue or search capabilities), use Shopify Headless Commerce and integrate Statsig SDKs to get full control for experimenting within business logic.

Using traditional Shopify + Sidecar for no-code testing

The traditional Shopify service is a fully-managed platform for businesses that provides both a backend administration tool for managing your product catalogue & site content, and powers the storefront experience for your shoppers.

While Statsig doesn't have an integration in the Shopify App Store, you can integrate Sidecar to run simple UX experiments on the storefront. The steps below walk through setting up Sidecar within the traditional Shopify stack.

Install Sidecar Chrome extension

Follow this guide on installing the Sidecar Chrome extension. This lightweight Chrome extension lets non-technical users build experiments and treatments. You can specify where the test runs based on URL, then configure treatments such as content changes, style changes, image swaps, or injected JavaScript for cases where the visual editor can't accommodate the requirement.

Add Sidecar JS to your storefront's page source

Log in to your Shopify dashboard
Click on Online Store, then Themes
Locate and click the more menu [...], find the Edit Code option

Locate your Sidecar script tag and copy the script tag to your clipboard
Navigate to the theme.liquid file in your Shopify theme editor
Paste the Sidecar script tag toward the top of the page <head> as shown below.

Save your theme.liquid file.
Sidecar is now installed across your entire website.

Configure event tracking

Shopify's Custom Pixel framework is ideal for tracking customer events to Statsig. The custom pixel framework offers a wide set of events you can subscribe to, and namely, the ability to perform tracking during the checkout experience. Code deployed outside the scope of a custom pixel won't fire during the checkout experience, as documented in the Shopify pixel sandbox documentation.

Below is boilerplate custom pixel code that provides a function to send events back to Statsig. You should subscribe to the various events and event metadata necessary for your experimentation practices. This sample GTM pixel shows some of the common events and metadata that you can capture and track to Statsig.

const getStableID = () => {  
  // New gen JS-SDK stores in dynamic keyed localstorage entries
  for (let i = 0; i < localStorage.length; i++) {
    const key = localStorage.key(i);    
    if (key && key.includes('statsig.stable_id.')) {
      const value = localStorage.getItem(key);
      return value.replace(/"/gi, '');
    }
  }
  // Old gen JS-SDK stores in specific localStorage key
  const fallback = localStorage.getItem('STATSIG_LOCAL_STORAGE_STABLE_ID');  
  if (fallback) {
    return fallback.replace(/"/gi, '');
  }
};

/**
* Util function for tracking events back to statsig
*/
const statsigEvent = async (eventKey, eventValue = null, metadata = {}, userObject = {}) => {
  Object.assign(userObject, {
    customIDs: {stableID: stableID} // attach stableID automatically
  });
  await fetch('https://events.statsigapi.net/v1/log_event', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json', 'statsig-api-key': 'client-STATSIG_CLIENT_KEY' },
    body: JSON.stringify({
      "events": [{"user": userObject, "eventName": eventKey, "metadata": metadata}]
    })
  });
}

analytics.subscribe("checkout_completed", event => {
  statsigEvent('checkout', null, {
    orderId: event.data?.checkout?.order?.id,
    currency: event.data?.checkout?.currencyCode,
    subtotal: event.data?.checkout?.subtotalPrice?.amount,
    shipping: event.data?.checkout?.shippingLine?.price?.amount,
    value: event.data?.checkout?.totalPrice?.amount,
    tax: event.data?.checkout?.totalTax?.amount,
  });
});

analytics.subscribe("product_viewed", (event) => {
  statsigEvent('product_viewed', null, {
    product_title: event.data?.productVariant?.title,
  });
});

For tracking behaviors in the main storefront experience, you can also use the following:

Autocapture: loads by default with Sidecar and automatically collects various user behaviors.
Custom event logging: tracks behaviors using code.

Using Shopify Headless + Statsig SDKs for deeper experimentation

Using Shopify Headless gives you full control over customizing your storefront by decoupling the Shopify admin backend and the storefront application. This means that Shopify effectively serves as a data store, providing APIs to fetch & serve products, content, and manage the entire shopping experience using code.Whether you are using Shopify's Hydrogen app and its frameworks or a custom headless stack, you can integrate Statsig's SDK to assign users to experiments. Integrating Statsig in this architecture follows a similar pattern to integrating with headless CMS platforms.

Integrating data sources for experiment metrics

Along with tracking click stream and point-of-sale behavior (described in Configure event tracking), commerce businesses performing deeper experimentation often want to integrate offline data systems and measure experiments using existing business metrics.The Data Warehouse is commonly the source of truth for user purchase data and other offline data. This lets customers define more bespoke metrics using filtering, aggregations, and other warehouse datasets to segment experiment results.

Was this helpful?