Skip to main content

CDN Edge Testing for Cached Resources

Background​

Most customers with heavy web traffic use a CDN to serve resources from cache, in order to minimize hit to their web servers. This has historically made testing challenging because you don’t have the luxury of calling the SDK for all requests — but now with the emergence of Edge compute (offered by most providers), customers can now run code at their CDN edge, allowing them to assign users to tests and determine which resources to serve in a convenient and performant way. This pattern is optimal for testing with cached content without sacrificing cache-hit ratio. For scenarios where running the Statsig SDK at the edge is not possible, the sdk must be implemented on the origin server, or you should consider using a client SDK.

architecture

example only — implementation may vary depending on your provider

Best Practices​

  • Use a provider-specific "Config Sync" integration — Statsig will automatically sync your Statsig configuration data to your provider's Edge Storage.
  • Use a provider-specific Data Adapter — This will allow the Statsig SDK to initialize using the configuration stored at the edge near your function. Links for each provider provided below.
  • Use statsig-node-lite — This is a slimmed down version of the Node SDK, which includes only the essentials and dramatically improves initialization performance for cold-starts requests.
  • Persist a uuid — As always, you'll need some sort of user identifier that can be used to consistently assign a user to your test buckets. This can typically be solved by generating a uuid in your function and setting it to a cookie.
  • Persist assignments in a cookie — Some customers will set assignments to a cookie for the purpose of a performance optimization, allowing your code to skip sdk calls if the user is already assigned to a test. This is best defined as a session-cookie (a cookie that expires when user closes their browser).
  • Persist client instance — This pattern allows the Statsig client instance to persists across requests when the edge function remains warm and will improve performance.
  • Use Target Apps — Target apps will allow you to sync a specific subset of experiments/gates to your edge function, reducing the footprint of your project config and improving performance. Target Apps are compatible with all of the "config sync" integrations.
  • Consider peaking at experiment assignments and avoid logging exposures. You should only log exposure events once a user has been exposed to the treatment, otherwise your test results may become polluted with users that didn't see the treatment.
    • You should consider how this can be managed downstream in your application. All SDKs have a means of tracking exposures manually, and you can also consider using the HTTP API for logging exposures.
    • If you choose to log exposures from your edge function, you should consider using the context.waitUntil method if supported by your edge provider (examples).

Cloudflare Implementation​

Cloudflare allows its users to easily stand up a serverless edge worker that will get invoked prior to cache lookup and also allow you to modify the response to the viewer. The Worker architecture gives you full code control over how you'd like to map test assignments to resources you should serve to the user, allow you to modify both the URL (serving as cache-key) and headers used to fetch your resource from the CDN, passing the modified request through to origin on cache-miss.

The Cloudflare Worker runtime allows you to install Statsig Node Lite SDK as a dependency and use it to determine test assignment to determine how to fetch a given resource.

We offer both an integration with Cloudflare KV, and a pre-built KV Data Adapter, ensuring that SDK initialization is performant and doesn't depend on requests over the network back to Statsig.

Fastly Implementation​

Fastly Compute platform supports Functions at the Edge, affording customers the ability to handle assignment at the edge.

Fastly's KV storage solution is touted as being highly-performant and is now their recommended solution over their legacy ConfigStore documented below. Statsig offers both a Config-Sync for Fastly KV as well as a KV DataAdapter that should be installed in your function for performant initialization.

We initially built the integration with Fastly's ConfigStore, which was their only offering at the time. ConfigStore values are limited to 8 kilobytes, which will be met rather quickly as the number of gates and tests in your project increases. This will be paired with our Fastly Config Store Data Adapter, allow the SDK to initialize from ConfigStore rather than making a request over the the network back to Statsig.

The Fastly pattern is a bit different than some of the other providers — as recommended by the Fastly AB Testing guide, it does not involve modifying the cache-key (resource URL), but instead attaching Fastly-ABTest-<TEST ID> headers, which will tell your origin server how to render the resource, and the origin response should include a Vary response header to tell Fastly CDN what to cache. This approach is documented thoroughly by Fastly, and it's recommended that their guide serves as the source of truth for the design pattern.

info

Fastly VCL Implementation​

Their legacy Varnish-based platform only supports configuration based caching and minimal scripting at the edge using VCL scripting language.

Practically speaking, there is no Statsig SDK (or any SDK for that matter) support at the edge for this reason. In this instance, the Statsig SDK must be implemented on the origin server, and cache rules must be configured to force a cache-miss and allow assignments to take place on the origin server.

AWS Implementation​

AWS has a variety of serverless solutions, below we will detail the recommended pattern and various limitations associated with each.

Lambda@Edge Implementation​

Lambda@Edge are Lambda functions that are designed to be triggered by CloudFront events. Lambda@Edge implementations carry a few unique challenges:

  • At the time of writing this, AWS does not offer a KV store solution compatible with Lambda@Edge. Their KeyValueStore offering is only compatible with Cloudflare Functions. This means, you will either have to:
    • Use the default Statsig SDK initialization, which will incur a hit over the network to Statsig to load configurations
    • Bundle your project config statically into the function package. You can consider using our Config-Change Webhook to automate this.
    • Store your project config in S3 and use Cloudfront as means to serve it optimally when fetching it from your Lambda@Edge function.
  • Function size is capped at 1MB in package size (for viewer request and viewer response triggers, which is what is required for this setup).
  • Lambda model uses four events triggers, calling your function at various stages of the request lifecycle. For this use case, you’ll need to use two: Viewer Request and View Response. AWS provides documentation on how requests can be handled and manipulated here.

Viewer Request function​

Viewer Request will be called before cache lookup, and will be used to define a uuid, generate test assignment and modify the request URL via Request.URI. This type of function cannot modify response headers, so you must pass any assignment and uuid info through to the Viewer Response function in order to set cookies. Event structure for each event documented here.

Viewer Response function​

Viewer Response function is responsible only for setting uuid and assignment cookies as needed.

CloudFront Functions Implementation​

SDKs are not supported here. Cloudfront Functions have 1ms max runtime and 10KB max function size. You’ll have to use Lambda@Edge, or consider incurring Cache-misses to do assignment at origin.

Akamai Implementation​

Akamai has a documented pattern of using EdgeWorkers and EdgeKV for running AB tests using cached content, providing both code samples, as well as an overview of the architecture. You’ll need to add the Statsig SDK to the worker function as a dependency and call it for assignment (rather than the rudimentary coin flip using Math.random() shown in their code examples).

info

In progress​

At the time of writing this, we are currently working on validating SDK usage within their EdgeWorker environment and building both the Config-Sync for Akamai EdgeKV as well as the accompanying Data Adapter.