---
title: Deploy Forward Proxy ID list enrichment
description: "Deploy Statsig Forward Proxy with Redis, enable ID list synchronization, and verify the user enrichment endpoint."
product: general
lang: en
token_estimate: 3993
---
# Deploy Forward Proxy ID list enrichment

Statsig Forward Proxy synchronizes ID lists into Redis and serves membership lookups through `POST /v1/user_enrichment`. Use this deployment to share ID list data across application processes instead of downloading every list into each SDK.

> For AI agents: a documentation index is available at [/llms.txt](/llms.txt). Append `.md` to any page URL for markdown, or send `Accept: text/markdown`.

This guide covers proxy deployment and operation. After the endpoint is ready, follow [SDK user enrichment](https://docs.statsig.com/server/concepts/async-user-enrichment) to build enriched users and evaluate them locally. For config-spec routing alone, use the [regular Forward Proxy setup](https://docs.statsig.com/server/concepts/forward_proxy).

## How synchronization works

All Forward Proxy pods participate in downloading ID lists and updating shared Redis data. The proxy periodically checks for changes, downloads new or updated list data, and removes lists that are no longer needed.

Server keys for the same company share the stored ID list data. Each key's enrichment responses include only the lists available to that key.

Configure the same `SFP_IDLIST_SYNC_KEYS_JSON` key set on all pods that share a `REDIS_KEY_PREFIX` so they synchronize the same lists.

## Supported ID list sizes

The standard project limit is 10 million IDs across all ID lists. To exceed this limit, contact Statsig Support for enablement. This requirement applies to both standard SDK downloads and Forward Proxy enrichment. Deploying Forward Proxy doesn't raise the limit automatically.

With standard SDK downloads, each process needs memory for its lists. With enrichment, Redis stores the shared lists and membership indexes. Size Redis for the enabled workload before deployment.

## Prerequisites

- A Statsig server secret key for the project that owns the ID lists.
- An enrichment-capable Forward Proxy image supplied for early access. Pin its tag or digest. Contact Statsig Support for a compatible build.
- Helm 3 for Kubernetes, or Docker with Docker Compose for local use.
- Redis that all Forward Proxy replicas can reach. Use persistence and `noeviction` for production because evicting reverse-index keys can produce incomplete memberships.

## Start Forward Proxy and Redis with Docker Compose

Create `.env` next to your Compose file:

```dotenv
STATSIG_SERVER_SDK_KEY=secret-your-server-key
STATSIG_FORWARD_PROXY_IMAGE=statsig/statsig-forward-proxy:<enrichment-image-tag>
```

Keep `.env` out of version control. In production, inject `STATSIG_SERVER_SDK_KEY` from your secret manager.

Create `compose.yaml`:

```yaml
services:
  redis:
    image: redis:7-alpine
    command:
      - redis-server
      - --appendonly
      - "yes"
      - --maxmemory-policy
      - noeviction
    volumes:
      - redis-data:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 20
    restart: unless-stopped

  forward-proxy:
    image: ${STATSIG_FORWARD_PROXY_IMAGE:?Set an enrichment-capable image}
    depends_on:
      redis:
        condition: service_healthy
    ports:
      - "127.0.0.1:8000:8000"
    environment:
      REDIS_ENTERPRISE_HOST: redis
      REDIS_ENTERPRISE_PORT: "6379"
      REDIS_TLS: "false"
      REDIS_CONNECTION_POOL_MAX_SIZE: "64"
      SFP_IDLIST_SYNC_KEYS_JSON: '["${STATSIG_SERVER_SDK_KEY}"]'
      IDLIST_SYNC_INTERVAL_IN_S: "60"
      IDLIST_REDIS_POOL_SIZE: "32"
      IDLIST_FETCH_TIMEOUT_MS: "50"
      USER_ENRICHMENT_MAX_LOOKUPS: "100"
    command:
      - http
      - redis
      - --http-request-timeout-in-s
      - "1800"
      - --http-read-timeout-in-s
      - "60"
    shm_size: "1gb"
    restart: unless-stopped

volumes:
  redis-data:
```

Replace `<enrichment-image-tag>` with the compatible tag before starting the services. Setting `SFP_IDLIST_SYNC_KEYS_JSON` enables synchronization and the enrichment endpoint together.

Start the services:

```bash
docker compose up -d
docker compose logs -f forward-proxy
```

`--http-request-timeout-in-s 1800` gives large ID list downloads time to stream. Size this value for your largest list and available network throughput. The default 30-second request timeout is often too short for large lists.

## Start Forward Proxy with Helm

Use the official Helm chart for Kubernetes with an enrichment-capable image, Redis cache mode, and `SFP_IDLIST_SYNC_KEYS_JSON`.

```bash
helm repo add statsig https://statsig-helm.storage.googleapis.com
helm repo update
```

Store authorized server keys in a Kubernetes Secret:

```bash
kubectl create secret generic sfp-idlist-sync-keys \
  --from-literal=SFP_IDLIST_SYNC_KEYS_JSON='["secret-YOUR_SERVER_SDK_KEY"]'
```

Create `values.yaml`:

```yaml
replicaCount: 2
image:
  repository: statsig/statsig-forward-proxy
  tag: "<enrichment-image-tag>"
sfp:
  args:
    - http
    - redis
    - --http-request-timeout-in-s
    - "1800"
    - --http-read-timeout-in-s
    - "60"
  environment:
    REDIS_ENTERPRISE_HOST: redis.example.internal
    REDIS_ENTERPRISE_PORT: "6379"
    REDIS_TLS: "false"
    REDIS_CONNECTION_POOL_MAX_SIZE: "64"
    IDLIST_SYNC_INTERVAL_IN_S: "60"
    IDLIST_REDIS_POOL_SIZE: "32"
    IDLIST_FETCH_TIMEOUT_MS: "50"
    USER_ENRICHMENT_MAX_LOOKUPS: "100"
  secrets:
    - envName: SFP_IDLIST_SYNC_KEYS_JSON
      secretName: sfp-idlist-sync-keys
      secretKey: SFP_IDLIST_SYNC_KEYS_JSON
```

Install the chart:

```bash
helm install statsig-forward-proxy statsig/statsig-forward-proxy -f values.yaml
```

Point `REDIS_ENTERPRISE_HOST` at a Redis deployment that every replica can reach. Use persistence and `maxmemory-policy noeviction`. For additional Helm options, refer to the [Forward Proxy Helm chart documentation](https://github.com/statsig-io/statsig-forward-proxy/blob/main/chart/README.md).

## Verify the deployment

Check that the proxy responds:

```bash
curl --fail http://localhost:8000/v1/health
curl --fail http://localhost:8000/v1/ready
```

These checks report HTTP service health. They don't verify Redis connectivity or completion of ID list synchronization.

Export the same server key configured in `SFP_IDLIST_SYNC_KEYS_JSON`, then request memberships for a known test user:

```bash
export STATSIG_SERVER_SDK_KEY=secret-your-server-key
curl --fail-with-body http://localhost:8000/v1/user_enrichment \
  -H "statsig-api-key: ${STATSIG_SERVER_SDK_KEY}" \
  -H 'Content-Type: application/json' \
  --data '{"user":{"userID":"user-123","customIDs":{"accountID":"account-456"}}}'
```

The endpoint accepts raw IDs in `user` for this check and hashes them internally. The SDK sends prehashed values in `id_list_lookups`. A successful response contains `id_lists` and `unsupported_enrichments`. The `id_lists` object maps each lookup hash to an array of matching list names.

Use a known member and confirm the expected list name. An empty lookup request skips Redis, so it can't test index readiness.

ID list updates aren't available immediately. Until changes finish syncing, membership-based targeting can use stale data. A newly added user may appear as a non-member, while a removed user may still appear as a member. HTTP 200 confirms a successful lookup, but doesn't guarantee that the result reflects the latest list changes. Allow time for synchronization and verify expected membership results before relying on updated targeting.

| HTTP status | Meaning and action |
| --- | --- |
| `200` | Lookup succeeded. Memberships can be stale until the latest ID list changes finish syncing. |
| `401` | Missing or unauthorized `statsig-api-key`. Match the configured server keys. |
| `404` | Check the route, image, and whether `SFP_IDLIST_SYNC_KEYS_JSON` enables the endpoint. |
| `413` | Too many unique lookup IDs. Reduce the request or adjust `USER_ENRICHMENT_MAX_LOOKUPS`. |
| `429` | This replica's per-key rate limit rejected the request. |
| `503` | Redis lookup failed, timed out, or the namespace isn't ready. Inspect sync progress and Redis health; don't interpret the failure as empty membership. |

## Forward Proxy configuration

The background poller refreshes manifests through the normal proxy polling loop. Set this interval with `--polling-interval-in-s` (default `10`). `IDLIST_SYNC_INTERVAL_IN_S` controls producer scheduling separately. The old `--idlist-sync-enabled`, `--user-enrichment-enabled`, and `IDLIST_MANIFEST_POLL_INTERVAL_IN_S` settings don't apply to this implementation.

### Required settings

| Setting | Purpose |
| --- | --- |
| `redis` cache argument | Enables the shared Redis data store required for synchronization and enrichment. |
| `SFP_IDLIST_SYNC_KEYS_JSON` | Enables the producer, workers, and HTTP enrichment endpoint. Supply a non-empty JSON array of authorized server keys. If you leave this variable unset, all three stay off. |
| `REDIS_ENTERPRISE_HOST` | Redis hostname. |
| `REDIS_ENTERPRISE_PORT` | Redis port. |

### Enrichment request settings

| Environment variable | Default | Purpose |
| --- | --- | --- |
| `USER_ENRICHMENT_MAX_LOOKUPS` | `100` | Maximum deduplicated lookup IDs per request. The proxy returns HTTP 413 above this limit. |
| `IDLIST_REDIS_POOL_SIZE` | `32` | Dedicated Redis read-pool size for enrichment. |
| `IDLIST_REDIS_READ_URI` | Primary Redis address | Optional `redis://` or `rediss://` read-replica address. Authentication uses the shared Redis settings. |
| `IDLIST_FETCH_TIMEOUT_MS` | `50` | Timeout for one enrichment membership fetch from Redis. A failure returns HTTP 503 rather than empty memberships. |
| `IDLIST_DICT_REFRESH_IN_S` | `60` | Refresh cadence for the list-name-to-token dictionary used by the read path. |

### Synchronization settings

Normal version changes use partitioned streaming diff for every list size. Lists below the partition target use one partition. First loads and transitions from an empty list apply new memberships directly without building a difference set. The removed `IDLIST_STREAMED_DIFF_MIN_MEMBERS` setting has no effect.

| Environment variable | Default | Purpose |
| --- | --- | --- |
| `IDLIST_SYNC_INTERVAL_IN_S` | `60` | Producer lock window and sync cadence. |
| `IDLIST_QUEUE_POLL_INTERVAL_MS` | `2000` | Worker wait after an empty queue or a processing error. |
| `IDLIST_LOCK_TTL_IN_S` | `60` | Per-list worker claim TTL. The worker renews it while processing. Values below `3` are invalid. |
| `IDLIST_PIPELINE_BATCH_SIZE` | `5000` | Redis mutation batch size while applying files. |
| `IDLIST_SCAN_COUNT` | `1000` | Requested `SSCAN` page size. |
| `IDLIST_DIFF_PARTITION_MAX_MEMBERS` | `2000000` | Approximate maximum members in one streamed-diff partition. |
| `IDLIST_OLD_VERSION_TTL_IN_S` | `86400` | Retention time for the previous forward SET after rotation. |

### Redis connection settings

| Environment variable | Default | Purpose |
| --- | --- | --- |
| `REDIS_TLS` | `false` | Uses TLS for the Redis connection when `true`. |
| `REDIS_AUTH_MODE` | `password` | Uses static password/ACL authentication. Also supports OAuth with Microsoft Entra ID. |
| `REDIS_ENTERPRISE_USER` | unset | Optional Redis ACL username. |
| `REDIS_ENTERPRISE_PASSWORD` | unset | Optional Redis password. |
| `REDIS_KEY_PREFIX` | unset | Prefixes every Forward Proxy Redis key to isolate environments that share a Redis deployment. |
| `REDIS_CONNECTION_POOL_MAX_SIZE` | `10` | Maximum size of each shared-config Redis pool. Cache and synchronization use separate pools; enrichment uses its dedicated read pool. |
| `REDIS_CONNECTION_POOL_MIN_SIZE` | `1` | Minimum idle connection count. |

Use TLS and authentication outside a local development network. For OAuth and certificate requirements, refer to the [Forward Proxy Redis authentication guide](https://github.com/statsig-io/statsig-forward-proxy/blob/main/docs/redis_auth.md).

## Estimate Redis capacity

ID lists can outgrow a single Redis deployment. Budget for both the forward sets that store each list's members and the reverse mappings that identify which lists contain each user ID.

For initial planning, allow approximately 50–80 bytes per forward-set member and 80–120 bytes per distinct ID for reverse mappings. These are conservative estimates for eight-character lookup hashes, short Redis key prefixes, and compact integer list tokens. They assume one list, one current version, and no shared IDs with other lists. Redis version, allocator, encoding, and key lengths affect actual memory.

| IDs in one list | Compact input file | Forward sets | Reverse mappings | Redis data subtotal |
| --- | --- | --- | --- | --- |
| 1 million | About 10 MB | 50–80 MB | 80–120 MB | 130–200 MB |
| 10 million | About 100 MB | 0.5–0.8 GB | 0.8–1.2 GB | 1.3–2 GB |
| 100 million | About 1 GB | 5–8 GB | 8–12 GB | 13–20 GB |

The Redis subtotal excludes old and in-progress list versions, other proxy cache data, fragmentation, client buffers, persistence overhead, and replicas. Updates can retain an old forward set alongside the new one and temporarily increase reverse mappings. Use `IDLIST_OLD_VERSION_TTL_IN_S` to configure how long Redis retains an old forward set after a replacement completes. Allow for every version retained during your update cadence, not just one replacement. A replica stores another copy of the dataset; persistence and replication require additional capacity.

For 100 million IDs, one retained forward version adds 5–8 GB to the 13–20 GB subtotal. That totals 18–28 GB before other overhead and replicas.

Across multiple lists, forward storage grows with total list memberships. Reverse mappings share one key per distinct ID within a company and add list tokens for overlapping memberships. Use your actual overlap and membership counts to size storage across multiple lists.

Load representative data into your Redis version and use [MEMORY USAGE](https://redis.io/docs/latest/commands/memory-usage/) and [INFO memory](https://redis.io/docs/latest/commands/info/) to validate the estimate. Monitor resident memory as well as dataset memory, then exercise list replacements and persistence operations. Redis documents [additional memory requirements during persistence](https://redis.io/docs/latest/operate/oss_and_stack/management/admin/).

### Scale beyond one deployment

- Increase Redis memory capacity before the dataset and update overhead approach its limit. Keep persistence and `noeviction` enabled.
- Separate independent projects or workloads into dedicated Forward Proxy and Redis deployments. Route each application's enrichment requests to the deployment that holds its lists.
- Add Forward Proxy pods for request and download capacity. More pods don't increase Redis storage capacity. Read replicas can help read throughput, but each stores a full copy of the data.
- If one workload or list exceeds a practical Redis node's memory, contact Statsig Support to plan a supported deployment topology. Confirm the deployment plan before increasing the enabled size.

## Production operation

Run multiple proxy replicas against the same Redis deployment. Keep server keys and Redis credentials in a secret manager. Use TLS and authentication outside local development.

Use Redis persistence and `maxmemory-policy noeviction`. Include the capacity and update overhead described in [Estimate Redis capacity](#estimate-redis-capacity). Readiness checks detect missing initialization and dictionary state, but don't detect every missing reverse-index key; eviction can still cause incomplete memberships.

If you use `IDLIST_REDIS_READ_URI`, account for replication lag before relying on fresh memberships. Benchmark your list sizes, unique IDs, request volume, Redis latency, and replica count instead of assuming a fixed throughput or memory ratio.

Monitor queue depth, task failures, checkpoint lag, staleness, full reloads, delta application, and enrichment `503` responses. Tune the request and read timeouts for your network and largest list.