# Forward Proxy

## Statsig Forward Proxy

The Statsig Forward Proxy is a service you can host in your own infrastructure. When you configure SDKs to use the forward proxy, the proxy provides a closer and more reliable hop for retrieving Statsig configurations from within your infrastructure instead of reaching out to Statsig's networks.

The expected benefits of using the proxy include:

1. Reduced dependency on Statsig Infrastructure Availability
2. Improved performance through localization of download\_config\_spec to your cluster
3. Improved cost savings and reduced network overhead through minimizing requests to Statsig Infrastructure
4. Improved consistency of configurations, as the forward proxy becomes an articulation point for serving configs
5. Support for features such as a streaming API through gRPC

If you have questions about deployment or need assistance, contact Statsig on Slack or create a new GitHub issue.

### Installation

#### Helm installation

The recommended way to install Statsig Forward Proxy in a Kubernetes environment is the official Helm chart. Follow these steps to deploy:

```bash
# Add the Statsig Helm repository
helm repo add statsig https://statsig-helm.storage.googleapis.com
helm repo update

# Install the chart
helm install statsig-forward-proxy statsig/statsig-forward-proxy
```

The Helm chart provides extensive configuration options for customizing your deployment. For detailed configuration options, go to the [Statsig Forward Proxy Helm chart documentation](https://github.com/statsig-io/statsig-forward-proxy/blob/main/chart/README.md).

#### Manual deployment

For environments where Helm isn't available or for more customized deployments, you can deploy Statsig Forward Proxy manually. Statsig provides the proxy as a pre-built Docker image, and you can also build your own binary from source.

For detailed instructions on manual deployment options and available configuration parameters, go to the [Manual Deployment section in the GitHub repository](https://github.com/statsig-io/statsig-forward-proxy/tree/main?tab=readme-ov-file#manual-deployment).

### How Forward Proxy works

The forward proxy works by setting up a local HTTP or gRPC server. When you make requests to the forward proxy, only the initial request is blocking. After that, a background loop keeps configurations up to date without impacting the serving path.

The forward proxy also enables backup caches and monitoring through technologies such as Redis and Statsd.

## Integration with SDK

{% callout type="info" %}
`Legacy Java/Kotlin`, `Legacy Python`, `Python Server Core`, and `Node Server Core` SDKs support Forward Proxy integration with gRPC streaming. For other server SDKs, integrate with the Forward Proxy through HTTP by overriding the initialize endpoints in StatsigOptions. Contact Statsig in [Slack](https://statsig.com/slack) if you need guidance on this approach.
{% /callout %}

You can configure SDK networking to use different protocols for integration with the Statsig Forward Proxy. The SDK makes requests to three network endpoints: `download_config_specs`, `get_id_lists`, and `log_events`. You can configure `download_config_specs` to use the proxy. Support for `get_id_lists` and `log_events` is under active development.

#### Network protocol for different endpoints

For `download_config_specs` endpoint, where the SDK gets specs on evaluating gates/layers/experiments/configs

1. `http`: the default protocol. If you initialize the SDK with http, it polls config updates in a background thread.
2. `grpc_websocket`: Establishes a gRPC streaming connection from the SDK to the Statsig Forward Proxy and listens for updates pushed from proxy servers. You must use the Statsig Forward Proxy or a similar proxy server to use gRPC streaming. Not all SDKs support gRPC yet. If you need support for a specific SDK, reach out on the Statsig support channel on Slack.
3. `grpc`: Unary RPC, behaves similarly to the HTTP protocol. After initialization, the SDK polls changes from the forward proxy server in a background thread.

#### Listen to config change example

```typescript
const statsigOptions = {
  proxyConfigs: {
    download_config_specs:  {
      proxyAddress: proxyAddress // Your proxy address
      protocol: "grpc_websocket"
    }
  }
}
Statsig.initialize(server_key, statsigOptions)
// Statsig will use listen for config updates from statsig forward proxy using grpc_websocket protocol. And use http to get idlists and post log events from statsig servers.
```

For information on your specific SDK language, go to the language-specific docs in the left-hand column.

#### Failover behavior

**Initialization**

SDK behavior remains the same when the forward proxy is in use. Several configurations can improve data availability during initialization.
The default behavior is to get from the forward proxy. If a DataAdapter is present, the SDK gets from the DataAdapter first and falls back to the Forward Proxy if the data adapter returns no value.
Several ways exist to configure this behavior.

1. Set `fallbackToStatsigAPI = true` to fall back to the Statsig API when initialization or config sync from primary sources fails.
2. To customize the initialization order, set `initializeSources` with multiple sources. For example, `initializeSources=[DataSource.Network, DataSource.DataAdapter, DataSource.StatsigNetwork]` fetches from three sources sequentially until one succeeds.

**Config Sync (Post initialization)**

If the SDK is using a polling model (`grpc` or `http`):

1. Set `fallbackToStatsigAPI=true`.
2. Set `configSyncSources` with multiple sources. For example, `initializeSources=[DataSource.Network, DataSource.StatsigNetwork]` fetches from sources sequentially until one succeeds. Config sync sources don't apply when you're using streaming.

If the SDK is using the `grpc_websocket` protocol (listening mode for config changes), by default it does the following:

1. Retry connecting to the forward proxy with exponential backoff: 10s, 50s, 250s, 1250s, and so on, up to 10 retries.
2. If the 4th retry fails, the SDK starts polling from the Statsig endpoint. Use `StatsigOptions.pollingInterval` to control the polling frequency.
3. While polling from the Statsig endpoint, the SDK continues retrying the forward proxy connection until it succeeds.

All of the above behaviors are configurable through `StatsigOptions`.

#### Streaming failover in depth

When exceptions occur (for example, when a connection drops or the forward proxy is down), gRPC returns an error to the SDK, which then starts the failover behavior. The Python SDK checks whether the channel is idle every 2 hours to prevent edge-case behavior in the Python gRPC library.

#### TLS: advanced network authentication

The forward proxy supports TLS and mTLS when using the gRPC server, so it encrypts all gRPC network traffic (streaming and unary calls).

To enable it:

1. [Setup forward proxy server with valid certifications](https://github.com/statsig-io/statsig-forward-proxy?tab=readme-ov-file#deploying)
2. [Setup SDK with valid certifications (python example)](https://docs.statsig.com/server/pythonSDK).

{% callout type="note" %}
If certificates are misconfigured, the SDK treats the misconfiguration as an exception and starts the failover behavior you configured (for example, falling back to the Statsig API or retrying the connection).
{% /callout %}

#### Coordinate SDK DataAdapter / DataStore with Forward Proxy cache service

To increase reliability during initialization, configure a DataStore. If the Forward Proxy is unavailable at initialization time, the SDK initializes with values from the DataStore and then syncs from the forward proxy in the background. The SDK and the Forward Proxy should share the same cache service for consistency and to reduce cache I/O.

Example on DataStore setup in python:

```py
class DataAdapter(IDataStore):
def __init__(self):
  self.redis_cache = ExampleCache()

def get(self, key: str):
  # IDlist isn't currently supported by proxy, so do normal lookup
  if "statsig.id_lists" in key:
    return self.cache.get(key)
  # This logic must stay in sync with statsig-forward-proxy
  else:
    hashed_key = "statsig::" + hashlib.sha256(key.encode()).hexdigest()
    return self.cache.hget(hashed_key, 'config')

def set(self, key: str):
  # Don't implement set method if you are share the same cache between forward proxy and sdk, forward proxy will write to cache 
  pass

def shutdown(self):
  self.cache.shutdown()
```

#### Deployment Notes

Because every use case differs, there are no strict requirements. General recommendations include:

1. Understand your general QPS to Statsig. Contact the Statsig team if you need help.
2. Estimate the throughput a single pod can support, then calculate how much to scale out to handle your traffic.
3. Pre-scale the proxy for your peak QPS before gradually rolling it out to services.
4. Configure auto-scaling. For most deployments, scaling up at 70% CPU and memory utilization is sufficient, but this may need adjustment depending on usage.

If horizontal scaling doesn't resolve scaling issues, front the proxy with nginx to help with flow control.

For assistance, send a message to the [Statsig Slack channel](https://statsig.com/slack).