---
title: Configs
description: "Create and version configs in Statsig Agent Experimentation to manage the model, prompt messages, and parameters your app uses, and change them without deploying code."
product: general
lang: en
token_estimate: 4544
---
# Configs

A config stores the model, prompt messages, and parameters for one LLM task in your app, such as classifying a support ticket. Configs on this page are Agent Experimentation configs, not [Dynamic Configs](https://docs.statsig.com/dynamic-config/overview). Your app fetches the config at runtime with a Statsig AI SDK, so you can change the model or the prompt without deploying code. Each config has versions. Your app serves the Live version, and you evaluate other versions before you promote one to Live.

> For AI agents: a documentation index is available at [/llms.txt](/llms.txt). Append `.md` to any page URL for markdown, or send `Accept: text/markdown`.

Use a config to serve, evaluate, and roll out a prompt in production. To compare prompts side by side before you commit to one, use the [Playground](https://docs.statsig.com/agent-experimentation/playground) instead.

The Statsig SDKs and the Console API still call configs "prompts." For example, the Node AI SDK fetches a config with `getPrompt`.

## Find a config

Go to _Agent Experimentation > Configs_ to open the list of configs in your project. The list has these columns:

| Column | Shows |
| --- | --- |
| Name | The config's name |
| Last Modified | When someone last changed the config |
| Live Version | The version your app serves |
| Creator | The person who created the config |
| Target Applications | The target applications that receive the config. This column appears only when your project uses target applications. |
| Tags | The config's tags |

Search the list by name, or sort it by **Name** or **Last Modified**. Filter it by **Creator**, **Tag**, or **Target Application**. The **Target Application** filter appears only when your project uses target applications.

Click a config to open it. The config page has three tabs: _Setup_, _Runs_, and _Traces_.

## Create a config

1. Go to _Agent Experimentation > Configs_ and click **Create**. You can also click **Create** on the Statsig home page, then click **AI Config**.
2. In the _Create New Config_ dialog, enter a name. Statsig shows the config ID under the name, after `ai_config_name:`. The ID is the name in lowercase, with underscores in place of spaces. Your app uses the ID to fetch the config.
3. (Optional) Enter a description.
4. (Optional) Choose a **Team**, **Tags**, and **Target Apps**. By default, a new config uses your first team and that team's default target applications.
5. Click **Create**.

Statsig creates the config and opens its _Setup_ tab. The config starts with one Live version named "Version 1," with an empty System message and an empty User message. Saved changes to Version 1 reach your app right away, because Version 1 is Live.

To turn prompts that you tested in a playground into a config, go to [Promote prompts to a config](https://docs.statsig.com/agent-experimentation/playground#promote-prompts-to-an-ai-config).

## Configure a version

Configure a config version on the config's _Setup_ tab. The tab is one page with two groups of settings:

- **Config Setup**: The selected version's settings: _Version_, _Custom workflow_, _Model_, _Prompt_, and _Parameters_. Choose the version to configure from the **Version** list.
- **Run Setup**: The evaluators and the dataset that offline runs use. These settings apply to every version of the config. The group's _Schedule_ section can also start offline runs of the versions you choose every day. For details, go to [Schedule daily runs](https://docs.statsig.com/agent-experimentation/offline-evaluation#schedule-daily-runs).

To add evaluators, go to [Evaluators](https://docs.statsig.com/agent-experimentation/evaluators). Evaluators also decide a version's overall score. The config's [primary evaluator](https://docs.statsig.com/agent-experimentation/evaluators#what-is-a-primary-grader) sets the score, and a [critical evaluator](https://docs.statsig.com/agent-experimentation/evaluators#what-is-a-critical-grader) fails a run when it scores 0. To add a dataset, go to [Add a dataset to a config](https://docs.statsig.com/agent-experimentation/datasets#add-a-dataset-to-an-ai-config).

Changes on the _Setup_ tab stay unsaved until you click **Save**. Click **Discard Changes** to undo them.

### Choose a model

Under _Model_, choose a **Provider** and a **Model**. Statsig maintains the list of models for each provider: OpenAI, Anthropic, and Google.

Some models limit the other settings:

| Models | Limit |
| --- | --- |
| Models whose IDs start with `gpt-5`, `gpt-6`, `o1`, `o3`, or `o4` | **Temperature** stays at 1. The field's tooltip reads "This model only supports the default temperature of 1." |
| Anthropic and Google models | At least one message must have the User role. |
| `claude-fable-5`, `claude-opus-4.8`, `claude-sonnet-5`, `claude-sonnet-4.5`, and `claude-haiku-4.5` | These models reject **Temperature** and **Top P** set together. Leave **Top P** off. |

Statsig charges previews and runs to your Included Credits unless your company has an API key for the provider. Without a key, a notice under _Model_ shows an **Add API key** link. For details, go to [Credits and API keys](https://docs.statsig.com/agent-experimentation/overview#credits-and-api-keys).

### Write the prompt messages

Under _Prompt_, each message has a role: System, User, or Assistant. Click the role on a message to change it. A config version holds up to 10 messages.

- To add a message, click **Add New Message**.
- To copy or remove a message, open the message's menu and click **Clone Prompt** or **Delete Prompt**.

Messages can't be empty. When the config has a dataset, at least one message must reference a dataset column with a variable.

A variable is a dataset column name in double braces, such as `{{input}}`. In previews and offline runs, Statsig replaces each variable with the row's value from that column. The _Variables_ list under the messages shows every column in the config's dataset. Click a variable to insert it at the cursor. A variable uses underscores where its column name has spaces.

When a column holds JSON, `{{column}}` inserts the whole value. To insert one field, add a dot path of keys and array indexes, such as `{{order.items.0.name}}`. A negative index counts from the end of the array. A path that doesn't exist inserts an empty string. A path that points to an object or an array inserts it as JSON.

### Preview output

Results Preview runs the version on the first 5 rows of the config's dataset and shows each output. Use the preview to check a prompt before you start a full run. The preview uses the current settings, including unsaved changes.

To run the preview, click **Preview Output** under the messages. To run it again after a change, click **Reload Preview**. The config needs a dataset first.

### Improve the prompt with Prompt Advisor

Prompt Advisor uses Statsig AI to rate the prompt messages and suggest a rewrite. When Prompt Advisor is on, a badge under the messages rates them as Weak, Acceptable, or Great.

1. Click the badge to show the _Prompt Suggestion_ comments.
2. Click **Rewrite with suggestions** to get rewritten messages.
3. Click **Apply** to replace all of the version's messages with the suggestion.

An admin turns on Prompt Advisor in _Settings > Product Configuration > AI Features_. In the _AI Configs_ section, turn on **Prompt Advisor**.

### Set parameters

Under _Parameters_, set the values that Statsig sends to the model. **Top P**, **Frequency Penalty**, and **Presence Penalty** each have a switch. Statsig sends one of these values only when its switch is on.

| Parameter | Range | Default |
| --- | --- | --- |
| **Temperature** | 0 to 2 | 1 |
| **Max tokens** | 1 to 1,500 | 1,000 |
| **Top P** | 0 to 1 | Off |
| **Frequency Penalty** | -2 to 2 | Off |
| **Presence Penalty** | -2 to 2 | Off |

Under _Output Format_, choose the format of the model's response:

- **Text**: Plain text.
- **JSON Object**: A JSON object.
- **Structured Output (JSON Schema)**: JSON that matches a schema you enter. The schema is a JSON object with a `name` field and a `schema` field. The `schema` field needs `type` and `properties` fields.

## Call your own endpoint with a custom workflow

A custom workflow sends each row of an offline run to your own endpoint instead of to the model. Use a custom workflow when your app produces the output with more than one model call, such as an agent that calls tools or retrieves documents. Statsig grades your endpoint's response. Your app still receives the version's model and prompt through the SDK.

To set up a custom workflow, open the config's _Setup_ tab and turn on **Call your own endpoint for more complex workflows** under _Custom workflow_. Then fill in these fields. The version's _Prompt_ messages become optional.

| Field | Value |
| --- | --- |
| **Workflow URL** | The endpoint that Statsig calls for each row. Required. |
| **Output JSON field path** | If the endpoint returns JSON, the dot path to the output, such as `result.output_text`. Put quotes around a key that contains a dot. |
| **Workflow batch size** | The number of requests in each batch. Statsig sends batches one after another. Default: 500. |
| **Workflow Body** | The JSON request body. The body must contain at least one variable. |
| **Workflow Headers** | Request headers. Click **Add Header** to add one. Up to 15 headers. |
| Auth headers | Headers with secret values, such as an API key. Click **Add Auth Header** to add one. Statsig hides their values after you save. Up to 15 headers. |

To check the settings, click **Test Connection**. Statsig sends one request and shows the response, or the HTTP error code.

During a run, Statsig adds two headers to each request. `x-statsig-eval-run` holds the run's ID, so your endpoint can skip side effects for requests from a run. `traceparent` holds the W3C trace context of the row's trace in Traces. A header that you set with the same name replaces Statsig's. **Test Connection** doesn't send these headers. To show your endpoint's spans in the row's trace, go to [Join a workflow run's trace](https://docs.statsig.com/agent-experimentation/node#join-a-workflow-eval-trace).

The workflow body accepts dataset columns, such as `{{input}}`, and these variables from the version:

| Variable | Value |
| --- | --- |
| `{{prompt.prompts}}` | All messages as a JSON array of objects with `role` and `content` fields. Statsig fills in the dataset variables first. |
| `{{prompt.prompts.0.content}}`, `{{prompt.prompts.0.role}}` | One message's content or role. Messages count from 0. |
| `{{prompt.model}}` | The model ID |
| `{{prompt.temperature}}`, `{{prompt.top_p}}`, `{{prompt.frequency_penalty}}`, `{{prompt.presence_penalty}}`, `{{prompt.max_tokens}}` | The version's parameters |

For example, this body sends the messages, the model, and the row's `input` column:

```json
{
  "messages": {{prompt.prompts}},
  "model": "{{prompt.model}}",
  "question": "{{input}}"
}
```

The SDK receives the workflow URL, body, and output field path with the version. The headers, auth headers, and batch size stay on Statsig's servers.

## Manage versions

Each version of a config holds its own model, messages, and parameters. Your app serves the Live version by default, so you can test a change in another version first. Manage versions in the _Version_ section of the config's _Setup_ tab.

### Version types

Every version has one of four types. The type decides whether your app receives the version and whether you can promote it.

| Type | Sent to the SDK | Use it for |
| --- | --- | --- |
| Live | Yes | The version your app serves. Each config has exactly one Live version. |
| Candidate | Yes | A version that you evaluate against Live or test in an experiment. Read Candidate versions in code to shadow-run them without exposing them to users. |
| Draft | No | Work in progress in the console |
| Archived | No | Versions that you keep for reference. Set an Archived version to Draft or Candidate before you promote it. |

### Create and change versions

To create a version, open the **Version** list and click **New version**. A new version starts as a Draft copy of the Live version.

The menu next to the **Version** list has these actions:

| Action | What it does |
| --- | --- |
| **Promote to Live** | Makes the version Live. The previous Live version becomes a Candidate. |
| **Rename Version** | Changes the version's name. |
| **Clone Version** | Copies the version into a new version of the same type. A copy of the Live version is a Draft. |
| **Start Evaluation** or **Stop Evaluation** | Starts or stops an offline run of the version. |
| **Set to Candidate** | Makes the version a Candidate. Not available for the Live version. |
| **Set to Draft** | Makes the version a Draft. Not available for the Live version. |
| **Archive** | Archives the version. Not available for the Live version. |
| **Delete Version** | Deletes the version. Not available for the Live version or for a config's only version. |

### Promote a version to Live

**Promote to Live** compares the version's latest completed offline run with the Live version's latest completed offline run. If the Live version has no completed run, Statsig promotes the version without asking. Otherwise, Statsig asks you to confirm in these cases:

| Dialog | Appears when |
| --- | --- |
| Promote a lower-scoring version? | The version scores lower than the Live version. |
| Promote without a completed run? | The version has no completed run, for example because its run is still in progress. |

Click **Promote Anyway** to promote the version.

### Edit the Live version

Saving a change to the Live version changes what your app serves right away. If your app fetched the config recently, Statsig asks you to confirm in the _Save Changes to Live Version_ dialog. To test a change before your app serves it, click **New version**, make the change in the new version, and evaluate the new version.

## Start an offline run

In an offline run, Statsig runs versions on every row of the config's dataset and grades each output. The config needs a dataset and at least one evaluator first.

1. In the config's header, click **Run Evaluators**.
2. In the _Start Evaluation_ dialog, select the versions to evaluate. Save any changes to a version before you evaluate it.
3. Click **Start Evaluation**.

While a run is in progress, **Run Evaluators** becomes **Stop Evaluation**. Results appear on the _Runs_ tab. For how to read them, go to [Offline Evaluation](https://docs.statsig.com/agent-experimentation/offline-evaluation).

## Create an experiment from a config

An experiment serves different versions of a config to different users and compares their metrics. You need at least two versions that are Live or Candidate.

1. In the config's header, click **Create Experiment**.
2. In the _Create Experiment_ dialog, select at least two Live or Candidate versions, then click **Create Experiment**.
3. In the experiment form, choose the **Control** version and one or more **Treatment** versions.
4. Name the experiment, add a hypothesis, and create the experiment.

Statsig creates the experiment in the config's layer. The control group serves the Control version. Each Treatment version gets a test group, and Statsig splits traffic evenly across the groups. For each evaluator on the config, Statsig adds a primary metric named after the experiment and the evaluator.

The experiment appears in the config's header and in the _Linked Experiments_ list in the _Details_ panel. When you create an experiment from the _Experiments_ page instead, the form shows a "Testing prompts or agents?" notice that links to Agent Experimentation.

## Other config actions

Besides **Run Evaluators** and **Create Experiment**, the config's header has these actions. **History**, **Edit Targeting Rules**, and **Code Snippets** are icon buttons. The other actions are in the header's menu.

| Action | What it does |
| --- | --- |
| **History** | Shows the config's change history. |
| **Edit Targeting Rules** | Serves a different config to users who match a rule. Each rule has conditions, a pass percentage, and a target config. A config can't target itself or a config that targets it. |
| **Code Snippets** | Shows SDK code that fetches the config. |
| **Clone** | Copies the Live version, evaluators, and dataset into a new config. |
| **Delete** | Deletes the config. You can't delete a config that another config's targeting rules reference. |

The _Included Credits_ indicator next to the tabs shows the percentage of your Included Credits that your company has used. Hover over the indicator to find the **Add API key** link. The indicator disappears when your company has an API key for every provider in _AI Provider API Keys_.

## Review results on the Runs and Traces tabs

- **Runs**: The results of offline runs for one version, or for two versions side by side. For details, go to [Offline Evaluation](https://docs.statsig.com/agent-experimentation/offline-evaluation).
- **Traces**: The traces your app sends for the config, with online evaluator scores. For details, go to [Traces](https://docs.statsig.com/agent-experimentation/traces) and [Online Evaluation](https://docs.statsig.com/agent-experimentation/online-evaluation).

## Serve a config in your app

Your app fetches a config by its ID with a Statsig AI SDK, then reads the Live version or the Candidate versions. In the [Node AI SDK](https://docs.statsig.com/agent-experimentation/node#getting-a-prompt), call `getPrompt`. In the [Python AI SDK](https://docs.statsig.com/agent-experimentation/python#getting-a-prompt), call `get_prompt`. To copy code for a specific config, click **Code Snippets** in the config's header.

## Manage configs programmatically

To create, edit, and evaluate configs from code or from an AI agent, use the Console API or the [Statsig MCP server](https://docs.statsig.com/integrations/mcp/overview). The Console API's config endpoints are in the Prompts section of the reference, starting with [List Prompts](https://docs.statsig.com/api-reference/prompts/list-prompts).