---
title: Datasets
description: "Create and edit datasets for Statsig Agent Experimentation evals: upload a CSV, mark output and image columns, categorize rows, and add production traces as test cases."
product: general
lang: en
token_estimate: 2547
---
# Datasets

A dataset is a table of test cases that Statsig Agent Experimentation evals grade. Each row is one test case, such as a user question and the answer you expect. Each column is a variable that prompts and graders reference by name, such as `{{input}}`. Use a dataset to grade an AI Config version, or outputs your agent already produced, against the same cases every time. To grade live production traffic instead, use [online evals](https://docs.statsig.com/ai-evals/online-evals).

> For AI agents: a documentation index is available at [/llms.txt](/llms.txt). Append `.md` to any page URL for markdown, or send `Accept: text/markdown`.

A dataset holds test cases, not graders. An AI Config's version evals use the graders attached to the AI Config. A run on the _Evals_ page uses the graders you pick when you create the run. Refer to [Choose where a grader runs](https://docs.statsig.com/ai-evals/graders#choose-where-a-grader-runs).

## Global and local datasets

Agent Experimentation has two kinds of dataset:

| Kind | Where you create it | Listed on the _Datasets_ page |
| --- | --- | --- |
| Global | The _Datasets_ page, AI Traces, or **Make Dataset Global** in an AI Config | Yes |
| Local | An AI Config's _Eval Setup_, or a new local dataset for a run on the _Evals_ page | No |

Any AI Config or eval can use a global dataset. A local dataset belongs to the AI Config or run that created it.

## Create a dataset from a CSV file

1. Go to _Agent Experimentation > Datasets_ and select **Create**.
2. Enter a **Dataset Name** and an optional **Description**. Each global dataset in a project needs a unique name.
3. Drag a CSV file onto the upload area, or select **Upload CSV** and choose the file.
4. Check the preview, then select **Upload**.

Statsig creates a global dataset and opens its page. To mark output or image columns, use the column menu after the upload. Refer to [Output and image columns](#output-and-image-columns).

### CSV format

The first row of the file holds the column names, and each later row becomes one test case:

```csv
input,expected_output,categories
"My order arrived damaged. What should I do?","Apologize and offer a replacement.","shipping,refunds"
How do I reset my password?,Send the password reset link.,account
```

Statsig converts column names to lowercase, replaces spaces with underscores, and removes every other character that isn't a letter, digit, or underscore. For example, a column named `Expected Answer` becomes `expected_answer`, which you reference as `{{expected_answer}}`.

Two column names have special behavior:

| Column | What Statsig does with it |
| --- | --- |
| `categories` | Reads a comma-separated list of categories for each row and stores them as row categories, not as a regular column. Optional. |
| `annotations` | Suggests categories from the annotation text when more than five rows have a value. Optional. Refer to [Suggested categories](#suggested-categories). |

The upload accepts one CSV file of up to 2 MB and 1,000 rows. Statsig skips empty rows.

## Add a dataset to an AI Config

Every version of an [AI Config](https://docs.statsig.com/ai-evals/prompts) runs its offline evals against the same dataset. To choose that dataset, open the AI Config's _Setup_ tab. In the _Eval Setup_ group, under _Dataset_, select **Select a Dataset**, then choose one option:

- **Upload Dataset**: Upload a CSV file as the AI Config's local dataset.
- **Browse from Existing Datasets**: Search the global datasets and pick one.
- **AI Dataset Generation**: Describe the test cases you want in a prompt and set **Total rows**, from the default of 20 up to 300. Select **Preview Dataset** to inspect the rows, or **Apply Dataset** to use them as the AI Config's local dataset.
- **Make Dataset Global**: Convert the AI Config's local dataset into a global dataset. The dataset then appears on the _Datasets_ page, and other AI Configs and evals can use it.

The dataset table under _Dataset_ works like the table on a dataset's page, and you save its edits with the AI Config. Each kind of change affects the dataset differently:

- **Upload or generate a dataset**: Statsig replaces the rows of the AI Config's local dataset. If the AI Config uses a global dataset, Statsig detaches it and creates a new local dataset. The global dataset doesn't change.
- **Switch to another dataset**: Statsig deletes the rows of the AI Config's local dataset. To keep those rows, select **Make Dataset Global** before you switch.
- **Edit a global dataset**: Edits you make from inside an AI Config change that dataset for every AI Config and eval that uses it.

Each upload, generated dataset, or edit starts a new dataset version for the AI Config. The AI Config's _Evals_ tab compares two runs only when both used the same dataset version.

## Edit rows and columns

Open a dataset from the _Datasets_ page to edit it on the _Setup_ tab. Your edits apply only after you select **Save** in the save bar, and **Discard Changes** reverts them. If you try to leave the page with unsaved changes, Statsig asks you to confirm.

To work with rows:

- Filter the table with **Search**.
- Select **Add Row**, fill in each column in the drawer, and confirm the row.
- Select a row to open it in a drawer. Select **Edit** on a column, change the value, and select **Save**. Press the up and down arrow keys to move between rows.
- Select a row's delete icon to remove it. A dataset must keep at least one row.

To work with columns, open the menu in a column header:

- **Add Column**: Add a new column.
- **Mark as Output** or **Unmark as Output**: Set or clear the column's output flag.
- **Mark as Image** or **Unmark as Image**: Set or clear the column's image flag.
- **Delete Column**: Remove the column. A dataset must keep at least one column.

To rename a column, select its name and type a new one. Each column needs a unique name that isn't empty.

### Output and image columns

An icon in the column header shows each flag:

- **Output column**: Holds an output your agent already produced. A run on the _Evals_ page grades the values in the output column. A run can use a dataset only after you mark at least one output column. Evals that you start from an AI Config generate their own output, so they don't need an output column.
- **Image column**: Holds image URLs. Statsig sends those images to LLM-as-a-judge graders along with the grading prompt.

When you create a local dataset for a run on the _Evals_ page, the upload drawer also shows **Output Columns** and **Image Columns**. That drawer requires at least one output column.

## Categorize rows

Categories group rows so a run on the _Evals_ page can show scores for each category. A row's categories appear in the `categories` column at the end of the table.

- **Add a category to a row**: Hover over the row's `categories` cell, select the add icon, and pick a category. To create a category, type a name of up to 30 characters and select the **Create** option.
- **Remove a category from a row**: Select the delete icon on the category chip.
- **Rename or delete categories**: Select **Manage Categories** at the bottom of the category picker. Edit the list, select **Add New Category** to add one, and then select **Save**.

### Suggested categories

Statsig suggests categories when you create a dataset in which more than five rows have an `annotations` value. An LLM reads each annotation and proposes short categories for its row. Datasets you create from annotated traces often qualify. The console calls these suggestions tags.

After the suggestions are ready, the toolbar shows two options:

- **Apply all suggested tags**: Add every suggested category to its rows.
- **Review Results**: Open _Suggested Tags_, which lists each suggestion with the annotations behind it. Clear the suggestions you don't want, and then apply the rest.

When any row has an `annotations` value, **All** and **Annotated** tabs appear above the table. The **Annotated** tab shows only the rows that have annotations.

## Add traces from AI Traces

[AI Traces](https://docs.statsig.com/ai-evals/ai-traces) can turn production spans into test cases:

1. Go to _Agent Experimentation > AI Traces_.
2. Select one or more spans and select **Add to Dataset**. You can also open a trace and select **Add to dataset**.
3. Pick a global dataset, or find one with **Search datasets**. To start a new dataset, select **Create new Dataset**, enter a **Dataset Name**, and select **Create**.

Each selected span becomes one row with these columns:

| Column | Value |
| --- | --- |
| `input` | The span's input messages from `gen_ai.input.messages`, one `role: text` line per message. |
| `output` | The text of the final assistant message in `gen_ai.output.messages`, without tool calls or earlier steps. |
| `metadata` | JSON with the model, token counts, AI Config name (`promptName`), trace and span IDs, status, and timing. |
| `annotations` | JSON list of the span's annotations. Statsig adds this column only when the span has annotations. |

The span's tags become the row's categories. When you add spans to an existing dataset, Statsig adds any of these columns that the dataset lacks. The dataset keeps its output and image flags. A new dataset from AI Traces has no output column. To use it in a run on the _Evals_ page, select **Mark as Output** on the `output` column.

## View the runs that used a dataset

A dataset's _Results_ tab lists the runs on the _Evals_ page that used the dataset, with columns for Score, Run, Eval, and Started. A run that's still grading shows a Running badge, and a failed run shows an Error badge. Evals that you start from an AI Config don't appear here; their results are on the AI Config's _Evals_ tab. Refer to [Offline evals](https://docs.statsig.com/ai-evals/offline-evals).

## Export or delete a dataset

Open the more-actions menu in the dataset's header:

- **Export to CSV**: Download the rows as a CSV file. When rows have categories, the file includes a `categories` column, so you can upload it again. The file doesn't record output or image flags.
- **Delete**: Delete the dataset after you confirm.