---
title: Datasets
description: "Create and edit datasets for Statsig Agent Experimentation runs: upload a CSV, mark output and image columns, categorize rows, and add production traces as rows."
product: general
lang: en
token_estimate: 2570
---
# Datasets

A dataset is a table of test cases for Statsig Agent Experimentation runs to grade. Each row is one test case, such as a user question and the answer you expect. Each column is a variable that prompts and evaluators reference by name, such as `{{input}}`. Use a dataset to grade a [config](https://docs.statsig.com/agent-experimentation/configs) version, or outputs your agent already produced, against the same cases every time. To grade live production traffic instead, use [online evaluation](https://docs.statsig.com/agent-experimentation/online-evaluation).

> For AI agents: a documentation index is available at [/llms.txt](/llms.txt). Append `.md` to any page URL for markdown, or send `Accept: text/markdown`.

A dataset holds test cases, not evaluators. Runs of a config's versions use the evaluators attached to the config. A run on the _Runs_ page uses the evaluators you pick when you create the run. Refer to [Choose where an evaluator runs](https://docs.statsig.com/agent-experimentation/evaluators#choose-where-a-grader-runs).

## Global and local datasets

Agent Experimentation has two kinds of dataset:

| Kind | Where you create it | Listed on the _Datasets_ page |
| --- | --- | --- |
| Global | The _Datasets_ page, the _Traces_ page, or **Make Dataset Global** in a config | Yes |
| Local | A config's _Run Setup_, or a new local dataset for a run on the _Runs_ page | No |

Any config or run can use a global dataset. A local dataset belongs to the config or run that created it.

## Create a dataset from a CSV file

1. Go to _Agent Experimentation > Datasets_ and select **Create**.
2. Enter a **Dataset Name** and an optional **Description**. Each global dataset in a project needs a unique name.
3. Drag a CSV file onto the upload area, or select **Upload CSV** and choose the file.
4. Check the preview, then select **Upload**.

Statsig creates a global dataset and opens its page. To mark output or image columns, use the column menu after the upload. Refer to [Output and image columns](#output-and-image-columns).

### CSV format

The first row of the file holds the column names, and each later row becomes one test case:

```csv
input,expected_output,categories
"My order arrived damaged. What should I do?","Apologize and offer a replacement.","shipping,refunds"
How do I reset my password?,Send the password reset link.,account
```

Statsig converts column names to lowercase, replaces spaces with underscores, and removes every other character that isn't a letter, digit, or underscore. For example, a column named `Expected Answer` becomes `expected_answer`, which you reference as `{{expected_answer}}`.

Two column names have special behavior:

| Column | What Statsig does with it |
| --- | --- |
| `categories` | Reads a comma-separated list of categories for each row and stores them as row categories, not as a regular column. Optional. |
| `annotations` | Suggests categories from the annotation text when more than five rows have a value. Optional. Refer to [Suggested categories](#suggested-categories). |

The upload accepts one CSV file of up to 2 MB and 1,000 rows. Statsig skips empty rows.

## Add a dataset to a config

Every version of a [config](https://docs.statsig.com/agent-experimentation/configs) uses the same dataset for its offline runs. To choose that dataset, open the config's _Setup_ tab. In the _Run Setup_ group, under _Dataset_, select **Select a Dataset**, then choose one option:

- **Upload Dataset**: Upload a CSV file as the config's local dataset.
- **Browse from Existing Datasets**: Search the global datasets and pick one.
- **AI Dataset Generation**: Describe the test cases you want in a prompt and set **Total rows**, from the default of 20 up to 300. Select **Preview Dataset** to inspect the rows, or **Apply Dataset** to use them as the config's local dataset.
- **Make Dataset Global**: Convert the config's local dataset into a global dataset. The dataset then appears on the _Datasets_ page, and other configs and runs can use it.

The dataset table under _Dataset_ works like the table on a dataset's page, and you save its edits with the config. Each kind of change affects the dataset differently:

- **Upload or generate a dataset**: Statsig replaces the rows of the config's local dataset. If the config uses a global dataset, Statsig detaches it and creates a new local dataset. The global dataset doesn't change.
- **Switch to another dataset**: Statsig deletes the rows of the config's local dataset. To keep those rows, select **Make Dataset Global** before you switch.
- **Edit a global dataset**: Edits you make from inside a config change that dataset for every config and run that uses it.

Each upload, generated dataset, or edit starts a new dataset version for the config. The config's _Runs_ tab compares two runs only when both used the same dataset version.

## Edit rows and columns

Open a dataset from the _Datasets_ page to edit it on the _Setup_ tab. Your edits apply only after you select **Save** in the save bar, and **Discard Changes** reverts them. If you try to leave the page with unsaved changes, Statsig asks you to confirm.

To work with rows:

- Filter the table with **Search**.
- Select **Add Row**, fill in each column in the drawer, and confirm the row.
- Select a row to open it in a drawer. Select **Edit** on a column, change the value, and select **Save**. Press the up and down arrow keys to move between rows.
- Select a row's delete icon to remove it. A dataset must keep at least one row.

To work with columns, open the menu in a column header:

- **Add Column**: Add a new column.
- **Mark as Output** or **Unmark as Output**: Set or clear the column's output flag.
- **Mark as Image** or **Unmark as Image**: Set or clear the column's image flag.
- **Delete Column**: Remove the column. A dataset must keep at least one column.

To rename a column, select its name and type a new one. Each column needs a unique name that isn't empty.

### Output and image columns

An icon in the column header shows each flag:

- **Output column**: Holds an output your agent already produced. A run on the _Runs_ page grades the values in the output column. A run can use a dataset only after you mark at least one output column. Runs that you start from a config generate their own output, so they don't need an output column.
- **Image column**: Holds image URLs. Statsig sends those images to LLM-as-a-judge evaluators along with the grading prompt.

When you create a local dataset for a run on the _Runs_ page, the upload drawer also shows **Output Columns** and **Image Columns**. That drawer requires at least one output column.

## Categorize rows

Categories group rows so a run on the _Runs_ page can show scores for each category. A row's categories appear in the `categories` column at the end of the table.

- **Add a category to a row**: Hover over the row's `categories` cell, select the add icon, and pick a category. To create a category, type a name of up to 30 characters and select the **Create** option.
- **Remove a category from a row**: Select the delete icon on the category chip.
- **Rename or delete categories**: Select **Manage Categories** at the bottom of the category picker. Edit the list, select **Add New Category** to add one, and then select **Save**.

### Suggested categories

Statsig suggests categories when you create a dataset in which more than five rows have an `annotations` value. An LLM reads each annotation and proposes short categories for its row. Datasets you create from annotated traces often qualify. The console calls these suggestions tags.

After the suggestions are ready, the toolbar shows two options:

- **Apply all suggested tags**: Add every suggested category to its rows.
- **Review Results**: Open _Suggested Tags_, which lists each suggestion with the annotations behind it. Clear the suggestions you don't want, and then apply the rest.

When any row has an `annotations` value, **All** and **Annotated** tabs appear above the table. The **Annotated** tab shows only the rows that have annotations.

## Add traces from the Traces page

The [Traces](https://docs.statsig.com/agent-experimentation/traces) page can turn production spans into dataset rows:

1. Go to _Agent Experimentation > Traces_.
2. Select one or more spans and select **Add to Dataset**. You can also open a trace and select **Add to dataset**.
3. Pick a global dataset, or find one with **Search datasets**. To start a new dataset, select **Create new Dataset**, enter a **Dataset Name**, and select **Create**.

Each selected span becomes one row with these columns:

| Column | Value |
| --- | --- |
| `input` | The span's input messages from `gen_ai.input.messages`, one `role: text` line per message. |
| `output` | The text of the final assistant message in `gen_ai.output.messages`, without tool calls or earlier steps. |
| `metadata` | JSON with the model, token counts, config name (`promptName`), trace and span IDs, status, and timing. |
| `annotations` | JSON list of the span's annotations. Statsig adds this column only when the span has annotations. |

The span's tags become the row's categories. When you add spans to an existing dataset, Statsig adds any of these columns that the dataset lacks. The dataset keeps its output and image flags. A new dataset from the _Traces_ page has no output column. To use it in a run on the _Runs_ page, select **Mark as Output** on the `output` column.

## View the runs that used a dataset

A dataset's _Results_ tab lists the runs on the _Runs_ page that used the dataset, with columns for Score, Run, Run Group, and Started. A run that's still grading shows a Running badge, and a failed run shows an Error badge. Runs that you start from a config don't appear here; their results are on the config's _Runs_ tab. Refer to [Offline Evaluation](https://docs.statsig.com/agent-experimentation/offline-evaluation).

## Export or delete a dataset

Open the more-actions menu in the dataset's header:

- **Export to CSV**: Download the rows as a CSV file. When rows have categories, the file includes a `categories` column, so you can upload it again. The file doesn't record output or image flags.
- **Delete**: Delete the dataset after you confirm.