---
title: Traces
description: "Inspect, filter, and review the LLM spans your app sends to Statsig in Traces, and look up which span attributes power each view, filter, and online evaluator."
product: general
lang: en
token_estimate: 3689
---
# Traces

Traces lists the LLM spans your app sends to Statsig: model calls, tool calls, and agent steps, each with its messages, model, token count, cost, and evaluator scores. Use Traces to debug a single agent turn, find slow or failing calls, review outputs by hand, and save useful examples to a dataset. To query every span your services send, including HTTP and database spans, use the [Traces Explorer](https://docs.statsig.com/infra-analytics/send-traces) instead.

> For AI agents: a documentation index is available at [/llms.txt](/llms.txt). Append `.md` to any page URL for markdown, or send `Accept: text/markdown`.

## Where traces appear

Statsig shows LLM spans in three places in the console:

- **Traces page**: Every LLM span in your project. Open it from _Agent Experimentation > Traces_.
- **Traces tab on a [config](https://docs.statsig.com/agent-experimentation/configs)**: Spans tagged with that config, with a filter for the config version. The tab also shows online evaluator results, described in [Online Evaluation](https://docs.statsig.com/agent-experimentation/online-evaluation).
- **Traces tab on an evaluator**: Spans that the evaluator scored. The tab opens on the last day of spans.

In each of these views, click a span to open the trace drawer.

## Send traces to the Traces page

The _Traces_ page shows only spans whose `statsig.span.type` attribute is `gen_ai`. Spans you send by following the generic [Traces Explorer quick start](https://docs.statsig.com/infra-analytics/send-traces) don't carry that attribute, so they appear in the Traces Explorer but not on the _Traces_ page.

Choose one way to send LLM spans:

- **Node AI SDK**: Call `initializeTracing()` and wrap your model client with `wrapOpenAI`, `wrapAnthropic`, or `wrapGemini`. The SDK sets the required attributes for you. The SDK doesn't record message content by default. Turn on `captureOptions` to see messages in the trace drawer and to make spans eligible for online grading. Call the model inside `withLive()` or `withVersion()` so each span records its config and version. Go to [Node AI SDK](https://docs.statsig.com/agent-experimentation/node) for setup.
- **Raw OpenTelemetry**: Export spans to Statsig over OTLP/HTTP, as the [Traces Explorer quick start](https://docs.statsig.com/infra-analytics/send-traces) shows, and set the attributes in the [span attribute reference](#span-attribute-reference) on each LLM span yourself.

The following raw OpenTelemetry span sets the attributes that the _Traces_ page and online grading read. OpenTelemetry attributes can't hold nested objects, so serialize message arrays as JSON strings. `callModel` stands in for your own model call.

```js
import { trace } from "@opentelemetry/api";

const tracer = trace.getTracer("support-bot");

await tracer.startActiveSpan("chat gpt-5", async (span) => {
  const answer = await callModel(question);
  span.setAttributes({
    "statsig.span.type": "gen_ai",
    "statsig.span.gen_ai.type": "generate_content",
    "statsig.span.llm_root": "1", // a string, not a number
    "statsig.llm.prompt_name": "support_bot",
    "gen_ai.request.model": "gpt-5",
    "gen_ai.input.messages": JSON.stringify([
      { role: "user", parts: [{ type: "text", content: question }] },
    ]),
    "gen_ai.output.messages": JSON.stringify([
      { role: "assistant", parts: [{ type: "text", content: answer.text }] },
    ]),
    "gen_ai.usage.input_tokens": answer.inputTokens,
    "gen_ai.usage.output_tokens": answer.outputTokens,
  });
  span.end();
});
```

## Find spans in the explorer

The _Traces_ page opens on the past hour of spans. The search bar starts with `#environment!:eval,shadow,online_eval_judge`, which hides the spans Statsig generates itself. Remove that filter to include those spans. For what each value means, go to [Environments](#environments).

The span scope menu next to the search bar sets which spans the list includes:

- **All Spans**: Every LLM span.
- **Trace Root Spans**: Only the first span in each trace. A trace whose first span isn't an LLM span, such as an HTTP request span that wraps your model calls, doesn't appear in this view.

### Columns

| Column | Shows |
| --- | --- |
| Timestamp | When the span started. |
| Duration | How long the span took. |
| Name | The span name. |
| Service | The service that sent the span, from the `service.name` resource attribute. |
| Config | The config the span belongs to, from `statsig.llm.prompt_name`. |
| Evaluators | One chip per evaluator score, or `Not graded` when the span has no scores. |

Each evaluator chip shows the evaluator name and `pass` or `fail`. A score above 0.66 counts as `pass`. The trace drawer shows the exact score.

### Filters

Filters in the left rail narrow the span list.

| Filter | Filters on |
| --- | --- |
| Duration (ms) | Span duration. |
| Config | The config, from `statsig.llm.prompt_name`. |
| Evaluator | The evaluators that scored the span. |
| Environment | The `statsig.environment` attribute. Only equality filters work. |
| Status | Span status, such as `error`. |
| Span name | The span name. |
| Service | The `service.name` resource attribute. |
| Cluster | The `cluster_name` resource attribute. |
| Has activity ID | Whether the span has a `statsig.activity_id` custom ID. |

## Inspect a trace

Click a span in the list to open the trace drawer. The drawer header shows the trace ID, duration, total tokens, cost in USD, and service, plus the review actions described in [Review and save spans](#review-and-save-spans).

### Span tree

The left side of the drawer lists every span in the trace. Use **Search spans** to filter the list by span name, and switch between **Tree** and **Timeline** to show the spans as a hierarchy or on a time axis. Each row shows an icon for the span type (agent or workflow, tool, model call, or step), the span name, the tool or config name, duration, cost, and up to three evaluator chips.

When the selected span has a `statsig.activity_id` custom ID, the drawer loads every span with that activity ID, even across traces. Use an activity ID to group a multi-turn conversation or a long-running job into one view.

### Trace tab

The **Trace** tab shows the selected span's messages. Switch between two views:

- **Formatted**: A chat transcript that Statsig builds from `gen_ai.system_instructions`, `gen_ai.input.messages`, and `gen_ai.output.messages`. Each turn has a role label, such as System, User, Assistant, or Tool. A tool call appears as `Function call` with the tool name and its input, and a tool result appears as `Function output`. The tab shows `This span has no messages.` when the span has no message attributes.
- **JSON**: Input and Output cards with the raw messages, and a Metadata card with every span attribute. Attributes that hold JSON strings expand into objects.

### Details tab

The **Details** tab lists the span's Start, Duration, Service, Model, Agent, Trace ID, Tokens, and Cost (USD). Model, Agent, and Tokens appear only when the span sets the matching attribute. Go to the [span attribute reference](#span-attribute-reference) for the source of each field.

### Evaluators tab

The **Evaluators** tab shows one card per score on the selected span. The tab label shows a count of scores. Each card shows:

- The evaluator name and its score.
- A label for the evaluator type, such as LLM as a Judge, String Comparison, Text Similarity, Python, or Programmatic Grader. Scores with no evaluator type show their source instead: Online evaluator, Offline run, SDK, User feedback, Reviewer, or Other.
- The judge's reasoning steps, for LLM-as-a-judge scores.
- **View runs**, for scores from a run. The link opens that run.

A span with no scores shows `No evaluator scores this span.` If your organization runs out of Included Credits, the tab explains that Statsig didn't grade the span and how to add your own API key.

## Review and save spans

Use review actions to mark spans for follow-up and to build datasets from production traffic. Actions in the drawer header apply to the open span.

- **Thumbs up or thumbs down**: Record a human review of the span.
- **Add tag**: Apply an existing tag, or choose **Create New Tag**. Choose **Manage Tags** to rename or remove tags.
- **Add Annotation**: Write a comment on the span. Edit or delete an annotation from its menu.
- **Add to dataset**: Choose a dataset, or choose **Create new Dataset** and enter a **Dataset Name**.

To act on many spans at once, hover over a row in the list and select its checkbox. A bar shows the number of selected spans with **Add to Dataset**, **Add Tag**, and thumbs up or down.

When you add spans to a dataset, each span becomes a row. The `input` column holds the input messages, the `output` column holds the final answer, and the `metadata` column holds the model, token counts, and trace and span IDs. Spans with annotations add an `annotations` column, and a span's tags become categories on its row.

## Environments

The `statsig.environment` span attribute records which deployment environment a span came from. The _Traces_ page, the Environment filter, and online grading all read this attribute.

A span gets its `statsig.environment` value in one of two ways:

- The span sets `statsig.environment` itself.
- The span doesn't set it, and the `deployment.environment.name` resource attribute names a non-production environment, such as `staging`. Statsig copies that name into `statsig.environment`.

The Node AI SDK sets `deployment.environment.name` from the `environment` option of `initializeTracing()`, or from `NODE_ENV` when you don't pass one.

Production spans usually have no `statsig.environment` value, because Statsig copies only non-production names. Online grading treats a span with no value as production, and can grade spans from any of your environments.

Statsig reserves three values for spans it generates itself. The _Traces_ page hides these spans by default, and online grading never grades them.

| Value | Spans |
| --- | --- |
| `eval` | Offline runs, one trace per dataset row. |
| `online_eval_judge` | The model calls an online evaluator makes while it scores your spans. |
| `shadow` | Other runs that Statsig generates. |

Don't use these values for your own spans.

## Span attribute reference

These span attributes control what the _Traces_ page shows and which spans online grading can score. The Node AI SDK sets them for you. If you send raw OpenTelemetry spans, set them yourself.

### Span type and structure

| Attribute | Value | What it powers |
| --- | --- | --- |
| `statsig.span.type` | `gen_ai` | Required. Only spans with this value appear on the _Traces_ page, in the Traces tabs, and in online grading. |
| `statsig.span.gen_ai.type` | `workflow`, `agent`, `tool`, `generate_content`, `step` | The icon in the span tree. Online grading grades `workflow` spans and never grades `tool` spans. Without this attribute, Statsig reads `gen_ai.operation.name`, then the first word of the span name, such as `execute_tool` or `invoke_agent`. |
| `statsig.span.llm_root` | `"1"` (string) | Marks the outermost LLM call in an agent turn. Online grading grades an LLM span only when it carries this marker or is a `workflow` span. |

### Config

| Attribute | Value | What it powers |
| --- | --- | --- |
| `statsig.llm.prompt_name` | The config name you pass to `getPrompt()` | The Config column and filter, the config's Traces tab, and the **Config** condition in online rules. |
| `statsig.llm.prompt_version_id` | The config version ID | The version filter on the config's Traces tab, and the version online grading assigns each score to. Older SDKs send `statsig.llm.prompt_version_name` or `statsig.llm.prompt_version`. |

The SDK, the API, and these attribute names use `prompt`, the earlier name for a config.

### Messages

Set message attributes to JSON strings. The _Traces_ page accepts OpenTelemetry GenAI messages (`role` with `parts`) and OpenAI chat messages (`role` with `content`).

| Attribute | What it powers |
| --- | --- |
| `gen_ai.system_instructions` | System turns in the Formatted view. |
| `gen_ai.input.messages` | Input turns in the Formatted view, the Input card in the JSON view, the dataset `input` column, and the `{{input}}` evaluator variable. Online grading skips spans without input messages. |
| `gen_ai.output.messages` | Output turns in the Formatted view and the Output card. The final assistant answer becomes the dataset `output` column and the `{{output}}` evaluator variable. The tool calls and results before it become `{{trajectory}}`. Online grading skips spans without output messages. |

### Model, tokens, and cost

| Attribute | What it powers |
| --- | --- |
| `gen_ai.request.model` | Model in the Details tab. Statsig also uses it to estimate cost when `gen_ai.response.model` is missing. |
| `gen_ai.response.model` | The model Statsig uses first to estimate cost. |
| `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens` | Cost estimates. |
| `gen_ai.usage.input_cached_tokens` or `gen_ai.usage.cache_read.input_tokens` | Cost estimates. Statsig prices cached input tokens at the model's cached rate. |
| `statsig.gen_ai.usage.total_tokens`, `gen_ai.usage.total_tokens` | Tokens in the Details tab. |
| `statsig.gen_ai.usage.cost_usd` | Cost (USD) in the drawer and the span tree. If the span doesn't set a cost, Statsig estimates one from the model and token counts for models it has prices for. A cost you send always takes precedence. |

### Agents and tools

| Attribute | What it powers |
| --- | --- |
| `gen_ai.agent.name` | Agent in the Details tab. |
| `gen_ai.tool.name` | The tool name on the span's row in the span tree. |
| `gen_ai.operation.name` | The span type icon, when the span doesn't set `statsig.span.gen_ai.type`. |

### Grouping and environment

| Attribute | What it powers |
| --- | --- |
| `statsig.environment` | The Environment filter, the default filter that hides Statsig-generated spans, and which spans online grading skips. Go to [Environments](#environments). |
| `statsig.custom_ids` | Custom IDs for the span, as a JSON string. Include a `statsig.activity_id` key, for example `{"statsig.activity_id":"conversation-123"}`, to power the Has activity ID filter and group spans across traces in the drawer. |

### Resource attributes

Set these on the OpenTelemetry resource, not on individual spans.

| Attribute | What it powers |
| --- | --- |
| `service.name` | The Service column and filter, and Service in the drawer. |
| `cluster_name` | The Cluster filter. |
| `deployment.environment.name` | The environment. Statsig copies non-production names into `statsig.environment`. Go to [Environments](#environments). |