AI Traces

AI Traces lists the LLM spans your app sends to Statsig: model calls, tool calls, and agent steps, each with its messages, model, token count, cost, and grader scores. Use AI Traces to debug a single agent turn, find slow or failing calls, review outputs by hand, and save useful examples to a dataset. To query every span your services send, including HTTP and database spans, use the Traces Explorer instead.

Where traces appear

Statsig shows LLM spans in three places in the console:

• AI Traces page: Every LLM span in your project. Open it from Agent Experimentation > AI Traces.
• Traces tab on an AI Config: Spans tagged with that AI Config, with a filter for the AI Config version. The tab also shows online grader results, described in Online Evals.
• Traces tab on a grader: Spans that the grader scored. The tab opens on the last day of spans.

In each of these views, click a span to open the trace drawer.

Send traces to AI Traces

AI Traces shows only spans whose statsig.span.type attribute is gen_ai. Spans you send by following the generic Traces Explorer quick start don't carry that attribute, so they appear in the Traces Explorer but not in AI Traces.

Choose one way to send LLM spans:

• Node AI SDK: Call initializeTracing() and wrap your model client with wrapOpenAI, wrapAnthropic, or wrapGemini. The SDK sets the required attributes for you. The SDK doesn't record message content by default. Turn on captureOptions to see messages in the trace drawer and to make spans eligible for online grading. Call the model inside withLive() or withVersion() so each span records its AI Config and version. Go to Node AI SDK for setup.
• Raw OpenTelemetry: Export spans to Statsig over OTLP/HTTP, as the Traces Explorer quick start shows, and set the attributes in the span attribute reference on each LLM span yourself.

The following raw OpenTelemetry span sets the attributes that AI Traces and online grading read. OpenTelemetry attributes can't hold nested objects, so serialize message arrays as JSON strings. callModel stands in for your own model call.

import { trace } from "@opentelemetry/api";

const tracer = trace.getTracer("support-bot");

await tracer.startActiveSpan("chat gpt-5", async (span) => {
  const answer = await callModel(question);
  span.setAttributes({
    "statsig.span.type": "gen_ai",
    "statsig.span.gen_ai.type": "generate_content",
    "statsig.span.llm_root": "1", // a string, not a number
    "statsig.llm.prompt_name": "support_bot",
    "gen_ai.request.model": "gpt-5",
    "gen_ai.input.messages": JSON.stringify([
      { role: "user", parts: [{ type: "text", content: question }] },
    ]),
    "gen_ai.output.messages": JSON.stringify([
      { role: "assistant", parts: [{ type: "text", content: answer.text }] },
    ]),
    "gen_ai.usage.input_tokens": answer.inputTokens,
    "gen_ai.usage.output_tokens": answer.outputTokens,
  });
  span.end();
});

Find spans in the explorer

The AI Traces page opens on the past hour of spans. The search bar starts with #environment!:eval,shadow,online_eval_judge, which hides the spans Statsig generates itself. Remove that filter to include those spans. For what each value means, go to Environments.

The span scope menu next to the search bar sets which spans the list includes:

• All Spans: Every LLM span.
• Trace Root Spans: Only the first span in each trace. A trace whose first span isn't an LLM span, such as an HTTP request span that wraps your model calls, doesn't appear in this view.

Columns

Each grader chip shows the grader name and pass or fail. A score above 0.66 counts as pass. The trace drawer shows the exact score.

Filters

Filters in the left rail narrow the span list.

Inspect a trace

Click a span in the list to open the trace drawer. The drawer header shows the trace ID, duration, total tokens, cost in USD, and service, plus the review actions described in Review and save spans.

Span tree

The left side of the drawer lists every span in the trace. Use Search spans to filter the list by span name, and switch between Tree and Timeline to show the spans as a hierarchy or on a time axis. Each row shows an icon for the span type (agent or workflow, tool, model call, or step), the span name, the tool or AI Config name, duration, cost, and up to three grader chips.

When the selected span has a statsig.activity_id custom ID, the drawer loads every span with that activity ID, even across traces. Use an activity ID to group a multi-turn conversation or a long-running job into one view.

Trace tab

The Trace tab shows the selected span's messages. Switch between two views:

• Formatted: A chat transcript that Statsig builds from gen_ai.system_instructions, gen_ai.input.messages, and gen_ai.output.messages. Each turn has a role label, such as System, User, Assistant, or Tool. A tool call appears as Function call with the tool name and its input, and a tool result appears as Function output. The tab shows This span has no messages. when the span has no message attributes.
• JSON: Input and Output cards with the raw messages, and a Metadata card with every span attribute. Attributes that hold JSON strings expand into objects.

Details tab

The Details tab lists the span's Start, Duration, Service, Model, Agent, Trace ID, Tokens, and Cost (USD). Model, Agent, and Tokens appear only when the span sets the matching attribute. Go to the span attribute reference for the source of each field.

Graders tab

The Graders tab shows one card per score on the selected span. The tab label shows a count of scores. Each card shows:

• The grader name and its score.
• A label for the grader type, such as LLM as a Judge, String Comparison, Text Similarity, Python, or Programmatic Grader. Scores with no grader type show their source instead: Online eval, Offline eval, SDK, User feedback, Reviewer, or Other.
• The judge's reasoning steps, for LLM-as-a-judge scores.
• View eval, for scores from an eval run. The link opens that run.

A span with no scores shows No grader scores this span. If your organization runs out of Included Credits, the tab explains that Statsig didn't grade the span and how to add your own API key.

Review and save spans

Use review actions to mark spans for follow-up and to build datasets from production traffic. Actions in the drawer header apply to the open span.

• Thumbs up or thumbs down: Record a human review of the span.
• Add tag: Apply an existing tag, or choose Create New Tag. Choose Manage Tags to rename or remove tags.
• Add Annotation: Write a comment on the span. Edit or delete an annotation from its menu.
• Add to dataset: Choose a dataset, or choose Create new Dataset and enter a Dataset Name.

To act on many spans at once, hover over a row in the list and select its checkbox. A bar shows the number of selected spans with Add to Dataset, Add Tag, and thumbs up or down.

When you add spans to a dataset, each span becomes a row. The input column holds the input messages, the output column holds the final answer, and the metadata column holds the model, token counts, and trace and span IDs. Spans with annotations add an annotations column, and a span's tags become categories on its row.

Environments

The statsig.environment span attribute records which deployment environment a span came from. AI Traces, the Environment filter, and online grading all read this attribute.

A span gets its statsig.environment value in one of two ways:

• The span sets statsig.environment itself.
• The span doesn't set it, and the deployment.environment.name resource attribute names a non-production environment, such as staging. Statsig copies that name into statsig.environment.

The Node AI SDK sets deployment.environment.name from the environment option of initializeTracing(), or from NODE_ENV when you don't pass one.

Production spans usually have no statsig.environment value, because Statsig copies only non-production names. Online grading treats a span with no value as production, and can grade spans from any of your environments.

Statsig reserves three values for spans it generates itself. The AI Traces page hides these spans by default, and online grading never grades them.

Don't use these values for your own spans.

Span attribute reference

These span attributes control what AI Traces shows and which spans online grading can score. The Node AI SDK sets them for you. If you send raw OpenTelemetry spans, set them yourself.

Span type and structure

AI Config

The SDK, the API, and these attribute names use prompt, the earlier name for an AI Config.

Messages

Set message attributes to JSON strings. AI Traces accepts OpenTelemetry GenAI messages (role with parts) and OpenAI chat messages (role with content).

Model, tokens, and cost

Agents and tools

Grouping and environment

Resource attributes

Set these on the OpenTelemetry resource, not on individual spans.