---
title: Node AI SDK
description: "Use the Statsig Node AI SDK to serve Agent Experimentation configs, trace OpenAI, Anthropic, and Gemini calls with OpenTelemetry, log eval grades, and run evals from a Node.js server."
product: general
lang: en
token_estimate: 8239
---
# Node AI SDK

- **Repository:** [statsig-ai-node](https://github.com/statsig-io/statsig-ai-node)

The Statsig Node AI SDK (`@statsig/statsig-ai`) is the Node.js server SDK for Agent Experimentation. It serves configs, so you can change prompts and models without deploying code. These are Agent Experimentation configs, not [Dynamic Configs](https://docs.statsig.com/dynamic-config/overview). It also traces OpenAI, Anthropic, and Gemini calls to [Traces](https://docs.statsig.com/agent-experimentation/traces) as OpenTelemetry spans, logs eval grades that your code computes, and runs evals from code. The SDK builds on the [Statsig Node Server SDK](https://docs.statsig.com/server-core/node-core).

> For AI agents: a documentation index is available at [/llms.txt](/llms.txt). Append `.md` to any page URL for markdown, or send `Accept: text/markdown`.

The SDK uses `prompt`, the earlier name for a config, in its method names. For example, `getPrompt()` returns a config. The console shows `Eval()` scorers, and the `graderName` you pass to `logEvalGrade()`, as evaluators.

This page describes `@statsig/statsig-ai` version `0.0.1-beta.34`.

## Set up the SDK

1. **Install the SDK**

   Install the AI SDK and the Node Server SDK:

   #### npm

   ```shell
   npm install @statsig/statsig-ai @statsig/statsig-node-core
   ```

   #### pnpm

   ```shell
   pnpm add @statsig/statsig-ai @statsig/statsig-node-core
   ```

   #### yarn

   ```shell
   yarn add @statsig/statsig-ai @statsig/statsig-node-core
   ```

   > **Note:**
   >
   > Versions `0.0.1-beta.30` through `0.0.1-beta.33` load `@statsig/statsig-node-core-rc` at startup, but no package with that name exists on npm. To install one of those versions, add `@statsig/statsig-node-core-rc@npm:@statsig/statsig-node-core` to the install command. This alias installs the Node Server SDK under that name. Without it, the install fails or the import throws `Cannot find module '@statsig/statsig-node-core-rc'`.

   The Node Server SDK includes native binaries. If you use a frozen lockfile or bundle your server with webpack, esbuild, or Next.js, follow the [Node Server SDK setup steps](https://docs.statsig.com/server-core/node-core#set-up-the-sdk).
2. **Initialize the SDK**

   Initialize the AI SDK with a [Server Secret Key](https://docs.statsig.com/access-management/api-keys#server-secret-keys). The examples on this page read the key from the `STATSIG_SDK_KEY` environment variable, which `initializeTracing()` also reads.

   > **Warning:**
   >
   > Keep Server Secret Keys private. If you expose one, disable it and create a new key in the Statsig console.

   Choose one of these setups:

   #### Create a Statsig instance

   The AI SDK creates and owns its own Statsig instance. `statsigOptions` accepts any [Statsig option](https://docs.statsig.com/server-core/node-core#statsig-options).

   ```ts
   import { StatsigAI } from "@statsig/statsig-ai";

   const statsigAI = StatsigAI.newShared({
     sdkKey: process.env.STATSIG_SDK_KEY!,
     statsigOptions: { environment: "production" }, // optional
   });
   await statsigAI.initialize();
   ```

   #### Use an existing Statsig instance

   Pass in a Statsig instance that your app already initializes. The AI SDK doesn't initialize, flush, or shut down an instance you pass in.

   ```ts
   import { Statsig } from "@statsig/statsig-node-core";
   import { StatsigAI } from "@statsig/statsig-ai";

   const statsig = new Statsig(process.env.STATSIG_SDK_KEY!, {
     environment: "production",
   });
   await statsig.initialize();

   const statsigAI = StatsigAI.newShared({ statsig });
   ```

   `StatsigAI.newShared()` creates the instance and registers it as the shared instance. Get the shared instance anywhere in your app with `StatsigAI.shared()`. The tracing wrappers log each traced call as a `statsig::gen_ai` event through the shared instance. When no shared instance exists, they print a warning on every traced call. To create an instance that isn't shared, pass the same options to `new StatsigAI()`.

## Get a config

A config holds versions of a prompt and its model settings, so you can change them in the Statsig console without deploying code. For details, go to [Configs](https://docs.statsig.com/agent-experimentation/configs).

Call `getPrompt()` with a user and the config name. The SDK evaluates the config for that user and returns its Live and Candidate versions.

```ts
import { StatsigUser } from "@statsig/statsig-ai";

const user = new StatsigUser({ userID: "user-123" });
const prompt = statsigAI.getPrompt(user, "support_bot");

// The Live version, and any Candidate versions
const live = prompt.getLive();
const candidates = prompt.getCandidates();

// Read the version's settings, and render its messages
const model = live.getModel({ fallback: "gpt-5-mini" });
const messages = live.getPromptMessages({
  question: "How do I reset my password?",
});
```

A version (`PromptVersion`) has these methods:

| Method | Returns |
| --- | --- |
| `getName()`, `getID()` | The version name and ID. |
| `getModel()`, `getProvider()` | The model and provider strings. |
| `getTemperature()`, `getMaxTokens()`, `getTopP()`, `getFrequencyPenalty()`, `getPresencePenalty()` | Model parameters, as numbers. |
| `getValue(key, fallback)` | Any other value stored on the version. |
| `getPromptMessages(variables)` | The version's messages as `{ role, content }` objects, with each `{{variable}}` replaced. The SDK doesn't HTML-escape the values. |

The model, provider, and parameter getters accept an optional `{ fallback }`, which they return when the version doesn't set the value. Without a fallback, the parameter getters return `0`, so pass a fallback for any parameter you send to your model provider.

## Trace model calls

Tracing sends each model call to Statsig as an OpenTelemetry span. The spans appear in [Traces](https://docs.statsig.com/agent-experimentation/traces), where [online evaluators](https://docs.statsig.com/agent-experimentation/online-evaluation) can grade them.

### Initialize tracing

Call `initializeTracing()` once, when your app starts and before you wrap any model client or call `wrap()`. If you call it in its own module, such as `instrumentation.ts`, import that module before any module that wraps a client or calls `wrap()`:

```ts
// instrumentation.ts
import { initializeTracing } from "@statsig/statsig-ai";

initializeTracing({
  serviceName: "support-bot",
  environment: "production",
});
```

`initializeTracing()` throws `Statsig SDK Key is required for StatsigOTLPTraceExporter` unless you set the `STATSIG_SDK_KEY` environment variable or pass `exporterOptions.sdkKey`. Creating a `StatsigAI` instance doesn't set up tracing. Spans reach Statsig only after you call `initializeTracing()`.

| Option | Description |
| --- | --- |
| `exporterOptions.sdkKey` | Your Server Secret Key. Defaults to the `STATSIG_SDK_KEY` environment variable. |
| `exporterOptions.dsn` | A custom endpoint. The SDK appends `/v1/traces`. Defaults to `https://api.statsig.com/otlp`. |
| `serviceName` | The `service.name` resource attribute. Defaults to the `OTEL_SERVICE_NAME` or `SERVICE_NAME` environment variable. |
| `version` | The `service.version` resource attribute. |
| `environment` | The `deployment.environment.name` resource attribute. Defaults to `NODE_ENV`. Go to [Set the environment](#set-the-environment). |
| `enableGlobalTraceProviderRegistration` | When `true`, registers the SDK's tracer provider globally, so tracers you get from the OpenTelemetry API export to Statsig. |
| `globalTraceProvider` | A tracer provider for the SDK to use instead of creating one. |
| `globalContextManager` | A context manager for the SDK to use instead of creating one. |
| `skipGlobalContextManagerSetup` | When `true`, the SDK doesn't create a global context manager. |

`initializeTracing()` returns `{ exporter, processor, provider }`.

If your app already runs the OpenTelemetry `NodeSDK`, add the returned processor to it, so the SDK's spans and your existing spans both reach Statsig.

The AI SDK doesn't install `@opentelemetry/sdk-node`. Versions 0.300 and later of `@opentelemetry/sdk-node` use OpenTelemetry SDK 3.x, and the AI SDK uses 2.x. If your app doesn't list `@opentelemetry/sdk-node` as a dependency, install version 0.223:

#### npm

```shell
npm install @opentelemetry/sdk-node@0.223
```

#### pnpm

```shell
pnpm add @opentelemetry/sdk-node@0.223
```

#### yarn

```shell
yarn add @opentelemetry/sdk-node@0.223
```

Then add the processor to `NodeSDK`:

```ts
// instrumentation.ts
import { NodeSDK } from "@opentelemetry/sdk-node";
import { initializeTracing } from "@statsig/statsig-ai";

const { processor } = initializeTracing({
  serviceName: "support-bot",
  skipGlobalContextManagerSetup: true, // NodeSDK sets up its own
});

const sdk = new NodeSDK({
  spanProcessors: [processor],
  // ...your other NodeSDK options, such as instrumentations
});
sdk.start();
```

For more setups, go to the [SDK examples](https://github.com/statsig-io/statsig-ai-node/tree/main/examples/otel).

### Wrap your model client

Wrap each model client once, after you call `initializeTracing()`, and use the wrapped client everywhere you call the model. Each wrapper gets its tracer when you wrap the client. A client that you wrap before `initializeTracing()` doesn't send spans, and the SDK logs `NoopOtelSingleton instance is being used`.

```ts
import Anthropic from "@anthropic-ai/sdk";
import { GoogleGenAI } from "@google/genai";
import OpenAI from "openai";
import { wrapAnthropic, wrapGemini, wrapOpenAI } from "@statsig/statsig-ai";

const captureOptions = { capture_all: true };

const openai = wrapOpenAI(new OpenAI(), { captureOptions });
const anthropic = wrapAnthropic(new Anthropic(), { captureOptions });
const gemini = wrapGemini(new GoogleGenAI({}), { captureOptions });
```

Each wrapper traces these methods, including streaming calls:

| Wrapper | Client | Traced methods |
| --- | --- | --- |
| `wrapOpenAI` | `openai` | `chat.completions.create`, `completions.create`, `responses.create`, `responses.stream`, `responses.parse`, `embeddings.create`, `images.generate` |
| `wrapAnthropic` | `@anthropic-ai/sdk` | `messages.create`, `messages.stream` |
| `wrapGemini` | `@google/genai` | `models.generateContent`, `models.generateContentStream` |

Each wrapper accepts the same optional configuration:

| Option | Description |
| --- | --- |
| `captureOptions` | Which message content to record. All options are off by default. |
| `customAttributes` | Attributes to add to every span from this client, such as `{ team: 'support' }`. They can't override the attributes the SDK sets. |
| `maxJSONChars` | The longest JSON value to record per attribute, in characters. Defaults to `40000`. The SDK cuts off longer values and ends them with `…(truncated)`. |

### Record message content

By default, the wrappers record the model, token counts, and timing, but no message content. Turn on `captureOptions` to view messages in the trace drawer. [Online evaluators](https://docs.statsig.com/agent-experimentation/online-evaluation) grade only spans that record both input and output messages.

| Option | Records |
| --- | --- |
| `capture_all` | Everything in this table. |
| `capture_input_messages` | `gen_ai.input.messages`, plus the variables passed to `getPromptMessages()`. |
| `capture_output_messages` | `gen_ai.output.messages`. |
| `capture_system_instructions` | `gen_ai.system_instructions`: system and developer messages, and the OpenAI `instructions` parameter. |
| `capture_tool_definitions` | `gen_ai.tool.definitions`: the tools you pass to the model. |

The SDK records messages in the [OpenTelemetry GenAI format](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-spans/), where each message has a `role` and a list of `parts`. For every attribute the spans carry, go to the [span attribute reference](https://docs.statsig.com/agent-experimentation/traces#span-attribute-reference).

## Tag model calls with a config version

Run each model call inside `prompt.withLive()` or `prompt.withVersion()`, so its span records which config version produced it. Online evaluators attribute each grade to the version on the span. A span without these attributes isn't linked to a config: it doesn't appear on the config's Traces tab, and its grades don't count toward any version.

```ts
import type { PromptVersion } from "@statsig/statsig-ai";

async function answer(version: PromptVersion, question: string) {
  const response = await openai.chat.completions.create({
    model: version.getModel({ fallback: "gpt-5-mini" }),
    messages: version.getPromptMessages({
      question,
    }) as OpenAI.ChatCompletionMessageParam[],
  });
  return response.choices[0].message.content;
}

const prompt = statsigAI.getPrompt(user, "support_bot");

// Serve the Live version
const reply = await prompt.withLive((version) => answer(version, question));

// Serve a Candidate version
const [candidate] = prompt.getCandidates();
if (candidate) {
  await prompt.withVersion(candidate, (version) => answer(version, question));
}
```

`withLive()` and `withVersion()` return a promise that resolves to the function's return value. Wrapped model calls and `wrap()` spans inside the function get these attributes:

| Attribute | Value |
| --- | --- |
| `statsig.llm.prompt_name` | The config name. |
| `statsig.llm.prompt_version` | The version name. |
| `statsig.llm.prompt_version_id` | The version ID. |
| `statsig.llm.prompt_template_variables` | The variables passed to `getPromptMessages()`, as JSON. Model call spans record them only when you capture input messages. Workflow spans always record them. |

The spans also record the user you passed to `getPrompt()`.

## Trace workflows and tools

`wrap()` traces your own functions, such as an agent turn or a tool call, as spans in the same trace as the model calls they make. Pass a `type` and a `name`, and `wrap()` returns a function with the same signature. Call `wrap()` after `initializeTracing()`. A function that you wrap earlier doesn't send spans.

```ts
import { wrap } from "@statsig/statsig-ai";

const searchDocs = wrap(
  { type: "tool", name: "search_docs" },
  async (query: string) => {
    return docsIndex.search(query);
  },
);

const handleTurn = wrap(
  { type: "workflow", name: "support_turn" },
  async (version: PromptVersion, question: string) => {
    const docs = await searchDocs(question);
    const response = await openai.chat.completions.create({
      model: version.getModel({ fallback: "gpt-5-mini" }),
      messages: version.getPromptMessages({
        question,
        docs: docs.join("\n"),
      }) as OpenAI.ChatCompletionMessageParam[],
    });
    return response.choices[0].message.content;
  },
);

await prompt.withLive((version) => handleTurn(version, question));
```

| Option | Description |
| --- | --- |
| `type` | `workflow` for a unit of work such as an agent turn, or `tool` for a tool call. |
| `name` | The workflow or tool name, recorded as `statsig.gen_ai.workflow.name` or `statsig.gen_ai.tool.name`. |
| `toolType` | Tool spans only. Recorded as `gen_ai.tool.type`. |
| `attributes` | Extra span attributes. |
| `user`, `activityID` | Tag this span and the spans inside it, like [`withStatsigContext()`](#add-users-and-activity-ids-to-spans). |

The SDK names workflow spans `invoke_workflow` and tool spans `execute_tool`. Online evaluators grade a workflow span as the turn's answer, and never grade tool spans. The SDK doesn't mark the model calls inside a workflow span with `statsig.span.llm_root`, so online evaluators don't grade them on their own. `wrap()` doesn't record messages on workflow spans, and online evaluators skip a workflow span without them. To grade a workflow span, set its messages yourself, as [Grade a multi-step agent turn](https://docs.statsig.com/agent-experimentation/online-evaluation#grade-a-multi-step-agent-turn) shows.

## Add users and activity IDs to spans

Context helpers add a user, and optionally an activity ID, to the spans that start inside a function:

- `withStatsigUserContext(user, fn)`: Records the user's ID as `statsig.user_id` and the user's custom IDs as `statsig.custom_ids`, on every span that starts inside `fn`.
- `withStatsigContext({ user, activityID }, fn)`: Records the user, and adds `activityID` to the custom IDs as `statsig.activity_id`, on wrapped model calls and `wrap()` spans inside `fn`.

Use an activity ID to group the spans of a multi-turn conversation or a long-running job. The _Traces_ page groups spans with the same activity ID in the trace drawer, even across traces.

```ts
import { withStatsigContext } from "@statsig/statsig-ai";

await withStatsigContext({ user, activityID: conversationID }, () =>
  prompt.withLive((version) => handleTurn(version, question)),
);
```

## Set the environment

Statsig reads each span's environment to filter the _Traces_ page and to choose which spans online evaluators grade. Set the environment for all spans with the `environment` option of `initializeTracing()`. The option defaults to `NODE_ENV`, and the SDK records it as the `deployment.environment.name` resource attribute.

To tag only some spans, run the code that creates them inside `withStatsigEnvironment()`. Spans that start inside the function get a `statsig.environment` attribute, which overrides the resource value for those spans.

```ts
import { initializeTracing, withStatsigEnvironment } from "@statsig/statsig-ai";

initializeTracing({ environment: "production" });

// Tag canary traffic that runs in the production process
await withStatsigEnvironment("staging_canary", () => handleRequest(request));
```

Use `production` for live traffic. Statsig reserves `eval`, `online_eval_judge`, and `shadow` for spans it generates, and online evaluators never grade them. `Eval()` runs its tasks and scorers with `statsig.environment` set to `eval`. Don't use the reserved values for your own spans, except `eval` for the requests that a workflow run sends. For details, go to [Join a workflow run's trace](#join-a-workflow-eval-trace). For how Statsig reads environments, go to [Environments](https://docs.statsig.com/agent-experimentation/traces#environments).

## Join a workflow run's trace

In an offline run of a config version with a [custom workflow](https://docs.statsig.com/agent-experimentation/configs#call-your-own-endpoint-with-a-custom-workflow), Statsig sends each dataset row to your endpoint as a POST request. Each request carries these headers:

- `traceparent`: The W3C trace context of the row's trace on the _Traces_ page.
- `x-statsig-eval-run`: The ID of the run.

Run your handler inside `withIncomingTraceContext()`, so the wrapped model calls and `wrap()` spans it starts join the row's trace. The trace drawer then shows them under the row's root span.

```ts
import { withIncomingTraceContext } from "@statsig/statsig-ai";

// Node and Express
app.post("/agent", (req, res) =>
  withIncomingTraceContext(req.headers, async () => {
    res.json({ output: await runAgent(req.body) });
  }),
);

// Fetch-style handlers, such as Next.js route handlers
export async function POST(request: Request) {
  return withIncomingTraceContext(request.headers, async () =>
    Response.json({ output: await runAgent(await request.json()) }),
  );
}
```

`withIncomingTraceContext(headers, fn)` reads the `traceparent` and `tracestate` headers from Node request headers or a Fetch `Headers` object. It runs `fn` with the caller's span as the parent, and returns what `fn` returns. Without a valid `traceparent`, `fn` runs in the current context, so you can wrap every request. If OpenTelemetry HTTP instrumentation already started a span in the incoming trace, `withIncomingTraceContext()` keeps that span as the parent.

Spans from your endpoint use your app's environment, such as `production`. Online evaluators skip the spans that join a dataset row's trace in the config's project, so a run doesn't grade them a second time. The _Traces_ page still lists them with your production traces. To mark requests from a run as `eval` traffic everywhere, including spans that don't join the row's trace, run them inside `withStatsigEnvironment("eval", ...)`:

```ts
import {
  withIncomingTraceContext,
  withStatsigEnvironment,
} from "@statsig/statsig-ai";

app.post("/agent", (req, res) =>
  withIncomingTraceContext(req.headers, () => {
    const handle = async () => {
      res.json({ output: await runAgent(req.body) });
    };
    return req.headers["x-statsig-eval-run"]
      ? withStatsigEnvironment("eval", handle)
      : handle();
  }),
);
```

Only check `x-statsig-eval-run` on requests that your endpoint authenticates, such as with the auth header that you set on the workflow. Otherwise any caller can add the header to keep their requests out of online evaluation and the default _Traces_ view. Without the Node SDK, set the span attribute `statsig.environment` to `eval` on the spans that your handler starts.

Statsig sends these headers only during a run, not when you click **Test Connection**. A header that you set on the workflow with the same name replaces Statsig's.

## Log eval grades

Use `logEvalGrade()` to send a grade that your own code computes, such as user feedback or a check in your app, to Statsig. Statsig shows these grades with the source SDK.

```ts
const version = statsigAI.getPrompt(user, "support_bot").getLive();

statsigAI.logEvalGrade(user, version, 1, "user_feedback", {
  sessionId: conversationID,
  reasoning: "The user rated the reply as helpful.",
});
```

`logEvalGrade(user, version, score, graderName, options)` takes the config version that produced the output, and a score from 0 to 1. The SDK drops a score outside that range and prints a warning. Always pass the `options` argument. Pass `{}` when you have no options to set.

| Option | Description |
| --- | --- |
| `sessionId` | Groups grades from the same session. |
| `activityId` | Sets `statsig.activity_id` on the user's custom IDs. |
| `traceId`, `spanId` | The span the grade belongs to. When you set neither, the SDK uses the active span, so a grade logged inside a `wrap()` workflow belongs to that workflow span. To grade a span after it ends, such as from feedback that arrives later, save its IDs while it's active (`trace.getActiveSpan()?.spanContext()` from `@opentelemetry/api`), then pass both. |
| `reasoning` | Why the evaluator gave this score. |
| `metadata` | Extra string fields to log with the grade. |

The SDK logs each grade as a `statsig::eval_result` event. For how grades appear in Statsig, go to [Online Evaluation](https://docs.statsig.com/agent-experimentation/online-evaluation).

## Flush events and spans

The SDK sends events and spans in batches. Before a short-lived process exits, such as a script or a serverless function, flush them:

```ts
await statsigAI.flushEvents();
```

`flushEvents()` always flushes spans. It flushes Statsig events, including eval grades, only when the AI SDK created the Statsig instance. If you passed in your own instance, flush that instance too:

```ts
await statsig.flushEvents();
```

To flush and stop the SDK when your server shuts down, call `await statsigAI.shutdown()`.

## Run an eval from code

`Eval()` runs a task over records you define in code, scores each output, and sends the results to a run group in Statsig. Before you call `Eval()`:

- Set the `STATSIG_API_KEY` environment variable to a [Console API key](https://docs.statsig.com/access-management/api-keys#console-api-keys). `Eval()` throws when the variable isn't set.
- Create a run group in the Statsig console, under _Agent Experimentation > Runs_. Pass its name as the first argument to `Eval()`. If no run group has that name, Statsig rejects the results, and the SDK prints `[Statsig] Failed to send eval results: 404 Not Found`.

```ts
import { Eval } from "@statsig/statsig-ai";

const result = await Eval("greeting_eval", {
  data: [
    { input: "world", expected: "Hello world" },
    { input: "test", expected: "Hello test" },
  ],
  task: (input: string) => `Hello ${input}`,
  scorer: ({ output, expected }) => output === expected,
  evalRunName: "run-123", // optional
});
```

`Eval()` takes these options:

| Option | Description |
| --- | --- |
| `data` | Records with an `input`, an optional `expected` output, and an optional `category` (a string or an array of strings). Pass an array, a promise, a function that returns either, an async iterable, or `statsigDataset()`. |
| `dataset` | The name or ID of the [Statsig dataset](#run-an-eval-on-a-statsig-dataset) that the records come from. Each record then needs the `rowID` of a different row of the dataset. `statsigDataset()` sets this for you. |
| `task` | A function, sync or async, that returns the output for an input. It receives the input and `{ parameters, category }`. |
| `scorer` | A scorer function, or an object of named scorer functions. The SDK names a single scorer function `Grader`. A scorer returns a boolean, a number, or `{ score, metadata }`. `true` counts as 1 and `false` as 0. |
| `parameters` | Zod schemas whose default values the task receives as `parameters`. |
| `evalRunName` | A name for this run. |
| `summaryScoresFn` | A function that receives every result and returns summary scores, as an object of numbers. |

`Eval()` returns `{ results, metadata, summaryScores }`. Each result has the record's `input`, `expected`, and `category`, plus the `output`, a `scores` object, and an `error` flag. With tracing on, each result also has the record's [`traceID` and `spanID`](#traces-for-eval-records).

Statsig stores each result's input, output, expected value, and score metadata as strings. `Eval()` sends other values as JSON. The results that `Eval()` returns keep the original values.

Errors don't stop the run:

- When a task throws, the SDK prints a warning, scores every scorer 0 for that record, and sets the record's `error` to `true`. `metadata.error` is `true` when any record has an error.
- When a scorer throws, the SDK prints a warning and scores that scorer 0 for that record.

The following example uses named scorers, a parameter, categories, and a summary score:

```ts
import { Eval } from "@statsig/statsig-ai";
import { z } from "zod";

const result = await Eval("greeting_eval", {
  data: [
    { input: "world", expected: "Hi world", category: "greeting" },
    { input: "test", expected: "Hi test", category: ["greeting", "test"] },
  ],
  parameters: {
    greeting: z.string().default("Hi"),
  },
  task: (input: string, { parameters }) => `${parameters.greeting} ${input}`,
  scorer: {
    correctness: ({ output, expected }) => output === expected,
    startsWithGreeting: ({ output }) => ({
      score: output.startsWith("Hi"),
      metadata: { length: output.length },
    }),
  },
  summaryScoresFn: (results) => ({
    accuracy:
      results.filter((r) => r.scores.correctness === 1).length / results.length,
  }),
});
```

### Run an eval on a Statsig dataset

To run on a [dataset](https://docs.statsig.com/agent-experimentation/datasets) from the Statsig console, pass `statsigDataset(name, toRecord)` as `data`. `Eval()` loads every row of the dataset with the key in `STATSIG_API_KEY`, and `toRecord` turns each row into a record. The run records the dataset, so the run group's _Compare_ tab compares runs on the same dataset row by row.

```ts
import { Eval, statsigDataset } from "@statsig/statsig-ai";

await Eval("support_bot_quality", {
  data: statsigDataset("support_cases", (row) => ({
    input: row.columns.question,
    expected: row.columns.answer,
    category: row.groups,
  })),
  task: (input) => answer(input),
  scorer: ({ output, expected }) => output === expected,
});
```

`statsigDataset()` takes the dataset's name or ID. `toRecord` receives each row as `{ rowID, columns, groups }`:

- `columns` maps each column name to the row's value, as a string.
- `groups` lists the row's categories.

Each record keeps its row's `rowID`. When the dataset can't load, `Eval()` throws `[Statsig] Failed to load eval dataset` with the HTTP status, before any task runs.

To run on records that you build yourself from a dataset, set the `dataset` option, and give each record the `rowID` of its row. `Eval()` throws before any task runs when a record has no `rowID` or repeats one. Statsig rejects the results when a `rowID` isn't a row of the dataset.

### Traces for eval records

When you call `initializeTracing()` before `Eval()`, each record runs in its own trace, even when you call `Eval()` inside another span. The root of the trace is a workflow span with the run group name as its workflow name, and these attributes:

| Attribute | Value |
| --- | --- |
| `statsig.eval.name` | The run group name. |
| `statsig.eval.run_name` | The `evalRunName`, when you set one. |
| `statsig.eval.record_index` | The record's position in the data, counting from 0. |

Wrapped model calls and `wrap()` spans in the task become children of the root span. Each scorer runs in its own child workflow span, named after the scorer and tagged with `statsig.eval.scorer_name`. Every span in the trace has `statsig.environment` set to `eval`, so the _Traces_ page hides these spans by default, and online evaluators don't grade them.

`Eval()` sends each record's `traceID` and `spanID` with its scores, so the scores also appear on the root span in [Traces](https://docs.statsig.com/agent-experimentation/traces). `Eval()` flushes the spans before it returns. Without `initializeTracing()`, `Eval()` doesn't create traces.

## Use other Statsig SDK methods

`getStatsig()` returns the underlying Statsig instance, however you initialized the AI SDK. Use it for any [Node Server SDK](https://docs.statsig.com/server-core/node-core) method:

```ts
const statsig = statsigAI.getStatsig();

// Check a gate
const enabled = statsig.checkGate(user, "my_gate");

// Log an event
statsig.logEvent(user, "my_event", 1);
```

## Upgrade to version 0.0.1-beta.34

If you upgrade from version `0.0.1-beta.33`, check these changes:

- **No alias package**: The SDK no longer needs `@statsig/statsig-node-core-rc`. You can remove the alias from your dependencies.
- **Fewer OpenTelemetry dependencies**: The SDK no longer installs `@opentelemetry/sdk-node`, `@opentelemetry/auto-instrumentations-node`, `@opentelemetry/sdk-metrics`, or `@opentelemetry/sdk-trace-node`. With npm, an app could import them before without listing them as dependencies. If your app imports any of them, add it to your dependencies. For `NodeSDK`, go to [Initialize tracing](#initialize-tracing).
- **Model calls inside workflows**: The SDK no longer marks model calls inside a `wrap()` workflow span with `statsig.span.llm_root`, so online evaluators grade only the workflow span. If your workflow spans don't record input and output messages, online evaluators grade nothing in those turns. To set the messages, go to [Grade a multi-step agent turn](https://docs.statsig.com/agent-experimentation/online-evaluation#grade-a-multi-step-agent-turn).
- **Eval traces**: With `initializeTracing()`, `Eval()` creates a trace for each record and flushes spans before it returns. For details, go to [Traces for eval records](#traces-for-eval-records).
- **Eval result values**: `Eval()` sends non-string inputs, outputs, expected values, and score metadata as JSON. Before, Statsig rejected the whole run with a 400 error when any of them wasn't a string.

The release also adds [`statsigDataset()`](#run-an-eval-on-a-statsig-dataset), the `dataset` option of `Eval()`, and [`withIncomingTraceContext()`](#join-a-workflow-eval-trace). For every change, go to the [release notes](https://github.com/statsig-io/statsig-ai-node/releases).

## Upgrade to version 0.0.1-beta.33

Version `0.0.1-beta.33` is the first release on npm after `0.0.1-beta.30`. If you upgrade from an earlier version, check these changes:

- **Message format**: Wrappers record `gen_ai.input.messages` and `gen_ai.output.messages` in the OpenTelemetry GenAI format (`role` plus `parts`), not as OpenAI-shaped messages. Update any code that parses these attributes.
- **System instructions**: System and developer messages, and the OpenAI `instructions` parameter, move out of `gen_ai.input.messages` into `gen_ai.system_instructions`. Turn on `capture_system_instructions` or `capture_all` to record them.
- **Duplicate OpenAI requests**: In versions `0.0.1-beta.11` through `0.0.1-beta.30`, `wrapOpenAI` sent every request to OpenAI twice. Version `0.0.1-beta.33` sends each request once.

The release also adds `wrapAnthropic`, `wrapGemini`, `withVersion()`, `withStatsigEnvironment()`, and the `traceId`, `spanId`, and `reasoning` options of `logEvalGrade()`. For every change, go to the [release notes](https://github.com/statsig-io/statsig-ai-node/releases).