Node AI SDK

The Statsig Node AI SDK (@statsig/statsig-ai) is the Node.js server SDK for Agent Experimentation. It serves configs, so you can change prompts and models without deploying code. These are Agent Experimentation configs, not Dynamic Configs. It also traces OpenAI, Anthropic, and Gemini calls to Traces as OpenTelemetry spans, logs eval grades that your code computes, and runs evals from code. The SDK builds on the Statsig Node Server SDK.

The SDK uses prompt, the earlier name for a config, in its method names. For example, getPrompt() returns a config. The console shows Eval() scorers, and the graderName you pass to logEvalGrade(), as evaluators.

This page describes @statsig/statsig-ai version 0.0.1-beta.34.

Set up the SDK

1. Install the SDK

   Install the AI SDK and the Node Server SDK:

   npmpnpmyarn

   Copy

   npm install @statsig/statsig-ai @statsig/statsig-node-core
   

   Versions 0.0.1-beta.30 through 0.0.1-beta.33 load @statsig/statsig-node-core-rc at startup, but no package with that name exists on npm. To install one of those versions, add @statsig/statsig-node-core-rc@npm:@statsig/statsig-node-core to the install command. This alias installs the Node Server SDK under that name. Without it, the install fails or the import throws Cannot find module '@statsig/statsig-node-core-rc'.

   The Node Server SDK includes native binaries. If you use a frozen lockfile or bundle your server with webpack, esbuild, or Next.js, follow the Node Server SDK setup steps.

2. Initialize the SDK

   Initialize the AI SDK with a Server Secret Key. The examples on this page read the key from the STATSIG_SDK_KEY environment variable, which initializeTracing() also reads.

   Keep Server Secret Keys private. If you expose one, disable it and create a new key in the Statsig console.

   Choose one of these setups:

   Create a Statsig instanceUse an existing Statsig instance

   The AI SDK creates and owns its own Statsig instance. statsigOptions accepts any Statsig option.

   ts

   Copy

   import { StatsigAI } from "@statsig/statsig-ai";
   
   const statsigAI = StatsigAI.newShared({
     sdkKey: process.env.STATSIG_SDK_KEY!,
     statsigOptions: { environment: "production" }, // optional
   });
   await statsigAI.initialize();
   

   StatsigAI.newShared() creates the instance and registers it as the shared instance. Get the shared instance anywhere in your app with StatsigAI.shared(). The tracing wrappers log each traced call as a statsig::gen_ai event through the shared instance. When no shared instance exists, they print a warning on every traced call. To create an instance that isn't shared, pass the same options to new StatsigAI().

Get a config

A config holds versions of a prompt and its model settings, so you can change them in the Statsig console without deploying code. For details, go to Configs.

Call getPrompt() with a user and the config name. The SDK evaluates the config for that user and returns its Live and Candidate versions.

import { StatsigUser } from "@statsig/statsig-ai";

const user = new StatsigUser({ userID: "user-123" });
const prompt = statsigAI.getPrompt(user, "support_bot");

// The Live version, and any Candidate versions
const live = prompt.getLive();
const candidates = prompt.getCandidates();

// Read the version's settings, and render its messages
const model = live.getModel({ fallback: "gpt-5-mini" });
const messages = live.getPromptMessages({
  question: "How do I reset my password?",
});

A version (PromptVersion) has these methods:

The model, provider, and parameter getters accept an optional { fallback }, which they return when the version doesn't set the value. Without a fallback, the parameter getters return 0, so pass a fallback for any parameter you send to your model provider.

Trace model calls

Tracing sends each model call to Statsig as an OpenTelemetry span. The spans appear in Traces, where online evaluators can grade them.

Initialize tracing

Call initializeTracing() once, when your app starts and before you wrap any model client or call wrap(). If you call it in its own module, such as instrumentation.ts, import that module before any module that wraps a client or calls wrap():

// instrumentation.ts
import { initializeTracing } from "@statsig/statsig-ai";

initializeTracing({
  serviceName: "support-bot",
  environment: "production",
});

initializeTracing() throws Statsig SDK Key is required for StatsigOTLPTraceExporter unless you set the STATSIG_SDK_KEY environment variable or pass exporterOptions.sdkKey. Creating a StatsigAI instance doesn't set up tracing. Spans reach Statsig only after you call initializeTracing().

initializeTracing() returns { exporter, processor, provider }.

If your app already runs the OpenTelemetry NodeSDK, add the returned processor to it, so the SDK's spans and your existing spans both reach Statsig.

The AI SDK doesn't install @opentelemetry/sdk-node. Versions 0.300 and later of @opentelemetry/sdk-node use OpenTelemetry SDK 3.x, and the AI SDK uses 2.x. If your app doesn't list @opentelemetry/sdk-node as a dependency, install version 0.223:

npm install @opentelemetry/sdk-node@0.223

Then add the processor to NodeSDK:

// instrumentation.ts
import { NodeSDK } from "@opentelemetry/sdk-node";
import { initializeTracing } from "@statsig/statsig-ai";

const { processor } = initializeTracing({
  serviceName: "support-bot",
  skipGlobalContextManagerSetup: true, // NodeSDK sets up its own
});

const sdk = new NodeSDK({
  spanProcessors: [processor],
  // ...your other NodeSDK options, such as instrumentations
});
sdk.start();

For more setups, go to the SDK examples.

Wrap your model client

Wrap each model client once, after you call initializeTracing(), and use the wrapped client everywhere you call the model. Each wrapper gets its tracer when you wrap the client. A client that you wrap before initializeTracing() doesn't send spans, and the SDK logs NoopOtelSingleton instance is being used.

import Anthropic from "@anthropic-ai/sdk";
import { GoogleGenAI } from "@google/genai";
import OpenAI from "openai";
import { wrapAnthropic, wrapGemini, wrapOpenAI } from "@statsig/statsig-ai";

const captureOptions = { capture_all: true };

const openai = wrapOpenAI(new OpenAI(), { captureOptions });
const anthropic = wrapAnthropic(new Anthropic(), { captureOptions });
const gemini = wrapGemini(new GoogleGenAI({}), { captureOptions });

Each wrapper traces these methods, including streaming calls:

Each wrapper accepts the same optional configuration:

Record message content

By default, the wrappers record the model, token counts, and timing, but no message content. Turn on captureOptions to view messages in the trace drawer. Online evaluators grade only spans that record both input and output messages.

The SDK records messages in the OpenTelemetry GenAI format, where each message has a role and a list of parts. For every attribute the spans carry, go to the span attribute reference.

Tag model calls with a config version

Run each model call inside prompt.withLive() or prompt.withVersion(), so its span records which config version produced it. Online evaluators attribute each grade to the version on the span. A span without these attributes isn't linked to a config: it doesn't appear on the config's Traces tab, and its grades don't count toward any version.

import type { PromptVersion } from "@statsig/statsig-ai";

async function answer(version: PromptVersion, question: string) {
  const response = await openai.chat.completions.create({
    model: version.getModel({ fallback: "gpt-5-mini" }),
    messages: version.getPromptMessages({
      question,
    }) as OpenAI.ChatCompletionMessageParam[],
  });
  return response.choices[0].message.content;
}

const prompt = statsigAI.getPrompt(user, "support_bot");

// Serve the Live version
const reply = await prompt.withLive((version) => answer(version, question));

// Serve a Candidate version
const [candidate] = prompt.getCandidates();
if (candidate) {
  await prompt.withVersion(candidate, (version) => answer(version, question));
}

withLive() and withVersion() return a promise that resolves to the function's return value. Wrapped model calls and wrap() spans inside the function get these attributes:

The spans also record the user you passed to getPrompt().

Trace workflows and tools

wrap() traces your own functions, such as an agent turn or a tool call, as spans in the same trace as the model calls they make. Pass a type and a name, and wrap() returns a function with the same signature. Call wrap() after initializeTracing(). A function that you wrap earlier doesn't send spans.

import { wrap } from "@statsig/statsig-ai";

const searchDocs = wrap(
  { type: "tool", name: "search_docs" },
  async (query: string) => {
    return docsIndex.search(query);
  },
);

const handleTurn = wrap(
  { type: "workflow", name: "support_turn" },
  async (version: PromptVersion, question: string) => {
    const docs = await searchDocs(question);
    const response = await openai.chat.completions.create({
      model: version.getModel({ fallback: "gpt-5-mini" }),
      messages: version.getPromptMessages({
        question,
        docs: docs.join("\n"),
      }) as OpenAI.ChatCompletionMessageParam[],
    });
    return response.choices[0].message.content;
  },
);

await prompt.withLive((version) => handleTurn(version, question));

The SDK names workflow spans invoke_workflow and tool spans execute_tool. Online evaluators grade a workflow span as the turn's answer, and never grade tool spans. The SDK doesn't mark the model calls inside a workflow span with statsig.span.llm_root, so online evaluators don't grade them on their own. wrap() doesn't record messages on workflow spans, and online evaluators skip a workflow span without them. To grade a workflow span, set its messages yourself, as Grade a multi-step agent turn shows.

Add users and activity IDs to spans

Context helpers add a user, and optionally an activity ID, to the spans that start inside a function:

• withStatsigUserContext(user, fn): Records the user's ID as statsig.user_id and the user's custom IDs as statsig.custom_ids, on every span that starts inside fn.
• withStatsigContext({ user, activityID }, fn): Records the user, and adds activityID to the custom IDs as statsig.activity_id, on wrapped model calls and wrap() spans inside fn.

Use an activity ID to group the spans of a multi-turn conversation or a long-running job. The Traces page groups spans with the same activity ID in the trace drawer, even across traces.

import { withStatsigContext } from "@statsig/statsig-ai";

await withStatsigContext({ user, activityID: conversationID }, () =>
  prompt.withLive((version) => handleTurn(version, question)),
);

Set the environment

Statsig reads each span's environment to filter the Traces page and to choose which spans online evaluators grade. Set the environment for all spans with the environment option of initializeTracing(). The option defaults to NODE_ENV, and the SDK records it as the deployment.environment.name resource attribute.

To tag only some spans, run the code that creates them inside withStatsigEnvironment(). Spans that start inside the function get a statsig.environment attribute, which overrides the resource value for those spans.

import { initializeTracing, withStatsigEnvironment } from "@statsig/statsig-ai";

initializeTracing({ environment: "production" });

// Tag canary traffic that runs in the production process
await withStatsigEnvironment("staging_canary", () => handleRequest(request));

Use production for live traffic. Statsig reserves eval, online_eval_judge, and shadow for spans it generates, and online evaluators never grade them. Eval() runs its tasks and scorers with statsig.environment set to eval. Don't use the reserved values for your own spans, except eval for the requests that a workflow run sends. For details, go to Join a workflow run's trace. For how Statsig reads environments, go to Environments.

Join a workflow run's trace

In an offline run of a config version with a custom workflow, Statsig sends each dataset row to your endpoint as a POST request. Each request carries these headers:

• traceparent: The W3C trace context of the row's trace on the Traces page.
• x-statsig-eval-run: The ID of the run.

Run your handler inside withIncomingTraceContext(), so the wrapped model calls and wrap() spans it starts join the row's trace. The trace drawer then shows them under the row's root span.

import { withIncomingTraceContext } from "@statsig/statsig-ai";

// Node and Express
app.post("/agent", (req, res) =>
  withIncomingTraceContext(req.headers, async () => {
    res.json({ output: await runAgent(req.body) });
  }),
);

// Fetch-style handlers, such as Next.js route handlers
export async function POST(request: Request) {
  return withIncomingTraceContext(request.headers, async () =>
    Response.json({ output: await runAgent(await request.json()) }),
  );
}

withIncomingTraceContext(headers, fn) reads the traceparent and tracestate headers from Node request headers or a Fetch Headers object. It runs fn with the caller's span as the parent, and returns what fn returns. Without a valid traceparent, fn runs in the current context, so you can wrap every request. If OpenTelemetry HTTP instrumentation already started a span in the incoming trace, withIncomingTraceContext() keeps that span as the parent.

Spans from your endpoint use your app's environment, such as production. Online evaluators skip the spans that join a dataset row's trace in the config's project, so a run doesn't grade them a second time. The Traces page still lists them with your production traces. To mark requests from a run as eval traffic everywhere, including spans that don't join the row's trace, run them inside withStatsigEnvironment("eval", ...):

import {
  withIncomingTraceContext,
  withStatsigEnvironment,
} from "@statsig/statsig-ai";

app.post("/agent", (req, res) =>
  withIncomingTraceContext(req.headers, () => {
    const handle = async () => {
      res.json({ output: await runAgent(req.body) });
    };
    return req.headers["x-statsig-eval-run"]
      ? withStatsigEnvironment("eval", handle)
      : handle();
  }),
);

Only check x-statsig-eval-run on requests that your endpoint authenticates, such as with the auth header that you set on the workflow. Otherwise any caller can add the header to keep their requests out of online evaluation and the default Traces view. Without the Node SDK, set the span attribute statsig.environment to eval on the spans that your handler starts.

Statsig sends these headers only during a run, not when you click Test Connection. A header that you set on the workflow with the same name replaces Statsig's.

Log eval grades

Use logEvalGrade() to send a grade that your own code computes, such as user feedback or a check in your app, to Statsig. Statsig shows these grades with the source SDK.

const version = statsigAI.getPrompt(user, "support_bot").getLive();

statsigAI.logEvalGrade(user, version, 1, "user_feedback", {
  sessionId: conversationID,
  reasoning: "The user rated the reply as helpful.",
});

logEvalGrade(user, version, score, graderName, options) takes the config version that produced the output, and a score from 0 to 1. The SDK drops a score outside that range and prints a warning. Always pass the options argument. Pass {} when you have no options to set.

The SDK logs each grade as a statsig::eval_result event. For how grades appear in Statsig, go to Online Evaluation.

Flush events and spans

The SDK sends events and spans in batches. Before a short-lived process exits, such as a script or a serverless function, flush them:

await statsigAI.flushEvents();

flushEvents() always flushes spans. It flushes Statsig events, including eval grades, only when the AI SDK created the Statsig instance. If you passed in your own instance, flush that instance too:

await statsig.flushEvents();

To flush and stop the SDK when your server shuts down, call await statsigAI.shutdown().

Run an eval from code

Eval() runs a task over records you define in code, scores each output, and sends the results to a run group in Statsig. Before you call Eval():

• Set the STATSIG_API_KEY environment variable to a Console API key. Eval() throws when the variable isn't set.
• Create a run group in the Statsig console, under Agent Experimentation > Runs. Pass its name as the first argument to Eval(). If no run group has that name, Statsig rejects the results, and the SDK prints [Statsig] Failed to send eval results: 404 Not Found.

import { Eval } from "@statsig/statsig-ai";

const result = await Eval("greeting_eval", {
  data: [
    { input: "world", expected: "Hello world" },
    { input: "test", expected: "Hello test" },
  ],
  task: (input: string) => `Hello ${input}`,
  scorer: ({ output, expected }) => output === expected,
  evalRunName: "run-123", // optional
});

Eval() takes these options:

Eval() returns { results, metadata, summaryScores }. Each result has the record's input, expected, and category, plus the output, a scores object, and an error flag. With tracing on, each result also has the record's traceID and spanID.

Statsig stores each result's input, output, expected value, and score metadata as strings. Eval() sends other values as JSON. The results that Eval() returns keep the original values.

Errors don't stop the run:

• When a task throws, the SDK prints a warning, scores every scorer 0 for that record, and sets the record's error to true. metadata.error is true when any record has an error.
• When a scorer throws, the SDK prints a warning and scores that scorer 0 for that record.

The following example uses named scorers, a parameter, categories, and a summary score:

import { Eval } from "@statsig/statsig-ai";
import { z } from "zod";

const result = await Eval("greeting_eval", {
  data: [
    { input: "world", expected: "Hi world", category: "greeting" },
    { input: "test", expected: "Hi test", category: ["greeting", "test"] },
  ],
  parameters: {
    greeting: z.string().default("Hi"),
  },
  task: (input: string, { parameters }) => `${parameters.greeting} ${input}`,
  scorer: {
    correctness: ({ output, expected }) => output === expected,
    startsWithGreeting: ({ output }) => ({
      score: output.startsWith("Hi"),
      metadata: { length: output.length },
    }),
  },
  summaryScoresFn: (results) => ({
    accuracy:
      results.filter((r) => r.scores.correctness === 1).length / results.length,
  }),
});

Run an eval on a Statsig dataset

To run on a dataset from the Statsig console, pass statsigDataset(name, toRecord) as data. Eval() loads every row of the dataset with the key in STATSIG_API_KEY, and toRecord turns each row into a record. The run records the dataset, so the run group's Compare tab compares runs on the same dataset row by row.

import { Eval, statsigDataset } from "@statsig/statsig-ai";

await Eval("support_bot_quality", {
  data: statsigDataset("support_cases", (row) => ({
    input: row.columns.question,
    expected: row.columns.answer,
    category: row.groups,
  })),
  task: (input) => answer(input),
  scorer: ({ output, expected }) => output === expected,
});

statsigDataset() takes the dataset's name or ID. toRecord receives each row as { rowID, columns, groups }:

• columns maps each column name to the row's value, as a string.
• groups lists the row's categories.

Each record keeps its row's rowID. When the dataset can't load, Eval() throws [Statsig] Failed to load eval dataset with the HTTP status, before any task runs.

To run on records that you build yourself from a dataset, set the dataset option, and give each record the rowID of its row. Eval() throws before any task runs when a record has no rowID or repeats one. Statsig rejects the results when a rowID isn't a row of the dataset.

Traces for eval records

When you call initializeTracing() before Eval(), each record runs in its own trace, even when you call Eval() inside another span. The root of the trace is a workflow span with the run group name as its workflow name, and these attributes:

Wrapped model calls and wrap() spans in the task become children of the root span. Each scorer runs in its own child workflow span, named after the scorer and tagged with statsig.eval.scorer_name. Every span in the trace has statsig.environment set to eval, so the Traces page hides these spans by default, and online evaluators don't grade them.

Eval() sends each record's traceID and spanID with its scores, so the scores also appear on the root span in Traces. Eval() flushes the spans before it returns. Without initializeTracing(), Eval() doesn't create traces.

Use other Statsig SDK methods

getStatsig() returns the underlying Statsig instance, however you initialized the AI SDK. Use it for any Node Server SDK method:

const statsig = statsigAI.getStatsig();

// Check a gate
const enabled = statsig.checkGate(user, "my_gate");

// Log an event
statsig.logEvent(user, "my_event", 1);

Upgrade to version 0.0.1-beta.34

If you upgrade from version 0.0.1-beta.33, check these changes:

• No alias package: The SDK no longer needs @statsig/statsig-node-core-rc. You can remove the alias from your dependencies.
• Fewer OpenTelemetry dependencies: The SDK no longer installs @opentelemetry/sdk-node, @opentelemetry/auto-instrumentations-node, @opentelemetry/sdk-metrics, or @opentelemetry/sdk-trace-node. With npm, an app could import them before without listing them as dependencies. If your app imports any of them, add it to your dependencies. For NodeSDK, go to Initialize tracing.
• Model calls inside workflows: The SDK no longer marks model calls inside a wrap() workflow span with statsig.span.llm_root, so online evaluators grade only the workflow span. If your workflow spans don't record input and output messages, online evaluators grade nothing in those turns. To set the messages, go to Grade a multi-step agent turn.
• Eval traces: With initializeTracing(), Eval() creates a trace for each record and flushes spans before it returns. For details, go to Traces for eval records.
• Eval result values: Eval() sends non-string inputs, outputs, expected values, and score metadata as JSON. Before, Statsig rejected the whole run with a 400 error when any of them wasn't a string.

The release also adds statsigDataset(), the dataset option of Eval(), and withIncomingTraceContext(). For every change, go to the release notes.

Upgrade to version 0.0.1-beta.33

Version 0.0.1-beta.33 is the first release on npm after 0.0.1-beta.30. If you upgrade from an earlier version, check these changes:

• Message format: Wrappers record gen_ai.input.messages and gen_ai.output.messages in the OpenTelemetry GenAI format (role plus parts), not as OpenAI-shaped messages. Update any code that parses these attributes.
• System instructions: System and developer messages, and the OpenAI instructions parameter, move out of gen_ai.input.messages into gen_ai.system_instructions. Turn on capture_system_instructions or capture_all to record them.
• Duplicate OpenAI requests: In versions 0.0.1-beta.11 through 0.0.1-beta.30, wrapOpenAI sent every request to OpenAI twice. Version 0.0.1-beta.33 sends each request once.

The release also adds wrapAnthropic, wrapGemini, withVersion(), withStatsigEnvironment(), and the traceId, spanId, and reasoning options of logEvalGrade(). For every change, go to the release notes.