For AI agents: a documentation index is available at /llms.txt. Append .md to any page URL for markdown, or send Accept: text/markdown.

Create Eval Grader

Create a grader in the project catalog. Attach it to a prompt with Update Prompt Eval Settings, or pass its name to Create Eval Run.

Create a grader in the project catalog. Attach it to a prompt with Update Prompt Eval Settings, or pass its name to Create Eval Run.

posthttps://statsigapi.net/console/v1/eval_graders

Authorizations

STATSIG-API-KEYstringrequired

apiKey (header)

Body

application/json
namestringrequired

Grader name, unique in the project.

Constraints:min length: 1
promptobjectrequired
Show child attributes
typestringrequired

label_model is an LLM judge over {{output}} (content is the rubric); string_check compares {{output}} to a dataset column with operation; text_similarity scores {{output}} against a column with evaluation_metric; python runs source.

Allowed values:label_modelstring_checktext_similaritypython
contentstring

Rubric for label_model. May reference {{output}} and dataset columns.

Constraints:nullable
operationstring
Allowed values:eqnelikeilike
Constraints:nullable
evaluation_metricstring

Similarity measure for text_similarity.

Allowed values:fuzzy_matchbleugleumeteorrouge_1rouge_2rouge_3rouge_4rouge_5rouge_lcosine
Constraints:nullable
columnstring

Dataset column compared against for string_check and text_similarity.

Constraints:nullable
sourcestring

Python source for a python grader.

Constraints:nullable

Response

application/json
201· Create Eval Grader
messagestringrequired

A simple string explaining the result of the operation.

dataobjectrequired
Show child attributes
idstringrequired
namestringrequired
typestringrequired

Grader type: label_model, string_check, text_similarity, python, completion_tokens, completion_latency, or logged_score.

isGlobalbooleanrequired
promptoneOfrequired
Constraints:objectobjectobjectobjectobjectobjectobjectobject
typestringrequired
Allowed values:label_modelstring_checktext_similaritypythonprogrammatic_gradercompletion_tokenscompletion_latencylogged_score
contentstring
operationstring
Allowed values:eqnelikeilike
columnstring
evaluation_metricstring
Allowed values:fuzzy_matchbleugleumeteorrouge_1rouge_2rouge_3rouge_4rouge_5rouge_lcosine
sourcestring
choiceScoresobjectrequired
Constraints:nullable
createdTimenumberrequired
Constraints:format: double
lastModifiedTimenumberrequired
Constraints:format: double
400· Invalid request. Please check the request input and try again.
statusintegerrequired
Allowed values:400
messagestringrequired

Was this helpful?