OpenAI API (unofficial showcase)
POST/fine_tuning/alpha/graders/run

Run a grader

Runs a grader against a model sample and, optionally, a data source item. Supply a grader definition and model_sample to receive the overall reward, grader metadata, and any sub-rewards.

  • IdempotentThe SDK sends Idempotency-Key, so a retried request is only applied once.

3 body fields

Grader definition and the sample to evaluate, with an optional data source item.

graderobjectrequired
The grader used for the fine-tuning job.
itemobjectoptional
The dataset item provided to the grader. This will be used to populate the `item` namespace. See [the guide](https://developers.openai.com/api/docs/guides/graders) for more details.
model_samplestringrequired
The model sample to be evaluated. This value will be used to populate the `sample` namespace. See [the guide](https://developers.openai.com/api/docs/guides/graders) for more details. The `output_json` variable will be populated if the model sample is a valid JSON string.

3 status codes
200Returns the overall reward, grader metadata, sub-rewards, and model grader token usage.
rewardnumberrequired
metadataobjectrequired
sub_rewardsobjectrequired
model_grader_token_usage_per_modelobjectrequired
400Returned when the grader definition or request is invalid.
errorobjectrequired
429Returned when the request rate limit is exceeded.
errorobjectrequired

Error handling

A 400 is returned when the grader definition or request is invalid; grader and model_sample are required, and the grader must match a supported grader definition. A 429 is returned when the request rate limit is exceeded.