# Build a test set from judgments

POST

/v1/eval-suites/{suiteId}/versions/from-judgments

```sh
curl --request POST \
  --url https://example.com/v1/eval-suites/example/versions/from-judgments \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{ "version": "example", "projectId": "example", "agentId": "example", "agentVersion": "example", "flowId": "example", "since": "2026-04-15T12:00:00Z", "until": "2026-04-15T12:00:00Z", "judgeClassIds": [ "example" ], "minJudgments": 1, "segments": [ { "key": "example", "value": "example" } ], "description": "example" }'
```

Publishes a `judged` eval suite version whose cases are copies of judged runs of one agent (optionally one version) or flow, newest first, at most 1000. Each case holds the run’s input, what the turn read (`context`), the judged output, and each item’s judgments summed up: yes and no counts, the weight behind yes and behind all judgments (an unclassified judgment counts 1), and the reasons. `judgeClassIds` counts only judgments of those classes; `minJudgments` leaves out runs with fewer; `segments` keeps only runs started in that segment path or below it. Needs `admin` on the project.

## Authorizations

- **[bearerAuth](https://docs.kindgi.com/v0.1.8/reference/api.md#bearerauth)**

## Parameters

### Path Parameters

**suiteId**

required

string

\>= 1 characters

EvalSuiteId — stable string chosen by the caller (e.g. `acme.drafting-accuracy`).

### Header Parameters

**Idempotency-Key**

string

\>= 1 characters

Caller-supplied idempotency key, scoped to the caller: the same key from someone else in the tenant is their own request (from 0.1.5). Retries with the same key return the original response byte-identical (per `docs/API-ROUTE-CONVENTIONS.md` §3.1). A retry sent while the first request still runs is answered `409 idempotency-key-in-flight` with `Retry-After`, on a runtime whose store holds keys (from 0.1.5); retry after it, and you get the first request’s answer. An answer that carries a secret (a new API key or public run token, a session token, a generated signing secret) isn’t kept: a retry gets `409 idempotency-key-replay-withheld`, with the first request’s status and when it succeeded (from 0.1.5).

## Request Bodyrequired

application/json

Name the agent (`agentId`) or the flow (`flowId`) whose judged runs to use.

object

**version**

required

Semver version to publish, e.g. `1.0.0`.

string

**projectId**

required

string

\>= 1 characters

**agentId**

string

\>= 1 characters

**agentVersion**

Needs `agentId`.

string

\>= 1 characters

**flowId**

string

\>= 1 characters

**since**

Runs first judged at or after this time.

string format: date-time

**until**

Runs first judged before this time.

string format: date-time

**judgeClassIds**

Count only judgments recorded under these judge classes.

Array\<string>

**minJudgments**

Leave out runs with fewer counted judgments. Default 1.

integer

\>= 1

**segments**

Only runs started in this segment path or below it, coarse to fine (e.g. `company=acme`). A run judged before its segments were recorded with its judgments is in no segment, so it’s left out.

Array\<object>

object

**key**

required

Lowercase, like an identifier: `company`, `contact-role`.

string

/^\[a-z]\[a-z0-9\_-]{0,63}$/

**value**

required

string

\>= 1 characters <= 256 characters

**description**

string

##### Examplegenerated

```json
{
  "version": "example",
  "projectId": "example",
  "agentId": "example",
  "agentVersion": "example",
  "flowId": "example",
  "since": "2026-04-15T12:00:00Z",
  "until": "2026-04-15T12:00:00Z",
  "judgeClassIds": [
    "example"
  ],
  "minJudgments": 1,
  "segments": [
    {
      "key": "example",
      "value": "example"
    }
  ],
  "description": "example"
}
```

## Responses

### 201

Version published.

application/json

object

**suiteId**

required

string

**version**

required

string

**kind**

required

string

Allowed values: judged

**caseCount**

required

integer

**truncated**

required

Whether more judged runs matched than the 1000 cases a set holds.

boolean

##### Example

```json
{
  "kind": "judged"
}
```

### 400

Malformed body; or `projectId` isn’t a project id (a UUID).

application/json

object

**error**

required

object

**code**

required

Stable machine-readable discriminant. Values match domain error codes (see `docs/API-ROUTE-CONVENTIONS.md` §4.3).

string

**message**

required

string

**details**

Optional, kind-specific.

object

**_key_**

additional properties

any

**requestId**

required

Server-assigned request id; also echoed via `X-Request-Id` header.

string

##### Examplegenerated

```json
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}
```

### 401

Missing / malformed / expired / revoked bearer token.

application/json

object

**error**

required

object

**code**

required

Stable machine-readable discriminant. Values match domain error codes (see `docs/API-ROUTE-CONVENTIONS.md` §4.3).

string

**message**

required

string

**details**

Optional, kind-specific.

object

**_key_**

additional properties

any

**requestId**

required

Server-assigned request id; also echoed via `X-Request-Id` header.

string

##### Examplegenerated

```json
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}
```

### 403

`permission-denied`.

application/json

object

**error**

required

object

**code**

required

Stable machine-readable discriminant. Values match domain error codes (see `docs/API-ROUTE-CONVENTIONS.md` §4.3).

string

**message**

required

string

**details**

Optional, kind-specific.

object

**_key_**

additional properties

any

**requestId**

required

Server-assigned request id; also echoed via `X-Request-Id` header.

string

##### Examplegenerated

```json
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}
```

### 404

The body’s `projectId` names no project of this tenant (`project-not-found`).

application/json

object

**error**

required

object

**code**

required

Stable machine-readable discriminant. Values match domain error codes (see `docs/API-ROUTE-CONVENTIONS.md` §4.3).

string

**message**

required

string

**details**

Optional, kind-specific.

object

**_key_**

additional properties

any

**requestId**

required

Server-assigned request id; also echoed via `X-Request-Id` header.

string

##### Examplegenerated

```json
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}
```

### 409

`eval-suite-already-registered`: that (id, version) is taken. Or `eval-suite-project-mismatch`: the suite belongs to another project than the body’s `projectId` (suites never move).

application/json

object

**error**

required

object

**code**

required

Stable machine-readable discriminant. Values match domain error codes (see `docs/API-ROUTE-CONVENTIONS.md` §4.3).

string

**message**

required

string

**details**

Optional, kind-specific.

object

**_key_**

additional properties

any

**requestId**

required

Server-assigned request id; also echoed via `X-Request-Id` header.

string

##### Examplegenerated

```json
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}
```

### 413

The request body is over the API’s limit (`request-too-large`; `details.maxBytes` says how much).

application/json

object

**error**

required

object

**code**

required

Stable machine-readable discriminant. Values match domain error codes (see `docs/API-ROUTE-CONVENTIONS.md` §4.3).

string

**message**

required

string

**details**

Optional, kind-specific.

object

**_key_**

additional properties

any

**requestId**

required

Server-assigned request id; also echoed via `X-Request-Id` header.

string

##### Examplegenerated

```json
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}
```

### 500

Server error (unmapped domain code or framework crash).

application/json

object

**error**

required

object

**code**

required

Stable machine-readable discriminant. Values match domain error codes (see `docs/API-ROUTE-CONVENTIONS.md` §4.3).

string

**message**

required

string

**details**

Optional, kind-specific.

object

**_key_**

additional properties

any

**requestId**

required

Server-assigned request id; also echoed via `X-Request-Id` header.

string

##### Examplegenerated

```json
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}
```

### 501

`test-sets-not-supported`: this deployment cannot build test sets.

application/json

object

**error**

required

object

**code**

required

Stable machine-readable discriminant. Values match domain error codes (see `docs/API-ROUTE-CONVENTIONS.md` §4.3).

string

**message**

required

string

**details**

Optional, kind-specific.

object

**_key_**

additional properties

any

**requestId**

required

Server-assigned request id; also echoed via `X-Request-Id` header.

string

##### Examplegenerated

```json
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}
```

## See also

- [`GET /v1/eval-suites/{suiteId}`: Fetch an eval suite (latest version)](https://docs.kindgi.com/v0.1.8/reference/api/operations/evalsuitesget.md)
- [`GET /v1/eval-suites/{suiteId}/versions`: List versions of an eval suite](https://docs.kindgi.com/v0.1.8/reference/api/operations/evalsuitesversionslist.md)
- [`GET /v1/eval-suites/{suiteId}/versions/{version}`: Fetch a specific eval suite version](https://docs.kindgi.com/v0.1.8/reference/api/operations/evalsuitesversionsget.md)
- [`GET /v1/eval-suites/{suiteId}/versions/{version}/cases`: List the cases of a judged eval suite version](https://docs.kindgi.com/v0.1.8/reference/api/operations/evalsuiteslistcases.md)
- [Error code `eval-suite-already-registered`](https://docs.kindgi.com/v0.1.8/reference/errors.md#eval-suite-already-registered)
- [Error code `eval-suite-project-mismatch`](https://docs.kindgi.com/v0.1.8/reference/errors.md#eval-suite-project-mismatch)
- [Error code `permission-denied`](https://docs.kindgi.com/v0.1.8/reference/errors.md#permission-denied)
- [Error code `project-not-found`](https://docs.kindgi.com/v0.1.8/reference/errors.md#project-not-found)
- [Error code `request-too-large`](https://docs.kindgi.com/v0.1.8/reference/errors.md#request-too-large)
- [Error code `test-sets-not-supported`](https://docs.kindgi.com/v0.1.8/reference/errors.md#test-sets-not-supported)
- [Every error code](https://docs.kindgi.com/v0.1.8/reference/errors.md)
