Skip to content
Kindgi

Publish an eval suite

POST
/v1/eval-suites
curl --request POST \
  --url https://example.com/v1/eval-suites \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{ "id": "example", "projectId": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0", "tenantId": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0", "version": "example", "kind": "accuracy", "description": "example", "spec": {} }'

Body is a full EvalSuite — the server validates top-level shape (id, tenantId, semver version, kind ∈ closed enum, spec is an object). Deeper spec validation is the runtime consumer’s responsibility per kind (eval-judge adapter for accuracy/pairwise/regression, HITL bridge for human-review, sandbox handler for custom). Re-publishing an existing (suiteId, version) returns 409 eval-suite-already-registered. Idempotency-Key applies (retries with the same key replay the original 201).

Authorizations

Section titled “Authorizations”

Parameters

Section titled “Parameters”

Header Parameters

Section titled “Header Parameters”
Idempotency-Key
string
>= 1 characters

Caller-supplied idempotency key. Retries with the same key return the original response byte-identical (per docs/API-ROUTE-CONVENTIONS.md §3.1).

Request Bodyrequired

Section titled “Request Bodyrequired”
Media typeapplication/json
object
id
required
string
>= 1 characters
projectId
required

Project this belongs to (its content scope). Required: missing, or not a project in the caller’s tenant → 400 bad-input.

string format: uuid
tenantId

Optional. When present, must match the caller tenant (server-derived from the token). Cross-tenant publish is rejected.

string format: uuid
version
required
string
/^\d+\.\d+\.\d+$/
kind
required
string
Allowed values: accuracy pairwise regression human-review benchmark custom
description
string
spec
required
object
key
additional properties
any

Responses

Section titled “Responses”

201

Section titled “201”

Eval suite published.

Media typeapplication/json
object
suiteId
required
string
version
required
string
Examplegenerated
{
  "suiteId": "example",
  "version": "example"
}

400

Section titled “400”

Validation failed (see details.issues).

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

401

Section titled “401”

Missing / malformed / expired / revoked bearer token.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

409

Section titled “409”

Eval suite already registered at that (id, version).

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

500

Section titled “500”

Server error (unmapped domain code or framework crash).

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}