Publish a test set

POST

/v1/test-sets

curl --request POST \
  --url https://example.com/v1/test-sets \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{ "id": "example", "projectId": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0", "tenantId": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0", "version": "example", "kind": "accuracy", "description": "example", "spec": {} }'

• Same-origin — path relative to the deployment root.

Body is a full TestSet — the server validates top-level shape (id, tenantId, semver version, kind ∈ closed enum, spec is an object). Deeper spec validation is the runtime consumer’s responsibility per kind (eval-judge adapter for accuracy/pairwise/regression, HITL bridge for human-review, sandbox handler for custom). Re-publishing an existing (testSetId, version) returns 409 test-set-already-registered. Idempotency-Key applies (retries with the same key replay the original 201). An id that starts with agent. is refused with 400 bad-input: those are agents’ own test sets, whose versions only a check or a test run makes.

Authorizations

• bearerAuth

Parameters

Header Parameters

Idempotency-Key

string

>= 1 characters

Caller-supplied idempotency key, scoped to the caller: the same key from someone else in the tenant is their own request (from 0.1.5). Retries with the same key return the original response byte-identical (per docs/API-ROUTE-CONVENTIONS.md §3.1). A retry sent while the first request still runs is answered 409 idempotency-key-in-flight with Retry-After, on a runtime whose store holds keys (from 0.1.5); retry after it, and you get the first request’s answer. An answer that carries a secret (a new API key or public run token, a session token, a generated signing secret) isn’t kept: a retry gets 409 idempotency-key-replay-withheld, with the first request’s status and when it succeeded (from 0.1.5). A request whose key store can’t be read before it runs is answered 503 idempotency-store-unavailable with Retry-After: nothing ran, so retry with the same key (from 0.3.0).

Request Body^required

Media typeapplication/json

object

id

required

string

>= 1 characters

projectId

required

Project this belongs to (its content scope). Required: missing, or not a project in the caller’s tenant → 400 bad-input.

string format: uuid

tenantId

Optional. When present, must match the caller tenant (server-derived from the token). Cross-tenant publish is rejected.

string format: uuid

version

required

string

/^\d+\.\d+\.\d+$/

kind

required

string

Allowed values: accuracy pairwise regression human-review benchmark custom corrections

description

string

spec

required

object

key

additional properties

any

Responses

201

Test set published.

Media typeapplication/json

object

testSetId

required

string

version

required

string

Example^generated

{
  "testSetId": "example",
  "version": "example"
}

400

Validation failed (see details.issues); or projectId isn’t a project id (a UUID); or the id starts with agent. (bad-input), an agent’s own test set.

Media typeapplication/json

object

error

required

object

code

required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string

message

required

string

details

Optional, kind-specific.

object

key

additional properties

any

requestId

required

Server-assigned request id; also echoed via X-Request-Id header.

string

Example^generated

{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

401

Missing / malformed / expired / revoked bearer token.

Media typeapplication/json

object

error

required

object

code

required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string

message

required

string

details

Optional, kind-specific.

object

key

additional properties

any

requestId

required

Server-assigned request id; also echoed via X-Request-Id header.

string

Example^generated

{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

404

The body’s projectId names no project of this tenant (project-not-found).

Media typeapplication/json

object

error

required

object

code

required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string

message

required

string

details

Optional, kind-specific.

object

key

additional properties

any

requestId

required

Server-assigned request id; also echoed via X-Request-Id header.

string

Example^generated

{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

409

test-set-already-registered: that (id, version) is taken. Or test-set-project-mismatch: the test set’s versions live in another project (a test set belongs to the project its first version was published into and never moves; the message doesn’t name the project).

Media typeapplication/json

object

error

required

object

code

required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string

message

required

string

details

Optional, kind-specific.

object

key

additional properties

any

requestId

required

Server-assigned request id; also echoed via X-Request-Id header.

string

Example^generated

{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

413

The request body is over the API’s limit (request-too-large; details.maxBytes says how much).

Media typeapplication/json

object

error

required

object

code

required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string

message

required

string

details

Optional, kind-specific.

object

key

additional properties

any

requestId

required

Server-assigned request id; also echoed via X-Request-Id header.

string

Example^generated

{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

500

Server error (unmapped domain code or framework crash).

Media typeapplication/json

object

error

required

object

code

required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string

message

required

string

details

Optional, kind-specific.

object

key

additional properties

any

requestId

required

Server-assigned request id; also echoed via X-Request-Id header.

string

Example^generated

{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

503

The Idempotency-Key store couldn’t be read before the request ran (idempotency-store-unavailable), so nothing was changed; Retry-After (whole seconds) says when to try again with the same key.

Media typeapplication/json

object

error

required

object

code

required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string

message

required

string

details

Optional, kind-specific.

object

key

additional properties

any

requestId

required

Server-assigned request id; also echoed via X-Request-Id header.

string

Example^generated

{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

See also

• GET /v1/test-sets: List tenant test sets
• Kindgi 0.4.0
• Error code bad-input
• Error code idempotency-store-unavailable
• Error code project-not-found
• Error code request-too-large
• Error code test-set-already-registered
• Error code test-set-project-mismatch
• Every error code