Skip to content
Kindgi

Fetch an eval suite (latest version)

View as Markdown
GET
/v1/eval-suites/{suiteId}
curl --request GET \
  --url https://example.com/v1/eval-suites/example \
  --header 'Authorization: Bearer <token>'

An item more than one project serves (an environment order carries it to later environments) answers its newest active version that can run in a project the caller reads, and 404 when there is none. projectId narrows that to one project. An item with one project answers its latest active version.

Authorizations

Section titled “Authorizations”

Parameters

Section titled “Parameters”

Path Parameters

Section titled “Path Parameters”
suiteId
required
string
>= 1 characters

EvalSuiteId — stable string chosen by the caller (e.g. acme.drafting-accuracy).

Query Parameters

Section titled “Query Parameters”
projectId
string format: uuid

The newest active version that can run in this project, a project the caller reads; 404 when there’s none, or when the caller doesn’t read the project. An item with one project answers its latest version for its own project and 404 for any other; one that names no project (under kindgi dev, a pack’s agents, flows and tools) answers it for any. With authorization off, the caller reads every project. An older runtime ignores it and answers the latest version.

Responses

Section titled “Responses”

200

Section titled “200”

Latest eval suite version.

Media typeapplication/json
object
id
required
string
>= 1 characters
tenantId
required
string format: uuid
projectId

The test set’s project: every version of a test set is in the one project. Absent when the runtime doesn’t record it: a pack kindgi dev serves from disk, or a runtime before Kindgi 0.1.6.

string format: uuid
version
required

Semver — publishing a modified suite produces a new version.

string
/^\d+\.\d+\.\d+$/
kind
required
string
Allowed values: accuracy pairwise regression human-review benchmark custom judged
description
string
spec
required

Kind-specific suite body. For accuracy, typically { cases: [{ input, expectedOutput }], grader?: { adapterId, config? } }. For pairwise, typically { prompts, variantA, variantB }. For regression, typically { baseline, cases }. For human-review, typically { rubric, reviewerRole }. For benchmark, typically { benchmark: { name, version } }. For custom, typically { handler: { modulePath, entrypointPath }, cases }.

object
key
additional properties
any
unregisteredAt

Present only on an unregistered version: one GET …/versions?includeTombstoned=true lists, or a retired test set (every version unregistered) as GET /v1/eval-suites?includeRetired=true lists it.

string format: date-time
Example
{
  "kind": "accuracy"
}

400

Section titled “400”

projectId is not a project id (a UUID).

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

401

Section titled “401”

Missing / malformed / expired / revoked bearer token.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

404

Section titled “404”

No eval suite with that id under this tenant.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

See also