Skip to content
Kindgi

List tenant eval suites

View as Markdown
GET
/v1/eval-suites
curl --request GET \
  --url 'https://example.com/v1/eval-suites?limit=25&kind=accuracy&scopeKind=tenant&inherit=true&includeRetired=false' \
  --header 'Authorization: Bearer <token>'

Cursor-paginated. Optional ?kind= narrows to a single eval kind (exact match); optional ?name= is a prefix match on EvalSuite.id. Sort order: suite id ascending. Latest version per id. A retired suite (every version unregistered) is listed only with ?includeRetired=true, as its highest version with unregisteredAt, so it can be found and a version reinstated.

Authorizations

Section titled “Authorizations”

Parameters

Section titled “Parameters”

Query Parameters

Section titled “Query Parameters”
limit
integer
default: 25 >= 1 <= 100

1..100. Default 25.

cursor
string

Where the previous page ended: its nextCursor, as it came. A runtime that seals cursors takes one only for the same list, filters and caller, within a day; otherwise 400 bad-input, and the list starts again without it. Absent → first page.

kind
string
Allowed values: accuracy pairwise regression human-review benchmark custom judged

Filter to eval suites of a single kind. Values: accuracy | pairwise | regression | human-review | benchmark | custom | judged.

name
string

Prefix match on EvalSuite.id. Dotted namespaces are the natural filter shape.

scopeKind

Discriminator for the ?scopeKind + ?scopeId + ?inherit triplet. Tenant carries no id (implicit from session); org/project require scopeId.

string
Allowed values: tenant org project

Optional scope discriminator. If absent, no scope filter is applied.

scopeId
string
>= 1 characters

Required IF scopeKind is org or project. MUST be absent if scopeKind=tenant (tenant is implicit from the session). Malformed combinations return 400 scope-invalid.

inherit
boolean
default: true

Default true. false = literal-at-this-scope only (admin/audit view). Load-bearing for policy/config-scoped resources (mcp-endpoints); documented no-op for content-scoped resources (agents/flows/tools/…).

includeRetired
boolean

When true, retired items (every version unregistered) are listed too, each as its highest version with that version’s unregisteredAt. Default: false (items with an active version only).

Responses

Section titled “Responses”

200

Section titled “200”

Page of eval suites.

Media typeapplication/json
object
data
required
Array<object>
object
id
required
string
>= 1 characters
tenantId
required
string format: uuid
projectId

The test set’s project: every version of a test set is in the one project. Absent when the runtime doesn’t record it: a pack kindgi dev serves from disk, or a runtime before Kindgi 0.1.6.

string format: uuid
version
required

Semver — publishing a modified suite produces a new version.

string
/^\d+\.\d+\.\d+$/
kind
required
string
Allowed values: accuracy pairwise regression human-review benchmark custom judged
description
string
spec
required

Kind-specific suite body. For accuracy, typically { cases: [{ input, expectedOutput }], grader?: { adapterId, config? } }. For pairwise, typically { prompts, variantA, variantB }. For regression, typically { baseline, cases }. For human-review, typically { rubric, reviewerRole }. For benchmark, typically { benchmark: { name, version } }. For custom, typically { handler: { modulePath, entrypointPath }, cases }.

object
key
additional properties
any
unregisteredAt

Present only on an unregistered version: one GET …/versions?includeTombstoned=true lists, or a retired test set (every version unregistered) as GET /v1/eval-suites?includeRetired=true lists it.

string format: date-time
nextCursor
string
hasMore
required
boolean
Example
{
  "data": [
    {
      "kind": "accuracy"
    }
  ]
}

400

Section titled “400”

Malformed query parameter (unknown kind).

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

401

Section titled “401”

Missing / malformed / expired / revoked bearer token.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

See also