Cancel an eval run
const url = 'https://example.com/v1/eval-runs/2489E9AD-2EE2-8E00-8EC9-32D5F69181C0/cancel';const options = {method: 'POST', headers: {Authorization: 'Bearer <token>'}};
try { const response = await fetch(url, options); const data = await response.json(); console.log(data);} catch (error) { console.error(error);}curl --request POST \ --url https://example.com/v1/eval-runs/2489E9AD-2EE2-8E00-8EC9-32D5F69181C0/cancel \ --header 'Authorization: Bearer <token>'Best-effort cancel. Returns 409 when the run is already terminal.
Authorizations
Section titled “Authorizations”Parameters
Section titled “Parameters”Path Parameters
Section titled “Path Parameters”EvalRunId — opaque branded UUID.
Responses
Section titled “Responses”Cancel accepted.
object
The project the eval run is in: the one it was started in. Absent on a runtime before Kindgi 0.1.6.
object
object
Kind-specific opaque JSON. For accuracy, contains { passCount, totalCount, meanScore, perCase[] }. For judged (a comparison), { summary, perCase[] }: the summary has the baseline (the versions behind the recorded runs) and the candidate ({ kind: "agent", agentId, version } or { kind: "flow", flowId, version, versions? }), the case counts (cases, diverged, refusedWrites, errors, and stopped: flow cases that stopped at a write the replay refused, left out of the metrics), the models that answered, and metrics (weightedYesShare, judgedCoverage, weightedPrecisionAtK, each { baseline, candidate, delta, n, weight, baselineN, baselineWeight, direction, k?, spread? }); each case has its replay runs, the scores, the items kept, dropped and new, the tool calls with what happened to each, and stopped (what it would have done) when it stopped. Other kinds define their own shapes as their dispatchers ship.
object
A comparison eval run’s settings (a judged suite).
object
object
object
object
A segment path in projectId, coarse to fine, as live versions resolve it. Needs projectId.
object
Lowercase, like an identifier: company, contact-role.
Whether replayed reads use the past run’s results when it has them (recorded), or run live.
Agents and tools a flow runs at other exact versions than the flow version’s pins (“this flow, with acme.scorer at 0.4.0”), without publishing a new flow version: a comparison’s flow candidate (versions on the start body and the run’s comparison), and the flow runs that replay it (a run’s versions). Each id must be an agent or tool the flow uses.
object
Tool id → exact version.
object
Agent id → exact version, for agent nodes with or without a version of their own.
object
Which judgments count: each at its class’s weight (as-recorded, the default), or only those recorded while their class was restricted (Judgment.restricted), the others weighing 0 (restricted-only).
For an agent candidate: block content its replays run instead of the version’s pinned content (an improvement pass’s search). settings: values by settings block id, each a block the version pins, satisfying its schema. prompts: a template for the prompt block the version pins, which reads and names only what the agent has (its parameters, the variables the current template reads, the settings blocks it pins, its tools’ and blocks’ ids) and is at most twice as long. Anything else is 400 validation-failed. A comparison with overrides can’t gate a promotion.
object
object
object
object
object
Only part of the test set’s cases: split once into a hold-out part (about holdOutShare of them) and a search part (the rest), stratified by judgment (the cases with a “no” and the others are split on their own, a stratum of two or more giving each part at least one), in the order of a hash of each case id and seed. The same seed always splits the same test set the same way. A promotion gate refuses a comparison on the search part (comparison.sample).
object
Rescore that comparison eval run instead of replaying: its replays’ outputs are scored again, with the judgments recorded on them since (a changed answer judged on the replay itself). Nothing runs, and the run rescored stays as it was. Set by POST /v1/eval-runs/{runId}/rescore.
Example
{ "kind": "accuracy", "status": "pending", "comparison": { "baseline": "recorded", "reads": "recorded", "classWeights": "as-recorded", "sample": { "part": "search" } }}Missing / malformed / expired / revoked bearer token.
object
object
Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).
Optional, kind-specific.
object
Server-assigned request id; also echoed via X-Request-Id header.
Examplegenerated
{ "error": { "code": "example", "message": "example", "details": {}, "requestId": "example" }}No eval run with that id under this tenant.
object
object
Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).
Optional, kind-specific.
object
Server-assigned request id; also echoed via X-Request-Id header.
Examplegenerated
{ "error": { "code": "example", "message": "example", "details": {}, "requestId": "example" }}Run already terminal.
object
object
Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).
Optional, kind-specific.
object
Server-assigned request id; also echoed via X-Request-Id header.
Examplegenerated
{ "error": { "code": "example", "message": "example", "details": {}, "requestId": "example" }}