Skip to content
Kindgi

Create a judging rule

POST
/v1/projects/{projectId}/judging-rules
curl --request POST \
  --url https://example.com/v1/projects/example/judging-rules \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{ "name": "example", "when": { "agentIds": [ "example" ], "flowIds": [ "example" ], "versions": [ "example" ], "status": [ "completed" ], "includeDryRuns": true }, "sample": 1, "maxOpen": 1, "judgeClassId": "example", "enabled": true }'

Version 1 of a rule. From then on, a top-level run of the project that matches when as it ends (and falls in sample, while the rule has fewer than maxOpen open items) is queued for a person’s judgment; replays never are. A rule only lists runs: it never starts a model. Needs write on the project.

Authorizations

Section titled “Authorizations”

Parameters

Section titled “Parameters”

Path Parameters

Section titled “Path Parameters”
projectId
required
string

ProjectId — opaque branded string.

Header Parameters

Section titled “Header Parameters”
Idempotency-Key
string
>= 1 characters

Caller-supplied idempotency key, scoped to the caller: the same key from someone else in the tenant is their own request (from 0.1.5). Retries with the same key return the original response byte-identical (per docs/API-ROUTE-CONVENTIONS.md §3.1). A retry sent while the first request still runs is answered 409 idempotency-key-in-flight with Retry-After, on a runtime whose store holds keys (from 0.1.5); retry after it, and you get the first request’s answer. An answer that carries a secret (a new API key or public run token, a session token, a generated signing secret) isn’t kept: a retry gets 409 idempotency-key-replay-withheld, with the first request’s status and when it succeeded (from 0.1.5).

Request Bodyrequired

Section titled “Request Bodyrequired”
Media typeapplication/json

A judging rule as written. It only lists runs: nothing here starts a model.

object
name
required
string
>= 1 characters <= 200 characters
when
required

Which of a project’s runs a rule matches as they end. Every field narrows; absent fields don’t. agentIds or flowIds, not both. Top-level runs only (an agent’s own runs, not its turns as a flow’s step); replays never match.

object
agentIds
Array<string>
>= 1 items <= 50 items
flowIds
Array<string>
>= 1 items <= 50 items
versions

These versions exactly, or live: runs that got the agent’s live version, not one the caller named. Runs from before the runtime recorded how their version was chosen never match live.

Array<string>
>= 1 items <= 50 items
status

How the run ended. Only completed today: a judgment needs a completed run, so failed and cancelled are refused (400). Absent: completed.

Array<string>
>= 1 items
Allowed values: completed failed cancelled
includeDryRuns

Dry runs are left out unless true.

boolean
sample

The share of matching runs queued, decided by the run and rule ids: the same every time and across the rule’s versions, so raising it keeps the runs it took before. Default 1.

number
> 0 <= 1
maxOpen

Queue nothing while this rule has this many open items. Absent: no cap.

integer
>= 1 <= 10000
judgeClassId

Whose judgment it wants: one of this class closes it. Absent: any judgment does.

string
enabled

Default true.

boolean

Responses

Section titled “Responses”

201

Section titled “201”

The rule, version 1.

Media typeapplication/json

One version of a judging rule; the latest live one applies.

object
ruleId
required
string
projectId
required
string
version
required
integer
>= 1
name
required
string
>= 1 characters <= 200 characters
when
required

Which of a project’s runs a rule matches as they end. Every field narrows; absent fields don’t. agentIds or flowIds, not both. Top-level runs only (an agent’s own runs, not its turns as a flow’s step); replays never match.

object
agentIds
Array<string>
>= 1 items <= 50 items
flowIds
Array<string>
>= 1 items <= 50 items
versions

These versions exactly, or live: runs that got the agent’s live version, not one the caller named. Runs from before the runtime recorded how their version was chosen never match live.

Array<string>
>= 1 items <= 50 items
status

How the run ended. Only completed today: a judgment needs a completed run, so failed and cancelled are refused (400). Absent: completed.

Array<string>
>= 1 items
Allowed values: completed failed cancelled
includeDryRuns

Dry runs are left out unless true.

boolean
sample
required
number
> 0 <= 1
maxOpen

Queue nothing while this rule has this many open items. Absent: no cap.

integer
>= 1 <= 10000
judgeClassId

Whose judgment it wants: one of this class closes it. Absent: any judgment does.

string
enabled
required
boolean
createdBy

Who wrote this version.

string
createdAt
required
string format: date-time
unregisteredAt
string format: date-time
Example
{
  "when": {
    "status": [
      "completed"
    ]
  }
}

400

Section titled “400”

Malformed request body.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

401

Section titled “401”

Missing / malformed / expired / revoked bearer token.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

404

Section titled “404”

No project with that id (project-not-found), or no live judge class judgeClassId (judge-class-not-found).

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

409

Section titled “409”

Idempotency-Key was reused with a different body, or resource-state conflict.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

500

Section titled “500”

Server error (unmapped domain code or framework crash).

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}