Skip to content
Kindgi

Correct a run

View as Markdown
POST
/v1/corrections
curl --request POST \
  --url https://example.com/v1/corrections \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{ "runId": "example", "decisions": "right", "shouldCall": [ { "tool": "example", "args": {} } ], "mustNotCall": [ "example" ], "answer": "right", "shouldSay": "example", "why": "example", "items": [ { "key": "example", "pointer": "example", "rank": 1, "verdict": "right" } ], "correctorGroupId": "example", "participantId": "example" }'

Records whether a finished run’s decisions and answer were right, and what was right instead, optionally under a corrector group that applies to the run’s project or agent (unclassified corrections count with weight 1). Each item’s pointer (a JSON Pointer) must resolve in the run’s output. It’s sorted at save: an end user’s correction stays open and unsorted; decisions: "wrong" goes to the agent’s owner (with-owner); a wrong answer goes to Improve (open); nothing wrong is fixed as recorded-right. The first correction of a run also stores a copy of the run’s input and output. madeBy is the authenticated caller, never the body. A second correction by the same caller for the same run and participantId supersedes the earlier one. Needs write on the run’s project.

Authorizations

Section titled “Authorizations”

Parameters

Section titled “Parameters”

Header Parameters

Section titled “Header Parameters”
Idempotency-Key
string
>= 1 characters

Caller-supplied idempotency key, scoped to the caller: the same key from someone else in the tenant is their own request (from 0.1.5). Retries with the same key return the original response byte-identical (per docs/API-ROUTE-CONVENTIONS.md §3.1). A retry sent while the first request still runs is answered 409 idempotency-key-in-flight with Retry-After, on a runtime whose store holds keys (from 0.1.5); retry after it, and you get the first request’s answer. An answer that carries a secret (a new API key or public run token, a session token, a generated signing secret) isn’t kept: a retry gets 409 idempotency-key-replay-withheld, with the first request’s status and when it succeeded (from 0.1.5). A request whose key store can’t be read before it runs is answered 503 idempotency-store-unavailable with Retry-After: nothing ran, so retry with the same key (from 0.3.0).

Request Bodyrequired

Section titled “Request Bodyrequired”
Media typeapplication/json

At least one of decisions, answer and items. decisions: "wrong" needs an entry in shouldCall or mustNotCall; answer: "wrong" needs shouldSay or why.

object
runId
required
string
>= 1 characters
decisions
string
Allowed values: right wrong
shouldCall

Only with decisions: "wrong": the calls the agent should have made.

Array<object>
<= 50 items
object
tool
required
string
>= 1 characters <= 200 characters
args

Only when the inputs matter: the top-level argument fields to compare, by value. Left out, only the call is checked. At most 4000 characters in canonical JSON.

object
<= 100 properties
mustNotCall

Only with decisions: "wrong": tools that shouldn’t have been called. A tool can’t be in both.

Array<string>
<= 50 items
answer
string
Allowed values: right wrong
shouldSay

Only with answer: "wrong": what it should have said.

string
<= 4000 characters
why
string
<= 4000 characters
items

For an answer that is a ranked list: each item, and whether it was right. Each key once.

Array<object>
<= 100 items
object
key
required

The caller’s stable id for the item, such as a matched case’s id, never the item’s own text.

string
>= 1 characters <= 200 characters
pointer

Where the item is in the run’s output, as a JSON Pointer (RFC 6901), e.g. /matches/2.

string
<= 500 characters
rank

The item’s position in the ranked list (0 = first).

integer
verdict
required
string
Allowed values: right wrong
correctorGroupId

Optional. When given it must exist and apply to the run.

string
>= 1 characters
participantId

An app’s opaque id for its end user, when correcting on their behalf.

string
>= 1 characters

Responses

Section titled “Responses”

201

Section titled “201”

Correction recorded.

Media typeapplication/json

A person’s correction of one run: whether its decisions (the tool calls) and its answer were right, and what was right instead. It says at least one of decisions, answer and items.

object
id
required
string
tenantId
required
string
projectId
required
string
runId
required
string
subject
required

What a corrected run ran: an agent at a version, or a flow at a version.

object
kind
required
string
Allowed values: agent flow
id
required
string
version
required
string
decisions

Whether the run’s decisions were right. Absent when it says nothing about them.

string
Allowed values: right wrong
shouldCall

With decisions: "wrong": the calls the agent should have made.

Array<object>
object
tool
required
string
>= 1 characters <= 200 characters
args
required

The partial set of top-level argument fields to compare, by value. Fields left out are not compared; an empty object asks only that the tool was called. At most 4000 characters in canonical JSON.

object
<= 100 properties
mustNotCall

With decisions: "wrong": the tools that should not have been called.

Array<string>
answer

Whether the run’s answer was right. Absent when it says nothing about it.

string
Allowed values: right wrong
shouldSay

With answer: "wrong": what it should have said.

string
why
string
items

For an answer that is a ranked list: each item, and whether it was right there.

Array<object>
object
key
required

The caller’s stable id for the item, such as a matched case’s id, never the item’s own text.

string
>= 1 characters
pointer

Where the item is in the run’s output, as a JSON Pointer (RFC 6901), e.g. /matches/2.

string
rank

The item’s position in the ranked list (0 = first).

integer
verdict
required
string
Allowed values: right wrong
correctorGroupId

The corrector group the correction is recorded under. Absent when unclassified (counts with weight 1).

string
expert

Present (true) when the corrector group was restricted (assertableBy) when the correction was recorded, so whoever recorded it was checked against it. A restriction added or lifted later doesn’t change it.

boolean
madeBy
required

Who asserted a correction: the authenticated caller, never a typed name.

object
kind
required
string
Allowed values: user service
id
required

A user id, or for a service token its token or session id.

string
participantId

The app’s opaque id for its end user who corrected it, when an app corrected it on their behalf.

string
status
required

Where it is in being fixed. with-owner while the agent’s owner decides; fixed only when a pin or a deploy fixed it, or it said nothing was wrong; removed can’t come back.

string
Allowed values: open being-fixed with-owner fixed wont-fix removed
sortedTo

Who works on it: Improve, a developer, or the agent’s owner. Absent while unsorted.

string
Allowed values: improve developer owner
sorted
object
by
required
One of:
object
kind
required
string
Allowed values: fix-job rule
reason
string
at
required
string format: date-time
link
One of:
object
kind
required
string
Allowed values: improve-run suggested-change
id
required

The Improve run’s or suggested change’s id, without its prefix.

string
fixedIn

Present while it’s fixed, and beside reopenedIn after a test run reopened it.

object
version

The version that fixed it. Absent for recorded-right, which nothing had to fix.

string
how
required

pin: a pin fixed it. latest: a deploy or an add moved Latest in a project that follows it. recorded-right: it said nothing was wrong.

string
Allowed values: pin latest recorded-right
by

Who asserted a correction: the authenticated caller, never a typed name.

object
kind
required
string
Allowed values: user service
id
required

A user id, or for a service token its token or session id.

string
at
required
string format: date-time
reopenedIn

Present after a test run got a fixed correction wrong again: it’s open, or with-owner when it’s sorted to the owner, with the fixedIn it had beside it. A status set by hand clears both, and a re-sort alone keeps them. A test run that gets it right again makes it fixed with a new fixedIn.

object
version
required

The version that gets it wrong.

string
how
required

How the project came to run it. pin: a pin. went-back: a pin of a version earlier than the one it ran just before. unpin: an unpin. latest: a deploy or an add moved Latest in a project that follows it.

string
Allowed values: pin went-back unpin latest
by

Who pinned, went back or unpinned. Absent when Latest moved.

object
kind
required
string
Allowed values: user service
id
required

A user id, or for a service token its token or session id.

string
at
required
string format: date-time
statusNote
object
text
required
string
by
required

Who asserted a correction: the authenticated caller, never a typed name.

object
kind
required
string
Allowed values: user service
id
required

A user id, or for a service token its token or session id.

string
at
required
string format: date-time
createdAt
required
string format: date-time
updatedAt
required

When it last changed: createdAt until something changes it.

string format: date-time
unregisteredAt

Set when the correction was removed or superseded.

string format: date-time
supersededBy

The correction that replaced this one.

string
Example
{
  "subject": {
    "kind": "agent"
  },
  "decisions": "right",
  "answer": "right",
  "items": [
    {
      "verdict": "right"
    }
  ],
  "expert": true,
  "madeBy": {
    "kind": "user"
  },
  "status": "open",
  "sortedTo": "improve",
  "sorted": {
    "by": {
      "kind": "fix-job"
    }
  },
  "link": {
    "kind": "improve-run"
  },
  "fixedIn": {
    "how": "pin",
    "by": {
      "kind": "user"
    }
  },
  "reopenedIn": {
    "how": "pin",
    "by": {
      "kind": "user"
    }
  },
  "statusNote": {
    "by": {
      "kind": "user"
    }
  }
}

400

Section titled “400”

Malformed body, or item-not-found (an item’s pointer resolves to nothing in the output), or corrector-group-not-applicable.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

401

Section titled “401”

Missing / malformed / expired / revoked bearer token.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

403

Section titled “403”

permission-denied: not allowed to correct this run. corrector-group-not-allowed: the corrector group is restricted (assertableBy) and the caller may not assert it.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

404

Section titled “404”

run-not-found.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

409

Section titled “409”

run-not-finished: the run has no output to correct yet.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

410

Section titled “410”

run-erased: an erasure cleared the run’s content (a person’s words were removed); there’s nothing to correct.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

413

Section titled “413”

The request body is over the API’s limit (request-too-large; details.maxBytes says how much).

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

500

Section titled “500”

Server error (unmapped domain code or framework crash).

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

503

Section titled “503”

The Idempotency-Key store couldn’t be read before the request ran (idempotency-store-unavailable), so nothing was changed; Retry-After (whole seconds) says when to try again with the same key.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

See also