See why runs failed

A failed run says why in failure. Across a project, runs.failures counts a window's failed runs by cause and by version, so a new version that fails in a new way shows up as a line of its own.

Why one run failed

A run with status: "failed" has failure. Here a guardrail halted an agent's turn (GET /v1/runs/{runId}, runs.get in the clients):

{
  "id": "143475bd-9e28-49cf-9f68-d03d081317ef",
  …
  "flowId": "agent.turn",
  "flowVersion": "1.1.0",
  "status": "failed",
  …
  "completedAt": "2026-10-11T07:27:40.781Z",
  …
  "failure": {
    "code": "guardrail-violation",
    "message": "Turn blocked by guardrail 'acme.no-secret': output contains forbidden pattern(s): SECRET"
  },
  "agent": {
    "id": "acme.refunds",
    "version": "1.1.0",
    …
  },
  …
}

• code: the error's own code, such as guardrail-violation, capability-routing-failed or budget-exceeded; run-failed when it has none.

• message: what happened.

• reason: the error's own reason, when it gives one: for a turn whose approval was cancelled (hitl-cancelled), timeout when nobody decided in time.

• cause: what the error came from, when it says. A turn that no model could serve, because the agent needs a feature no registered model has:

  "failure": {
    "code": "capability-routing-failed",
    "message": "No registered provider satisfies the capability declaration",
    "cause": {
      "code": "capability-unsatisfiable",
      "message": "No registered provider satisfies the capability declaration",
      "reasons": []
    }
  }

Read failure, not failureMessage. failureMessage is the failure as the runtime recorded it, in its own format: for an agent's turn, that's JSON meant for the runtime.

A project's failed runs, by cause and version

TypeScript

const to = new Date();
const from = new Date(to.getTime() - 7 * 24 * 60 * 60 * 1000); // the last week
const failures = await kindgi.runs.failures({ projectId, from, to });
for (const g of failures.groups) console.log(g.code, g.subject.id, g.version, g.count);
for (const g of failures.outcomes) console.log(g.code, g.subject.id, g.version, g.count);
for (const g of failures.unrecorded) console.log('unrecorded', g.subject.id, g.version, g.count);

guardrail-violation acme.refunds 1.1.0 3
capability-routing-failed acme.vision 1.0.0 1
hitl-rejected acme.desk 1.0.0 1
unrecorded acme.refunds 1.0.0 2

Python

from datetime import datetime, timedelta, timezone

to = datetime.now(timezone.utc)
failures = kindgi.runs.failures(
    project_id=project_id, from_=(to - timedelta(days=7)).isoformat(), to=to.isoformat()
)
for g in failures.groups:
    print(g.code, g.subject.id, g.version, g.count)
for g in failures.outcomes:
    print(g.code, g.subject.id, g.version, g.count)
for g in failures.unrecorded:
    print("unrecorded", g.subject.id, g.version, g.count)

guardrail-violation acme.refunds 1.1.0 3
capability-routing-failed acme.vision 1.0.0 1
hitl-rejected acme.desk 1.0.0 1
unrecorded acme.refunds 1.0.0 2

In this project, version 1.1.0 of acme.refunds had three turns halted by a guardrail, and acme.vision one turn no model could serve. A reviewer rejected a turn of acme.desk. Two turns of acme.refunds 1.0.0 failed on a runtime from before 0.1.6, which didn't record causes.

The answer in full (GET /v1/runs/failures?projectId=…&from=…&to=…):

{
  "from": "2026-10-11T07:26:28.000Z",
  "to": "2026-10-11T07:28:44.000Z",
  "groups": [
    {
      "code": "guardrail-violation",
      "subject": { "kind": "agent", "id": "acme.refunds" },
      "version": "1.1.0",
      "count": 3,
      "firstSeen": "2026-10-11T07:27:39.298Z",
      "lastSeen": "2026-10-11T07:27:40.781Z",
      "exampleRunId": "143475bd-9e28-49cf-9f68-d03d081317ef"
    },
    {
      "code": "capability-routing-failed",
      "subject": { "kind": "agent", "id": "acme.vision" },
      "version": "1.0.0",
      "count": 1,
      "firstSeen": "2026-10-11T07:27:41.499Z",
      "lastSeen": "2026-10-11T07:27:41.499Z",
      "exampleRunId": "0c902f2b-5134-409c-b5cf-3a7a46fd8a47"
    }
  ],
  "outcomes": [
    {
      "code": "hitl-rejected",
      "subject": { "kind": "agent", "id": "acme.desk" },
      "version": "1.0.0",
      "count": 1,
      "firstSeen": "2026-10-11T07:27:44.263Z",
      "lastSeen": "2026-10-11T07:27:44.263Z",
      "exampleRunId": "bcd7da7e-28f4-428e-9b90-2e8c70ae656c"
    }
  ],
  "unrecorded": [
    {
      "subject": { "kind": "agent", "id": "acme.refunds" },
      "version": "1.0.0",
      "count": 2,
      "firstSeen": "2026-10-11T07:27:33.045Z",
      "lastSeen": "2026-10-11T07:27:33.712Z",
      "exampleRunId": "23f69746-a744-40a6-94c3-89b45d772f91"
    }
  ],
  "total": 7
}

• groups: failed runs with the same cause (code), subject and version, the most first. The subject is an agent (its turns) or a flow (its runs), and the version is the agent's or the flow's. count is how many failed in the window, firstSeen and lastSeen when the first and the latest did, and exampleRunId is the latest, to open.
• outcomes: people's decisions, which aren't errors: a turn a reviewer rejected (hitl-rejected), or whose approval was cancelled (hitl-cancelled, with reason: "timeout" when nobody decided in time).
• unrecorded: runs that failed on a runtime from before 0.1.6, which didn't record a cause. They're counted by subject and version only.
• total: every failed run in the three lists.

What's counted:

• from and to are when the runs failed, at most 90 days apart.
• Replays, the runs an eval run makes, and dry runs aren't counted. A turn inside a flow counts under its agent, and the flow's run under the flow.
• Failed runs go with their retention, so a window can hold fewer than failed then.
• Each list holds 50 groups at most (limit, up to 200).

It needs read on the project.

One agent, by version

agentId (or flowId) counts one subject's runs. groupBy: ['version'] (group_by="version") groups them by version alone, without the cause:

{
  "from": "2026-10-11T07:26:28.000Z",
  "to": "2026-10-11T07:28:44.000Z",
  "groups": [
    {
      "subject": { "kind": "agent", "id": "acme.refunds" },
      "version": "1.1.0",
      "count": 3,
      "firstSeen": "2026-10-11T07:27:39.298Z",
      "lastSeen": "2026-10-11T07:27:40.781Z",
      "exampleRunId": "143475bd-9e28-49cf-9f68-d03d081317ef"
    }
  ],
  "outcomes": [],
  "unrecorded": [
    {
      "subject": { "kind": "agent", "id": "acme.refunds" },
      "version": "1.0.0",
      "count": 2,
      "firstSeen": "2026-10-11T07:27:33.045Z",
      "lastSeen": "2026-10-11T07:27:33.712Z",
      "exampleRunId": "23f69746-a744-40a6-94c3-89b45d772f91"
    }
  ],
  "total": 5
}

groupBy: ['code'] groups by cause alone, across versions; the default is both.

In the console

A project's What failed page, under Runs, shows the same read for 24 hours, 7, 30 or 90 days. On the runs above it says "In the last 7 days, 7 runs failed: 4 errors (in 2 groups), 1 people's decision on approvals, 2 from before causes were recorded.", then a table for each kind:

• Errors, by cause and version: "A guardrail blocked the answer." for acme.refunds 1.1.0, 3 runs; "No model provider could take the call." for acme.vision 1.0.0, 1 run.
• People's decisions on approvals: "A reviewer rejected it, it was withdrawn, or no one decided in time: an outcome, not an error."
• From before causes were recorded: "These failed on a runtime that didn't record why; each run's page shows its message."

Each row opens its latest run and its failed runs. List runs has the rest of the page.

A failed run's page names its cause in the same words, then the runtime's own and the code. A turn a guardrail halted keeps the answer back ("Blocked by a guardrail, not sent: the answer the model wrote", with Show what was blocked) and names the guardrail:

A guardrail blocked the answer.
Turn blocked by guardrail 'acme.no-secret': output contains forbidden pattern(s): SECRET
Guardrail: acme.no-secret
Code guardrail-violation