# Correct a run's output

> Record whether a finished run's decisions and answer were right, and what was right instead, from your app, the CLI or the console, see where each correction goes, and weight corrections by corrector group.

This guide records whether what a run did and returned was right, counts some people's corrections more than others, and queues the runs that need a correction.

A **correction** says whether a finished run's decisions, the tools it called, were right, whether its answer was right, and what was right instead. For an answer that is a ranked list, it says which items were right. People who know the work, your users or your experts, correct it. Each correction goes to whoever fixes it, and moves on until it's fixed. The corrections also tell you how an agent or flow version does: they're the cases of the agent's [test set](https://docs.kindgi.com/v0.5.0/guides/improve/build-a-test-set.md), and you [try a new version](https://docs.kindgi.com/v0.5.0/guides/improve/compare-an-agent-version.md) on them. [Corrections reference](https://docs.kindgi.com/v0.5.0/guides/improve/expert-queue-reference.md) has every field, rule and console page.

## Before you start

- **A finished run to correct,** and its id, on a runtime such as `kindgi dev`, with Kindgi 0.2's CLI. [List runs](https://docs.kindgi.com/v0.5.0/guides/runs/list-runs.md) finds one.
- **Write access to the run's project,** when authorization is on.
- **For the app's half:** Node 22.12 or later, Python 3.11 or later, or Java 17 or later, with Kindgi 0.2's client, and `KINDGI_API_URL` and `KINDGI_API_TOKEN` set.

## Correct a run's answer

A correction records whether the run's answer was right.

From the CLI, correct the run's answer:

- **TypeScript**

  ```sh
  pnpm exec kindgi corrections add --run=<run-id> --answer=right --why="Echoes the message back"
  ```

- **Python**

  ```sh
  uv run kindgi corrections add --run=<run-id> --answer=right --why="Echoes the message back"
  ```

- **Java**

  ```sh
  ./kindgiw corrections add --run=<run-id> --answer=right --why="Echoes the message back"
  ```

- **Scala**

  ```sh
  ./kindgiw corrections add --run=<run-id> --answer=right --why="Echoes the message back"
  ```

* `--run=<run-id>` is the run's id.
* `--answer=right` says the run's answer was right, or `--answer=wrong` that it was wrong. For a flow run, the answer is its whole output.
* `--why` says why. It's optional, but a wrong answer needs it, or `--should-say` with what the answer should have said.

It prints the correction:

```json excerpt
{
  "id": "512df8f6-6e27-4c4a-934e-1b345dfc510a",
  "projectId": "d4910d76-7355-4022-8f3c-514361cfa986",
  "runId": "bc6d1b53-3562-48c9-9f29-719779e3d110",
  "subject": { "kind": "agent", "id": "acme-ops.echo-agent", "version": "0.1.0" },
  "answer": "right",
  "why": "Echoes the message back",
  "madeBy": { "kind": "user", "id": "26fc414a-b49f-4e1f-a022-162865770178" },
  "status": "fixed",
  "fixedIn": {
    "how": "recorded-right",
    "by": { "kind": "user", "id": "26fc414a-b49f-4e1f-a022-162865770178" },
    "at": "2026-10-06T04:08:27.667Z"
  },
  "createdAt": "2026-10-06T04:08:27.667Z",
  …
}
```

Nothing in it is wrong, so it's `fixed` as soon as it's saved, as `recorded-right`. [Where a correction goes](https://docs.kindgi.com/v0.5.0/guides/improve/correct-a-runs-output.md#where-a-correction-goes) has the other cases.

From your app, when your users correct runs in your own UI, ask your coding agent: "I want to record whether my user says a run's answer was right." For reference, the script looks like this, with the run's id as its argument:

- **TypeScript**

  `correct.ts`

  ```ts
  import { createClient } from '@kindgi/sdk/client';

  const kindgi = createClient(); // KINDGI_API_URL, KINDGI_API_TOKEN, or the running kindgi dev

  const correction = await kindgi.corrections.create({
    runId: process.argv[2],
    answer: 'right',
    why: 'Echoes the message back',
    participantId: 'customer-user-17', // your id for the user who corrected it, if you correct for your users
  });
  console.log(correction.id, correction.answer, correction.madeBy);
  ```

- **Python**

  `correct.py`

  ```python
  import sys

  from kindgi.client import Kindgi

  kindgi = Kindgi()  # KINDGI_API_URL, KINDGI_API_TOKEN, or the running kindgi dev

  correction = kindgi.corrections.create(
      run_id=sys.argv[1],
      answer="right",
      why="Echoes the message back",
      participant_id="customer-user-18",  # your id for the user who corrected it, if you correct for your users
  )
  print(correction.id, correction.answer, correction.made_by)
  ```

- **Java**

  `src/main/java/com/acme/app/RecordCorrection.java`

  ```java
  package com.acme.app;

  import com.kindgi.client.Kindgi;
  import com.kindgi.client.models.CreateCorrectionBody;
  import com.kindgi.client.models.Correction;

  public final class RecordCorrection {
    public static void main(String[] args) {
      Kindgi kindgi = Kindgi.create(); // KINDGI_API_URL, KINDGI_API_TOKEN, or the running kindgi dev

      Correction correction = kindgi.corrections().create(CreateCorrectionBody.builder()
          .runId(args[0])
          .answer(CreateCorrectionBody.Answer.RIGHT)
          .why("Echoes the message back")
          .participantId("customer-user-19") // your id for the user who corrected it, if you correct for your users
          .build());
      System.out.println(correction.id() + " " + correction.answer() + " " + correction.madeBy());
    }
  }
  ```

  ```text
  42a5cdac-a5b4-4bf1-a777-d46ead4ec743 right CorrectionMadeBy[kind=user, id=ebdba929-5422-4cb2-b6bb-2c046109feb8]
  ```

- **Scala**

  `src/main/scala/acme/app/RecordCorrection.scala`

  ```scala
  package acme.app

  import com.kindgi.client.Kindgi
  import com.kindgi.client.models.CreateCorrectionBody

  object RecordCorrection {
    def main(args: Array[String]): Unit = {
      val kindgi = Kindgi.create() // KINDGI_API_URL, KINDGI_API_TOKEN, or the running kindgi dev

      val correction = kindgi.corrections().create(CreateCorrectionBody.builder()
        .runId(args(0))
        .answer(CreateCorrectionBody.Answer.RIGHT)
        .why("Echoes the message back")
        .participantId("customer-user-20") // your id for the user who corrected it, if you correct for your users
        .build())
      println(s"${correction.id} ${correction.answer} ${correction.madeBy}")
    }
  }
  ```

  ```text
  339dbf38-f4b0-4a9a-a1f6-0dc4585dae93 right CorrectionMadeBy[kind=user, id=ebdba929-5422-4cb2-b6bb-2c046109feb8]
  ```

* **`answer`** is `right` or `wrong`. A wrong answer needs what it should have said, `shouldSay`, or why it was wrong, `why`.
* **`participantId`** is your app's own opaque id for the user who made the correction. Who called comes from the token, as `madeBy`. Your users' corrections stay `open` and unsorted until a person sorts them, unless the project [sorts them](https://docs.kindgi.com/v0.5.0/guides/improve/correct-a-runs-output.md#where-a-correction-goes). [Record your users' thumbs](https://docs.kindgi.com/v0.5.0/guides/improve/record-your-users-thumbs.md) puts a thumbs up and down under your app's answers.
* **For an answer that is a ranked list,** `items` says which items were right, as [An agent's test set](https://docs.kindgi.com/v0.5.0/guides/improve/build-a-test-set.md#correct-some-runs) shows. Each item's `key` is an id, such as the matched case's id, never the item's text.
* **One correction per run and person.** Correcting the run again, as the same caller and participant, replaces your earlier correction, and the earlier one stays in the history. From a client, send the whole correction each time. The CLI does that for you: each `kindgi corrections add` replaces the parts it gives and keeps the rest. Saying a part was right drops what should have happened there, and an empty `--why=` clears the why. A correction that was being fixed or with the owner starts again, sorted afresh, and the CLI says which it replaced.

The run now has the correction. Only a finished run can be corrected, and each item's pointer must find something. [Corrections](https://docs.kindgi.com/v0.5.0/guides/improve/expert-queue-reference.md#corrections) has the rules and the refusals.

In the console, a finished run you may correct has a **Corrections** page, one of the run's pages in the ▾ at the end of the breadcrumbs, and **Correct this run** opens it. It lists the answer and each element of a list in the output, each with **Yes** and **No** and the counts so far. It lists each correction with who made it, its group and its reason.

Once yours is saved, it shows under the form: "Saved as" its id, its status, who fixes it, with **Hand it to someone else…**, and links to what comes next: **In Corrections › To fix** while it isn't fixed, **Run Improve on** the agent when Improve fixes it, **A case in its test set** when it's one, and **Next run to review**, the next run waiting in [To review](https://docs.kindgi.com/v0.5.0/guides/improve/expert-queue-reference.md#to-review). The run's header has **Next run to review** too.

## Say what it should have done

A correction of an agent's run can also say what the run should have done (runtime 0.2.0 or later): the tool calls it should have made, with the arguments that matter, and the tools it shouldn't have called. It's about the run's decisions, not its answer. When you [try a version](https://docs.kindgi.com/v0.5.0/guides/improve/compare-an-agent-version.md#what-it-should-have-done) on test cases built from the run, it checks what that version does against it.

Record it with your mark:

- **TypeScript**

  ```sh
  pnpm exec kindgi corrections add --run=<run-id> --decisions=wrong --should-call='acme-store.offer-store-credit:{"orderId":"A-1042"}' --must-not-call=acme-store.refund --why="Final sale: store credit, never a refund"
  ```

- **Python**

  ```sh
  uv run kindgi corrections add --run=<run-id> --decisions=wrong --should-call='acme-store.offer-store-credit:{"orderId":"A-1042"}' --must-not-call=acme-store.refund --why="Final sale: store credit, never a refund"
  ```

- **Java**

  ```sh
  ./kindgiw corrections add --run=<run-id> --decisions=wrong --should-call='acme-store.offer-store-credit:{"orderId":"A-1042"}' --must-not-call=acme-store.refund --why="Final sale: store credit, never a refund"
  ```

- **Scala**

  ```sh
  ./kindgiw corrections add --run=<run-id> --decisions=wrong --should-call='acme-store.offer-store-credit:{"orderId":"A-1042"}' --must-not-call=acme-store.refund --why="Final sale: store credit, never a refund"
  ```

* `--decisions=wrong` marks the run's decisions wrong.
* `--should-call` is a call it should have made: the tool, then, after a `:`, the top-level arguments that matter, as JSON. Each is compared whole. Arguments left out aren't compared, and the tool alone asks only that it was called. Repeat it for each call.
* `--must-not-call` is a tool it shouldn't have called at all. Repeat it for each tool.

It prints the correction, with what it should have done as its `shouldCall` and `mustNotCall`:

```text
CAPTURE: the printed correction, with "decisions": "wrong", its "shouldCall" and "mustNotCall", "why", "status": "with-owner" and "sortedTo": "owner"
```

- **Part of your one correction of the run.** Saving it again replaces what you said about the decisions, and keeps what you said about the answer.
- **It goes to the agent's owner.** Wrong decisions are `with-owner` as soon as they're saved: a change to what the agent does may loosen a limit, and only the owner decides that.
- **Only the agent's own tools.** A tool the agent version that ran doesn't have is refused: `The corrected version has no tool "<tool>".`

From your app, ask your coding agent: "I want to record what my expert says the run should have done." The clients' `corrections.create` takes it as `decisions: 'wrong'`, with `shouldCall`, the calls it should have made (`[{ tool, args? }]`, with `args` only when the inputs matter), and `mustNotCall`, the tool ids it shouldn't have called. In Python, they're `should_call` and `must_not_call`. To say its decisions were right, send `decisions: 'right'` alone.

In the console, a completed agent run's **Corrections** page, which **Correct this run** opens, has **What should it have done?**:

1. It lists the run's calls. Keep the ones that were right, and edit or untick the argument fields that matter.
2. Flag the calls that shouldn't have happened, and add any the run missed with **Add a call…**.
3. Choose **The decisions were wrong** or **The decisions were right**, add **Why (optional)**, and **Save correction**.

## See the corrections

A run's corrections say how its output was corrected, and by whom:

- **TypeScript**

  ```sh
  pnpm exec kindgi corrections list --run=<run-id>      # newest first; replaced and removed ones are left out
  pnpm exec kindgi corrections show <correction-id>       # with copies of the run's input and output
  pnpm exec kindgi corrections remove <correction-id>
  ```

- **Python**

  ```sh
  uv run kindgi corrections list --run=<run-id>      # newest first; replaced and removed ones are left out
  uv run kindgi corrections show <correction-id>       # with copies of the run's input and output
  uv run kindgi corrections remove <correction-id>
  ```

- **Java**

  ```sh
  ./kindgiw corrections list --run=<run-id>      # newest first; replaced and removed ones are left out
  ./kindgiw corrections show <correction-id>       # with copies of the run's input and output
  ./kindgiw corrections remove <correction-id>
  ```

- **Scala**

  ```sh
  ./kindgiw corrections list --run=<run-id>      # newest first; replaced and removed ones are left out
  ./kindgiw corrections show <correction-id>       # with copies of the run's input and output
  ./kindgiw corrections remove <correction-id>
  ```

* `<run-id>` is the run's id, and `<correction-id>` a correction's `id`, from the list.
* `list` also filters by agent and version, flow, corrector group, participant and project.
* `remove` is for the correction's author, or an admin of its project. The correction becomes `removed`, and it can't come back.

`list` prints a page of corrections as JSON: `data`, `hasMore`, and `nextCursor` when there's another page. `show` prints one correction, and `remove` prints its id with `"removed": true`.

A correction keeps copies of the run's input and output, so it outlives the run's own retention. The clients have the same: `corrections.list`, `corrections.get` and `corrections.remove`. Their `corrections.list` also filters by status, sorting, who made it and when, as [Where a correction goes](https://docs.kindgi.com/v0.5.0/guides/improve/correct-a-runs-output.md#where-a-correction-goes) shows.

## Export the corrections

An agent's corrections export as a CSV file, for a spreadsheet, or as JSON lines, for a script:

- **TypeScript**

  ```sh
  pnpm exec kindgi corrections export --agent=<agent-id> --since=2026-10-01 --until=2026-11-01 --out=october.csv
  pnpm exec kindgi corrections export --agent=<agent-id> --format=jsonl --status=not-fixed --out=not-fixed.jsonl
  ```

- **Python**

  ```sh
  uv run kindgi corrections export --agent=<agent-id> --since=2026-10-01 --until=2026-11-01 --out=october.csv
  uv run kindgi corrections export --agent=<agent-id> --format=jsonl --status=not-fixed --out=not-fixed.jsonl
  ```

- **Java**

  ```sh
  ./kindgiw corrections export --agent=<agent-id> --since=2026-10-01 --until=2026-11-01 --out=october.csv
  ./kindgiw corrections export --agent=<agent-id> --format=jsonl --status=not-fixed --out=not-fixed.jsonl
  ```

- **Scala**

  ```sh
  ./kindgiw corrections export --agent=<agent-id> --since=2026-10-01 --until=2026-11-01 --out=october.csv
  ./kindgiw corrections export --agent=<agent-id> --format=jsonl --status=not-fixed --out=not-fixed.jsonl
  ```

* `<agent-id>` is the agent whose corrections you export. It filters as the clients' `corrections.list` does: `--project`, `--status` and `--sorted-to` (each repeatable), `--from` (`experts`, `end-users` or `all`), `--since` and `--until`. A date, such as `2026-10-01`, is midnight UTC.
* Like `list`, it leaves out replaced corrections, and removed ones unless `--status=removed` asks for them.
* `--out` names the file. On macOS and Linux, only you can read it. It takes that name only once the whole export has arrived, and an existing file is kept unless you add `--force`. Then it prints the agent, the format, the file and its size in bytes, as JSON. Without `--out`, the export is printed.
* Each correction comes only when you may read its project.

The CSV has a header row, then one correction per line: `id`, `projectId`, `runId`, `agentVersion`, `madeBy` (`user:<id>` or `service:<id>`), `participantId`, `expert`, `decisions`, `shouldCall` and `mustNotCall` (these two as JSON), `answer`, `shouldSay`, `why`, `sortedTo`, `status`, `fixedIn` (the version that fixed it), `reopenedIn` (the version whose test run reopened it) and `createdAt`. A field that starts with `=`, `+`, `-`, `@`, a tab or a carriage return gets a leading `'`, so a spreadsheet shows it rather than running it as a formula. Where a spreadsheet's list separator is `;`, import the file with a comma as the separator rather than opening it directly. JSON lines have one correction per line, as each one in `list`'s `data`. Neither has the run's own input or output, only the run's id.

Every export is in the audit log, with who exported, the filters and how many rows went out. In the TypeScript and Python clients it's `corrections.export`: TypeScript's returns the body as a stream, and Python's yields it in chunks. The Java client has no method for it: use the CLI, or call `GET /v1/corrections/export`, as [Export](https://docs.kindgi.com/v0.5.0/guides/improve/expert-queue-reference.md#export) shows.

## Where a correction goes

Each correction has a `status`, and `sortedTo` says who fixes it: `improve`, a `developer`, or the agent's `owner`. It's sorted when it's saved:

- **Your users' corrections,** made with a `participantId`, stay `open` and unsorted until a person sorts them, and never start Improve, unless the project sorts them, as below. One under a restricted corrector group is an expert's, and is sorted like any other.
- **Wrong decisions** go to the agent's owner: `with-owner`, sorted to `owner`. This is so even when the answer was wrong too.
- **A wrong answer,** or a wrong item of it, goes to Improve: `open`, sorted to `improve`.
- **Nothing wrong** is `fixed`, as `recorded-right`.

To sort your users' corrections like the others, turn on the project's `sortEndUserCorrections`, with `admin` on the project:

```sh
curl -X PATCH "$KINDGI_API_URL/v1/projects/<project-id>" \
  -H @<(printf 'Authorization: Bearer %s\n' "$KINDGI_API_TOKEN") -H 'content-type: application/json' \
  -d '{"sortEndUserCorrections":true}'
```

In TypeScript, that's `kindgi.projects.update(projectId, { sortEndUserCorrections: true })`, and in Python, `kindgi.projects.update(project_id, sort_end_user_corrections=True)`. A project reads it back as `sortEndUserCorrections`, `false` until it's on.

Sorting applies to the corrections saved after you turn it on: one saved before stays unsorted until a person sorts it. A schedule that counts everyone's corrections (`from: everyone`) counts your users' corrections sorted to improve only while it's on. Once it's off, the schedule leaves them out, even ones sorted while it was on or by a person, though an Improve run that starts still takes them in.

A person then moves it on. With `publish` on the agent, `corrections.change` re-sorts it, changes its status, or both. Here a developer says they're fixing it, with a link to the work:

```sh
curl -X PATCH "$KINDGI_API_URL/v1/corrections/<correction-id>" \
  -H @<(printf 'Authorization: Bearer %s\n' "$KINDGI_API_TOKEN") -H 'content-type: application/json' \
  -d '{"status":"being-fixed","link":"https://code.acme.example/store/pull/42"}'
```

In TypeScript, that's `kindgi.corrections.change(correctionId, { status: 'being-fixed', link })`. It answers the changed correction. A change its status doesn't allow is refused with `409 correction-status-change-not-allowed`. [Status and sorting](https://docs.kindgi.com/v0.5.0/guides/improve/expert-queue-reference.md#status-and-sorting) has every status, and [Change a correction](https://docs.kindgi.com/v0.5.0/guides/improve/expert-queue-reference.md#change-a-correction) the changes each allows.

To see how many of a project's corrections are in each status, or one agent's, count them:

```sh
curl -s -H @<(printf 'Authorization: Bearer %s\n' "$KINDGI_API_TOKEN") \
  "$KINDGI_API_URL/v1/corrections/counts?scopeKind=project&scopeId=<project-id>&agentId=acme-store.refunds"
```

```json
{ "open": 3, "beingFixed": 1, "withOwner": 1, "fixed": 12, "wontFix": 0, "removed": 0 }
```

In TypeScript, that's `kindgi.corrections.counts({ projectId, agentId })`. To list the ones behind a count, `corrections.list` takes `status`, such as `not-fixed`, short for `open`, `being-fixed` and `with-owner`, and `sortedTo`.

## Weight corrections by corrector group

Without a group, every correction counts once. A **corrector group** counts some people's corrections more, for the workspace, a project, or an agent in a project.

Add a group, then correct as it:

- **TypeScript**

  ```sh
  pnpm exec kindgi corrector-groups add --name=expert --weight=3 --project=<project-id>
  pnpm exec kindgi corrections add --run=<run-id> --answer=right --group=<corrector-group-id>
  ```

- **Python**

  ```sh
  uv run kindgi corrector-groups add --name=expert --weight=3 --project=<project-id>
  uv run kindgi corrections add --run=<run-id> --answer=right --group=<corrector-group-id>
  ```

- **Java**

  ```sh
  ./kindgiw corrector-groups add --name=expert --weight=3 --project=<project-id>
  ./kindgiw corrections add --run=<run-id> --answer=right --group=<corrector-group-id>
  ```

- **Scala**

  ```sh
  ./kindgiw corrector-groups add --name=expert --weight=3 --project=<project-id>
  ./kindgiw corrections add --run=<run-id> --answer=right --group=<corrector-group-id>
  ```

* `--name` names the group, and `--weight=3` counts its corrections three times.
* `--project=<project-id>` is the project it applies to. [Read the test set](https://docs.kindgi.com/v0.5.0/guides/improve/build-a-test-set.md#read-the-test-set) shows how to find a project's id.
* `--group=<corrector-group-id>` is the group's `id`, which `add` printed. In the clients, it's `correctorGroupId`.

The correction now counts with the group's weight.

### Restrict who may correct as a group

An expert group says who may correct as it, so the people using your app can't move the figures that release a version.

Add the group with a restriction, before your reviewers correct runs:

- **TypeScript**

  ```sh
  pnpm exec kindgi corrector-groups add --name=expert --weight=3 --tenant --min-reviewer-role=senior --description="Order desk leads"
  ```

- **Python**

  ```sh
  uv run kindgi corrector-groups add --name=expert --weight=3 --tenant --min-reviewer-role=senior --description="Order desk leads"
  ```

- **Java**

  ```sh
  ./kindgiw corrector-groups add --name=expert --weight=3 --tenant --min-reviewer-role=senior --description="Order desk leads"
  ```

- **Scala**

  ```sh
  ./kindgiw corrector-groups add --name=expert --weight=3 --tenant --min-reviewer-role=senior --description="Order desk leads"
  ```

* `--tenant` makes it the workspace's group.
* `--min-reviewer-role=senior` lets only a [reviewer](https://docs.kindgi.com/v0.5.0/guides/approvals/decide-an-approval.md#reviewers-and-roles) of that role or above correct as it: `standard`, `senior` or `admin`.
* `--description` says who it is for.
* `--principal-kind=user|service` and `--principal-id=<id>`, which repeat, restrict it to people or service tokens, or to named ones.

A caller must meet each restriction you set. One who doesn't is refused with `403`, and nothing is recorded:

```text
Error [auth]: You can't correct as "expert": it needs a senior reviewer or above; you're a standard reviewer.
Error [auth]: You can't correct as "tokens": only service tokens may assert it.
```

A correction made under a restriction carries `"expert": true`. It's decided when the correction is made, so corrections recorded before the restriction don't count as expert. [Corrector groups](https://docs.kindgi.com/v0.5.0/guides/improve/expert-queue-reference.md#corrector-groups) has the rest.

## Queue the runs to correct

**Queue rules** pick the project's runs that need a person's correction, so your experts don't pick runs one at a time. As a run of the project ends, each rule checks it. A run that any rule matches joins the project's queue of answers waiting for a correction, **To review** in the console's **Corrections**, once.

To make a rule, ask your coding agent: "Queue a sample of my agent's pinned-version runs for people to correct." For reference, the code looks like this:

- **TypeScript**

  `queue-rule.ts`

  ```ts
  import { createClient } from '@kindgi/sdk/client';

  const kindgi = createClient();
  const project = await kindgi.projects.getDefault();

  const rule = await kindgi.projects.queueRules.create(project.id, {
    name: 'Pinned refund runs',
    when: { agentIds: ['acme.refunds'], versions: ['pinned'] },
    sample: 0.05, // 5% of them
    maxOpen: 20, // at most 20 waiting at once
  });
  console.log(rule.ruleId, rule.version);
  ```

- **Python**

  `queue_rule.py`

  ```python
  from kindgi.client import Kindgi

  kindgi = Kindgi()
  project = kindgi.projects.get_default()

  rule = kindgi.projects.queue_rules.create(
      project.id,
      name="Pinned refund runs",
      when={"agent_ids": ["acme.refunds"], "versions": ["pinned"]},
      sample=0.05,  # 5% of them
      max_open=20,  # at most 20 waiting at once
  )
  print(rule.rule_id, rule.version)
  ```

- **Java**

  `src/main/java/com/acme/app/QueueRule.java`

  ```java
  package com.acme.app;

  import com.kindgi.client.Kindgi;
  import com.kindgi.client.models.QueueRuleSpec;
  import com.kindgi.client.models.QueueRuleWhen;
  import com.kindgi.client.models.Project;
  import java.util.List;

  public final class QueueRule {
    public static void main(String[] args) {
      Kindgi kindgi = Kindgi.create(); // KINDGI_API_URL, KINDGI_API_TOKEN, or the running kindgi dev

      Project project = kindgi.projects().getDefault();
      var rule = kindgi.projects().queueRules().create(project.id(), QueueRuleSpec.builder()
          .name("Pinned refund runs")
          .when(QueueRuleWhen.builder().agentIds(List.of("acme.refunds")).versions(List.of("pinned")).build())
          .sample(0.05) // 5% of them
          .maxOpen(20L) // at most 20 waiting at once
          .build());
      System.out.println(rule.ruleId() + " " + rule.version());
    }
  }
  ```

  ```text
  0867c21c-c911-4d68-9fca-bf3234768931 1
  ```

- **Scala**

  `src/main/scala/acme/app/QueueRule.scala`

  ```scala
  package acme.app

  import com.kindgi.client.Kindgi
  import com.kindgi.client.models.{QueueRuleSpec, QueueRuleWhen}
  import scala.jdk.CollectionConverters._

  object QueueRule {
    def main(args: Array[String]): Unit = {
      val kindgi = Kindgi.create() // KINDGI_API_URL, KINDGI_API_TOKEN, or the running kindgi dev

      val project = kindgi.projects().getDefault()
      val rule = kindgi.projects().queueRules().create(project.id, QueueRuleSpec.builder()
        .name("Pinned refund runs")
        .when(QueueRuleWhen.builder().agentIds(List("acme.refunds").asJava).versions(List("pinned").asJava).build())
        .sample(0.05) // 5% of them
        .maxOpen(20L) // at most 20 waiting at once
        .build())
      println(s"${rule.ruleId} ${rule.version}")
    }
  }
  ```

  ```text
  d57204bd-e0d2-4a2c-ade9-d87b83761e41 1
  ```

* `when` is which runs it matches: here, runs of `acme.refunds`'s pinned version.
* `sample` is the share of them to queue, `0.05` for 5%.
* `maxOpen` stops the rule adding runs while that many of its runs wait.

It prints the rule's id and its version, `1`. A rule as the runtime returns it, here one made with `sample: 1` and `maxOpen: 2`:

```json
{
  "ruleId": "19191620-85c3-4e2a-bd87-a0270d15ab53",
  "projectId": "153a0091-613e-4c34-9535-da8194f9068e",
  "version": 1,
  "name": "Pinned refund runs",
  "when": {
    "agentIds": [
      "acme.refunds"
    ],
    "versions": [
      "pinned"
    ]
  },
  "sample": 1,
  "maxOpen": 2,
  "enabled": true,
  "createdBy": "user:6161fd71-761c-4b92-be40-153923bad755",
  "createdAt": "2026-10-10T02:44:01.075Z"
}
```

From now on, the runs it matches join the queue as they end. It never runs a model or spends anything. [Queue rules](https://docs.kindgi.com/v0.5.0/guides/improve/expert-queue-reference.md#queue-rules) has every field, and how to try a rule before you make it.

### Work through the queue

The queue lists the runs waiting for a correction, oldest first. To list the ones you can close, ask your coding agent: "Show me the runs waiting for my correction." For reference, the code looks like this:

- **TypeScript**

  `expert-queue.ts`

  ```ts
  import { createClient } from '@kindgi/sdk/client';

  const kindgi = createClient();
  const project = await kindgi.projects.getDefault();

  const waiting = await kindgi.projects.expertQueue.list(project.id, { forMe: true });
  console.log(waiting.total, waiting.data.length);
  ```

- **Python**

  `expert_queue.py`

  ```python
  from kindgi.client import Kindgi

  kindgi = Kindgi()
  project = kindgi.projects.get_default()

  waiting = kindgi.projects.expert_queue.list(project.id, for_me=True)
  print(waiting.total, len(waiting.data))
  ```

- **Java**

  `src/main/java/com/acme/app/ExpertQueue.java`

  ```java
  package com.acme.app;

  import com.kindgi.client.Kindgi;
  import com.kindgi.client.models.ExpertQueuePage;
  import com.kindgi.client.models.Project;
  import com.kindgi.client.resources.ProjectsExpertQueueListParams;

  public final class ExpertQueue {
    public static void main(String[] args) {
      Kindgi kindgi = Kindgi.create(); // KINDGI_API_URL, KINDGI_API_TOKEN, or the running kindgi dev

      Project project = kindgi.projects().getDefault();
      ExpertQueuePage waiting = kindgi.projects().expertQueue()
          .list(project.id(), ProjectsExpertQueueListParams.builder().forMe(true).build());
      System.out.println(waiting.total() + " " + waiting.data().size());
    }
  }
  ```

  ```text
  0 0
  ```

- **Scala**

  `src/main/scala/acme/app/ExpertQueue.scala`

  ```scala
  package acme.app

  import com.kindgi.client.Kindgi
  import com.kindgi.client.resources.ProjectsExpertQueueListParams

  object ExpertQueue {
    def main(args: Array[String]): Unit = {
      val kindgi = Kindgi.create() // KINDGI_API_URL, KINDGI_API_TOKEN, or the running kindgi dev

      val project = kindgi.projects().getDefault()
      val waiting = kindgi.projects().expertQueue()
        .list(project.id, ProjectsExpertQueueListParams.builder().forMe(true).build())
      println(s"${waiting.total} ${waiting.data.size}")
    }
  }
  ```

  ```text
  0 0
  ```

`forMe: true` keeps the items you can close: those that want a group you may correct as, or any correction. It prints how many such items wait in all, and how many are on this page. One item, as that rule queued it:

```json
{
  "runId": "4d5fb3c0-6835-436d-ba29-fd182661d189",
  "projectId": "153a0091-613e-4c34-9535-da8194f9068e",
  "agentId": "acme.refunds",
  "agentVersion": "2.1.0",
  "flowId": "agent.turn",
  "runStatus": "completed",
  "completedAt": "2026-10-10T02:44:01.450Z",
  "rules": [
    {
      "ruleId": "19191620-85c3-4e2a-bd87-a0270d15ab53",
      "version": 1
    }
  ],
  "wantedGroupIds": [],
  "anyCorrection": true,
  "progress": {
    "total": 0,
    "byGroup": []
  },
  "addedAt": "2026-10-10T02:44:01.447Z",
  "state": "open",
  "can": {
    "dismiss": true,
    "reopen": false
  }
}
```

Correct each item's run, as [Correct a run's answer](https://docs.kindgi.com/v0.5.0/guides/improve/correct-a-runs-output.md#correct-a-runs-answer) shows. An item closes as `corrected` once each rule that queued it has the correction it wants. `expertQueue.dismiss`, with an optional reason, takes a run not worth correcting out of the queue. [Answers waiting for a correction](https://docs.kindgi.com/v0.5.0/guides/improve/expert-queue-reference.md#answers-waiting-for-a-correction) has every field and count.

Reading the rules and the queue takes read access to the project. Changing a rule, and dismissing or reopening an item, take write access, as correcting a run does.

## Next

- [Record your users' thumbs](https://docs.kindgi.com/v0.5.0/guides/improve/record-your-users-thumbs.md)
- [An agent's test set](https://docs.kindgi.com/v0.5.0/guides/improve/build-a-test-set.md)
- [Try a version on the test cases](https://docs.kindgi.com/v0.5.0/guides/improve/compare-an-agent-version.md)
- [Corrections reference](https://docs.kindgi.com/v0.5.0/guides/improve/expert-queue-reference.md)