# kindgi judging-rules

> A project's judging rules — which finished runs a person is asked to correct (create / list / show / update / remove / versions / results / preview / queue-recent)

## `kindgi judging-rules`

**Synopsis**

```sh signature
kindgi judging-rules <command>
```

A project's judging rules — which finished runs a person is asked to correct (create / list / show / update / remove / versions / results / preview / queue-recent).

**Commands**

| Command                                                                                                                                | Description                                                                                                      |
| -------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| [`kindgi judging-rules create`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules-create)             | Create a judging rule: which of the project’s finished runs a person is asked to correct, and how many to queue. |
| [`kindgi judging-rules list`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules-list)                 | List the project’s live judging rules (their latest versions), oldest first.                                     |
| [`kindgi judging-rules show`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules-show)                 | Show a judging rule’s latest version.                                                                            |
| [`kindgi judging-rules update`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules-update)             | Change a judging rule: a new version with the given fields replaced.                                             |
| [`kindgi judging-rules remove`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules-remove)             | Unregister a judging rule, so it queues nothing more.                                                            |
| [`kindgi judging-rules versions`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules-versions)         | List every version of a judging rule, newest first.                                                              |
| [`kindgi judging-rules results`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules-results)           | A judging rule’s results since a time, by the rule’s version and the agent’s, never pooled.                      |
| [`kindgi judging-rules preview`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules-preview)           | What a rule would queue among the project’s recent runs, without queueing anything.                              |
| [`kindgi judging-rules queue-recent`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules-queue-recent) | Queue the rule’s matches among recent runs, to catch it up on runs that ended before it was there.               |

**See also**

- [Correct a run's output](https://docs.kindgi.com/v0.3.1/guides/evals/judge-a-runs-output.md)
- [Corrections reference](https://docs.kindgi.com/v0.3.1/guides/evals/judging-reference.md)

### `kindgi judging-rules create`

**Synopsis**

```sh signature
kindgi judging-rules create --name=<name> [--project=<id>] [--agent=<id>] [--flow=<id>] [--agent-version=<v>] [--status=completed] [--include-dry-runs] [--sample=<0-1>] [--max-open=<n>] [--class=<judge-class-id>] [--disabled] [--queue-last=<n>] [--queue-since=<time>]
```

Create a judging rule: which of the project’s finished runs a person is asked to correct, and how many to queue.

**Options**

| Option                    | Default                                                                              | Description                                                                                                                                           |
| ------------------------- | ------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--agent <value>`         | none (runs of any agent)                                                             | Match runs of this agent (repeatable). Repeatable.                                                                                                    |
| `--agent-version <value>` | none (any version)                                                                   | Match runs of this agent version exactly, or `live` for runs that got the agent's live version (repeatable). Repeatable.                              |
| `--class <value>`         | none (any correction closes an item)                                                 | The judge class whose correction closes an item. Without it, any correction closes it.                                                                |
| `--disabled`              | off                                                                                  | Create the rule disabled (it matches nothing).                                                                                                        |
| `--flow <value>`          | none (runs of any flow)                                                              | Match runs of this flow (repeatable). Repeatable.                                                                                                     |
| `--include-dry-runs`      | off                                                                                  | Also match dry runs; off by default. Replays never match.                                                                                             |
| `--max-open <value>`      | none (no cap)                                                                        | Add nothing while this rule has this many open items. Without it, no cap.                                                                             |
| `--name <value>`          | required                                                                             | A name for the rule. Required.                                                                                                                        |
| `--project <value>`       | the tenant's Default project                                                         | The project the rule is in. Without it, the tenant's Default project.                                                                                 |
| `--queue-last <value>`    | none (nothing is queued on create unless `--queue-last` or `--queue-since` is given) | Also queue the rule's matches among the last N finished runs, right after creating it (1 to 1000).                                                    |
| `--queue-since <value>`   | none                                                                                 | Also queue the rule’s matches among runs finished at or after this time (an ISO 8601 time with a zone, or a date).                                    |
| `--sample <value>`        | `1` (every matching run)                                                             | The share of matching runs to queue, greater than 0 and at most 1 (default 1). The same runs every time, by run and rule id.                          |
| `--status <value>`        | `completed` (only completed runs match)                                              | Match runs by how they ended. Only `completed` is accepted — a correction needs a completed run, so `failed` and `cancelled` are refused. Repeatable. |

Every command also takes the [global flags](https://docs.kindgi.com/v0.3.1/reference/cli.md#global-flags).

**Examples**

Queue a quarter of an agent's completed runs for correction.

```sh
kindgi judging-rules create --name="refund tone" --agent=acme.refunds --sample=0.25
```

Create a rule and catch it up at once on the last 500 finished runs.

```sh
kindgi judging-rules create --name="all refunds" --agent=acme.refunds --queue-last=500
```

**See also**

- [Correct a run's output](https://docs.kindgi.com/v0.3.1/guides/evals/judge-a-runs-output.md)
- [Corrections reference](https://docs.kindgi.com/v0.3.1/guides/evals/judging-reference.md)
- [`kindgi judging-rules`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules)

### `kindgi judging-rules list`

**Synopsis**

```sh signature
kindgi judging-rules list [--project=<id>] [--limit=<n>] [--cursor=<c>]
```

List the project’s live judging rules (their latest versions), oldest first.

**Options**

| Option              | Default                      | Description                                                           |
| ------------------- | ---------------------------- | --------------------------------------------------------------------- |
| `--cursor <value>`  | none                         | Resume after this cursor, from the previous page's `nextCursor`.      |
| `--limit <value>`   | `25` (at most `100`)         | The most rules to return.                                             |
| `--project <value>` | the tenant's Default project | The project the rule is in. Without it, the tenant's Default project. |

Every command also takes the [global flags](https://docs.kindgi.com/v0.3.1/reference/cli.md#global-flags).

**Examples**

List a project's judging rules.

```sh
kindgi judging-rules list --project=<project-id>
```

**See also**

- [Correct a run's output](https://docs.kindgi.com/v0.3.1/guides/evals/judge-a-runs-output.md)
- [Corrections reference](https://docs.kindgi.com/v0.3.1/guides/evals/judging-reference.md)
- [`kindgi judging-rules`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules)

### `kindgi judging-rules show`

**Synopsis**

```sh signature
kindgi judging-rules show <rule-id> [--project=<id>]
```

Show a judging rule’s latest version.

**Options**

| Option              | Default                      | Description                                                           |
| ------------------- | ---------------------------- | --------------------------------------------------------------------- |
| `--project <value>` | the tenant's Default project | The project the rule is in. Without it, the tenant's Default project. |

Every command also takes the [global flags](https://docs.kindgi.com/v0.3.1/reference/cli.md#global-flags).

**Examples**

Show a rule's latest version.

```sh
kindgi judging-rules show <rule-id>
```

**See also**

- [Correct a run's output](https://docs.kindgi.com/v0.3.1/guides/evals/judge-a-runs-output.md)
- [Corrections reference](https://docs.kindgi.com/v0.3.1/guides/evals/judging-reference.md)
- [`kindgi judging-rules`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules)

### `kindgi judging-rules update`

**Synopsis**

```sh signature
kindgi judging-rules update <rule-id> [--project=<id>] [--name=<name>] [--agent=<id>] [--flow=<id>] [--agent-version=<v>] [--status=completed] [--include-dry-runs] [--sample=<0-1>] [--max-open=<n> | --clear-max-open] [--class=<id> | --clear-class] [--enable | --disable]
```

Change a judging rule: a new version with the given fields replaced. The match flags replace the rule’s `when` whole.

**Options**

| Option                    | Default                                          | Description                                                                                                                                           |
| ------------------------- | ------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--agent <value>`         | unchanged (any match flag replaces `when` whole) | Match runs of this agent (repeatable). Repeatable.                                                                                                    |
| `--agent-version <value>` | unchanged (any match flag replaces `when` whole) | Match runs of this agent version exactly, or `live` for runs that got the agent's live version (repeatable). Repeatable.                              |
| `--class <value>`         | unchanged                                        | A new judge class whose correction closes an item.                                                                                                    |
| `--clear-class`           | off                                              | Remove the judge class; any correction closes an item.                                                                                                |
| `--clear-max-open`        | off                                              | Remove the open-items cap.                                                                                                                            |
| `--disable`               | off                                              | Disable the rule.                                                                                                                                     |
| `--enable`                | off                                              | Enable the rule.                                                                                                                                      |
| `--flow <value>`          | unchanged (any match flag replaces `when` whole) | Match runs of this flow (repeatable). Repeatable.                                                                                                     |
| `--include-dry-runs`      | off                                              | Also match dry runs; off by default. Replays never match.                                                                                             |
| `--max-open <value>`      | unchanged                                        | A new open-items cap.                                                                                                                                 |
| `--name <value>`          | unchanged                                        | A new name for the rule.                                                                                                                              |
| `--project <value>`       | the tenant's Default project                     | The project the rule is in. Without it, the tenant's Default project.                                                                                 |
| `--sample <value>`        | unchanged                                        | A new sample share, greater than 0 and at most 1.                                                                                                     |
| `--status <value>`        | unchanged (any match flag replaces `when` whole) | Match runs by how they ended. Only `completed` is accepted — a correction needs a completed run, so `failed` and `cancelled` are refused. Repeatable. |

Every command also takes the [global flags](https://docs.kindgi.com/v0.3.1/reference/cli.md#global-flags).

**Examples**

Raise a rule's open-items cap and enable it.

```sh
kindgi judging-rules update <rule-id> --max-open=50 --enable
```

Replace what the rule matches and drop its cap.

```sh
kindgi judging-rules update <rule-id> --agent=acme.refunds --clear-max-open
```

**See also**

- [Correct a run's output](https://docs.kindgi.com/v0.3.1/guides/evals/judge-a-runs-output.md)
- [Corrections reference](https://docs.kindgi.com/v0.3.1/guides/evals/judging-reference.md)
- [`kindgi judging-rules`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules)

### `kindgi judging-rules remove`

**Synopsis**

```sh signature
kindgi judging-rules remove <rule-id> [--project=<id>]
```

Unregister a judging rule, so it queues nothing more. Its queued items stay.

**Options**

| Option              | Default                      | Description                                                           |
| ------------------- | ---------------------------- | --------------------------------------------------------------------- |
| `--project <value>` | the tenant's Default project | The project the rule is in. Without it, the tenant's Default project. |

Every command also takes the [global flags](https://docs.kindgi.com/v0.3.1/reference/cli.md#global-flags).

**Examples**

Unregister a rule so it queues nothing more; its queued items stay.

```sh
kindgi judging-rules remove <rule-id>
```

**See also**

- [Correct a run's output](https://docs.kindgi.com/v0.3.1/guides/evals/judge-a-runs-output.md)
- [Corrections reference](https://docs.kindgi.com/v0.3.1/guides/evals/judging-reference.md)
- [`kindgi judging-rules`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules)

### `kindgi judging-rules versions`

**Synopsis**

```sh signature
kindgi judging-rules versions <rule-id> [--project=<id>] [--limit=<n>] [--cursor=<c>]
```

List every version of a judging rule, newest first.

**Options**

| Option              | Default                      | Description                                                           |
| ------------------- | ---------------------------- | --------------------------------------------------------------------- |
| `--cursor <value>`  | none                         | Resume after this cursor, from the previous page's `nextCursor`.      |
| `--limit <value>`   | `25` (at most `100`)         | The most versions to return.                                          |
| `--project <value>` | the tenant's Default project | The project the rule is in. Without it, the tenant's Default project. |

Every command also takes the [global flags](https://docs.kindgi.com/v0.3.1/reference/cli.md#global-flags).

**Examples**

List every version of a rule, newest first.

```sh
kindgi judging-rules versions <rule-id>
```

**See also**

- [Correct a run's output](https://docs.kindgi.com/v0.3.1/guides/evals/judge-a-runs-output.md)
- [Corrections reference](https://docs.kindgi.com/v0.3.1/guides/evals/judging-reference.md)
- [`kindgi judging-rules`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules)

### `kindgi judging-rules results`

**Synopsis**

```sh signature
kindgi judging-rules results <rule-id> [--project=<id>] [--since=<time>]
```

A judging rule’s results since a time, by the rule’s version and the agent’s, never pooled.

**Options**

| Option              | Default                      | Description                                                                                               |
| ------------------- | ---------------------------- | --------------------------------------------------------------------------------------------------------- |
| `--project <value>` | the tenant's Default project | The project the rule is in. Without it, the tenant's Default project.                                     |
| `--since <value>`   | none (every result)          | Only runs queued (or skipped by the cap) at or after this time (an ISO 8601 time with a zone, or a date). |

Every command also takes the [global flags](https://docs.kindgi.com/v0.3.1/reference/cli.md#global-flags).

**Examples**

Show a rule's results for runs queued since a date, by the rule's version and the agent's.

```sh
kindgi judging-rules results <rule-id> --since=2026-10-01
```

**See also**

- [Correct a run's output](https://docs.kindgi.com/v0.3.1/guides/evals/judge-a-runs-output.md)
- [Corrections reference](https://docs.kindgi.com/v0.3.1/guides/evals/judging-reference.md)
- [`kindgi judging-rules`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules)

### `kindgi judging-rules preview`

**Synopsis**

```sh signature
kindgi judging-rules preview [--project=<id>] [--agent=<id>] [--flow=<id>] [--agent-version=<v>] [--status=completed] [--include-dry-runs] [--sample=<0-1>] [--last=<n>] [--rule=<rule-id>]
```

What a rule would queue among the project’s recent runs, without queueing anything.

**Options**

| Option                    | Default                                              | Description                                                                                                                                           |
| ------------------------- | ---------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--agent <value>`         | none (runs of any agent)                             | Match runs of this agent (repeatable). Repeatable.                                                                                                    |
| `--agent-version <value>` | none (any version)                                   | Match runs of this agent version exactly, or `live` for runs that got the agent's live version (repeatable). Repeatable.                              |
| `--flow <value>`          | none (runs of any flow)                              | Match runs of this flow (repeatable). Repeatable.                                                                                                     |
| `--include-dry-runs`      | off                                                  | Also match dry runs; off by default. Replays never match.                                                                                             |
| `--last <value>`          | `100` (1 to 500)                                     | How many recent runs to look at, 1 to 500 (default 100).                                                                                              |
| `--project <value>`       | the tenant's Default project                         | The project the rule is in. Without it, the tenant's Default project.                                                                                 |
| `--rule <value>`          | none (preview the given flags, not an existing rule) | An existing rule's id: sample as it does, so the count is exactly what it would take.                                                                 |
| `--sample <value>`        | `1` (every matching run)                             | The sample share to preview with, greater than 0 and at most 1.                                                                                       |
| `--status <value>`        | `completed` (only completed runs match)              | Match runs by how they ended. Only `completed` is accepted — a correction needs a completed run, so `failed` and `cancelled` are refused. Repeatable. |

Every command also takes the [global flags](https://docs.kindgi.com/v0.3.1/reference/cli.md#global-flags).

**Examples**

See how many of the last 200 runs a rule would queue, changing nothing.

```sh
kindgi judging-rules preview --agent=acme.refunds --last=200
```

Preview exactly what an existing rule would take.

```sh
kindgi judging-rules preview --rule=<rule-id>
```

**See also**

- [Correct a run's output](https://docs.kindgi.com/v0.3.1/guides/evals/judge-a-runs-output.md)
- [Corrections reference](https://docs.kindgi.com/v0.3.1/guides/evals/judging-reference.md)
- [`kindgi judging-rules`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules)

### `kindgi judging-rules queue-recent`

**Synopsis**

```sh signature
kindgi judging-rules queue-recent <rule-id> [--project=<id>] [--last=<n>] [--since=<time>]
```

Queue the rule’s matches among recent runs, to catch it up on runs that ended before it was there. The open-items cap is not applied.

**Options**

| Option              | Default                               | Description                                                                                                |
| ------------------- | ------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| `--last <value>`    | none (give `--last` and/or `--since`) | This many of the project's most recent finished runs, 1 to 1000 (newest first).                            |
| `--project <value>` | the tenant's Default project          | The project the rule is in. Without it, the tenant's Default project.                                      |
| `--since <value>`   | none (give `--last` and/or `--since`) | Runs finished at or after this time (an ISO 8601 time with a zone, or a date). Give --last and/or --since. |

Every command also takes the [global flags](https://docs.kindgi.com/v0.3.1/reference/cli.md#global-flags).

**Examples**

Catch a rule up on the last 1,000 finished runs.

```sh
kindgi judging-rules queue-recent <rule-id> --last=1000
```

Queue its matches among runs finished on or after a date.

```sh
kindgi judging-rules queue-recent <rule-id> --since=2026-10-01
```

**See also**

- [Correct a run's output](https://docs.kindgi.com/v0.3.1/guides/evals/judge-a-runs-output.md)
- [Corrections reference](https://docs.kindgi.com/v0.3.1/guides/evals/judging-reference.md)
- [`kindgi judging-rules`](https://docs.kindgi.com/v0.3.1/reference/cli/judging-rules.md#kindgi-judging-rules)