# kindgi judgments > Judge items of a run's output, yes or no (add / list / show / remove) ## `kindgi judgments` **Synopsis** ```sh signature kindgi judgments ``` Judge items of a run's output, yes or no (add / list / show / remove). **Commands** | Command | Description | | -------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | | [`kindgi judgments add`](https://docs.kindgi.com/v0.1.8/reference/cli/judgments.md#kindgi-judgments-add) | Judge one item of a finished run's output: yes or no, with an optional reason. | | [`kindgi judgments list`](https://docs.kindgi.com/v0.1.8/reference/cli/judgments.md#kindgi-judgments-list) | List judgments, newest first (replaced and removed ones are left out). | | [`kindgi judgments show`](https://docs.kindgi.com/v0.1.8/reference/cli/judgments.md#kindgi-judgments-show) | Show a judgment with the stored copies of the run's input and output, and the judged item's value. | | [`kindgi judgments remove`](https://docs.kindgi.com/v0.1.8/reference/cli/judgments.md#kindgi-judgments-remove) | Remove a judgment. | **See also** - [Build a test set from judgments](https://docs.kindgi.com/v0.1.8/guides/evals/build-a-test-set.md) - [Judge a run's output](https://docs.kindgi.com/v0.1.8/guides/evals/judge-a-runs-output.md) - [Propose a change, and test it](https://docs.kindgi.com/v0.1.8/guides/evals/propose-a-change.md) ### `kindgi judgments add` **Synopsis** ```sh signature kindgi judgments add --run= --item= [--pointer=] [--rank=] (--yes | --no) [--reason=] [--class=] [--participant=] ``` Judge one item of a finished run's output: yes or no, with an optional reason. Judging the same item again as the same caller replaces the earlier judgment. **Options** | Option | Default | Description | | ----------------------- | -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | | `--class ` | none (the judgment is unclassified) | The judge class id to record the judgment under. Without one, the judgment is unclassified. | | `--item ` | required | Your stable id for the judged item, e.g. a matched case id. Required. | | `--no` | off (exactly one of `--yes` or `--no` is required) | The verdict is no. Give `--yes` or `--no`. | | `--participant ` | none | Your opaque id for the end user who judged, when you judge on their behalf. | | `--pointer ` | none (item recorded without a pointer) | Where the item is in the run's output, as a JSON Pointer such as `/matches/2`. Its value is kept with the judgment. | | `--rank ` | none | The item's position in a ranked list (0 = first). | | `--reason ` | none | Why, recorded with the judgment. | | `--run ` | required | The run whose output is judged. Required. | | `--yes` | off (exactly one of `--yes` or `--no` is required) | The verdict is yes. Give `--yes` or `--no`. | Every command also takes the [global flags](https://docs.kindgi.com/v0.1.8/reference/cli.md#global-flags). **Examples** From [Build a test set from judgments](https://docs.kindgi.com/v0.1.8/guides/evals/build-a-test-set.md): ```sh pnpm exec kindgi judgments add --run= --item=kb-101 --pointer=/output/articles/0 --rank=0 --yes --reason="Resets the password" pnpm exec kindgi judgments add --run= --item=kb-102 --pointer=/output/articles/1 --rank=1 --no --reason="Not about signing in" ``` **See also** - [Build a test set from judgments](https://docs.kindgi.com/v0.1.8/guides/evals/build-a-test-set.md) - [Judge a run's output](https://docs.kindgi.com/v0.1.8/guides/evals/judge-a-runs-output.md) - [Propose a change, and test it](https://docs.kindgi.com/v0.1.8/guides/evals/propose-a-change.md) - [`kindgi judgments`](https://docs.kindgi.com/v0.1.8/reference/cli/judgments.md#kindgi-judgments) ### `kindgi judgments list` **Synopsis** ```sh signature kindgi judgments list [--run=] [--agent= [--agent-version=]] [--flow=] [--verdict=yes|no] [--class=] [--participant=] [--project=] [--limit=] [--cursor=] ``` List judgments, newest first (replaced and removed ones are left out). **Options** | Option | Default | Description | | ------------------------- | -------------------- | ---------------------------------------------------------------- | | `--agent ` | none | Only judgments of runs of this agent. | | `--agent-version ` | none | Only judgments of runs of this agent version. Needs `--agent`. | | `--class ` | none | Only judgments under this judge class id. | | `--cursor ` | none | Resume after this cursor, from the previous page's `nextCursor`. | | `--flow ` | none | Only judgments of runs of this flow. | | `--limit ` | `25` (at most `100`) | The most judgments to return (default 25, at most 100). | | `--participant ` | none | Only judgments made for this end user's opaque id. | | `--project ` | none | Only judgments in this project. | | `--run ` | none | Only judgments of this run. | | `--verdict ` | none | Only this verdict: `yes` or `no`. | Every command also takes the [global flags](https://docs.kindgi.com/v0.1.8/reference/cli.md#global-flags). **Examples** From [Judge a run's output](https://docs.kindgi.com/v0.1.8/guides/evals/judge-a-runs-output.md): ```sh pnpm exec kindgi judgments list --run= # newest first; replaced and removed ones are left out ``` **See also** - [Judge a run's output](https://docs.kindgi.com/v0.1.8/guides/evals/judge-a-runs-output.md) - [`kindgi judgments`](https://docs.kindgi.com/v0.1.8/reference/cli/judgments.md#kindgi-judgments) ### `kindgi judgments show` **Synopsis** ```sh signature kindgi judgments show ``` Show a judgment with the stored copies of the run's input and output, and the judged item's value. **Options** Every command also takes the [global flags](https://docs.kindgi.com/v0.1.8/reference/cli.md#global-flags). **Examples** From [Judge a run's output](https://docs.kindgi.com/v0.1.8/guides/evals/judge-a-runs-output.md): ```sh pnpm exec kindgi judgments show # with copies of the run's input and output, and the item's value ``` **See also** - [Judge a run's output](https://docs.kindgi.com/v0.1.8/guides/evals/judge-a-runs-output.md) - [`kindgi judgments`](https://docs.kindgi.com/v0.1.8/reference/cli/judgments.md#kindgi-judgments) ### `kindgi judgments remove` **Synopsis** ```sh signature kindgi judgments remove ``` Remove a judgment. **Options** Every command also takes the [global flags](https://docs.kindgi.com/v0.1.8/reference/cli.md#global-flags). **Examples** From [Judge a run's output](https://docs.kindgi.com/v0.1.8/guides/evals/judge-a-runs-output.md): ```sh pnpm exec kindgi judgments remove ``` **See also** - [Judge a run's output](https://docs.kindgi.com/v0.1.8/guides/evals/judge-a-runs-output.md) - [`kindgi judgments`](https://docs.kindgi.com/v0.1.8/reference/cli/judgments.md#kindgi-judgments)