kindgi judgments
kindgi judgments
Section titled “kindgi judgments”Synopsis
kindgi judgments <command>Signature
Judge items of a run's output, yes or no (add / list / show / remove).
Commands
| Command | Description |
|---|---|
kindgi judgments add |
Judge one item of a finished run's output: yes or no, with an optional reason. |
kindgi judgments list |
List judgments, newest first (replaced and removed ones are left out). |
kindgi judgments show |
Show a judgment with the stored copies of the run's input and output, and the judged item's value. |
kindgi judgments remove |
Remove a judgment. |
See also
kindgi judgments add
Section titled “kindgi judgments add”Synopsis
kindgi judgments add --run=<run-id> --item=<key> [--pointer=<json-pointer>] [--rank=<n>] (--yes | --no) [--reason=<text>] [--class=<judge-class-id>] [--participant=<id>]Signature
Judge one item of a finished run's output: yes or no, with an optional reason. Judging the same item again as the same caller replaces the earlier judgment.
Options
| Option | Default | Description |
|---|---|---|
--class <value> |
none (the judgment is unclassified) | The judge class id to record the judgment under. Without one, the judgment is unclassified. |
--item <value> |
required | Your stable id for the judged item, e.g. a matched case id. Required. |
--no |
off (exactly one of --yes or --no is required) |
The verdict is no. Give --yes or --no. |
--participant <value> |
none | Your opaque id for the end user who judged, when you judge on their behalf. |
--pointer <value> |
none (item recorded without a pointer) | Where the item is in the run's output, as a JSON Pointer such as /matches/2. Its value is kept with the judgment. |
--rank <value> |
none | The item's position in a ranked list (0 = first). |
--reason <value> |
none | Why, recorded with the judgment. |
--run <value> |
required | The run whose output is judged. Required. |
--yes |
off (exactly one of --yes or --no is required) |
The verdict is yes. Give --yes or --no. |
Every command also takes the global flags.
Examples
From Build a test set from judgments:
pnpm exec kindgi judgments add --run=<run-id> --item=kb-101 --pointer=/output/articles/0 --rank=0 --yes --reason="Resets the password"pnpm exec kindgi judgments add --run=<run-id> --item=kb-102 --pointer=/output/articles/1 --rank=1 --no --reason="Not about signing in"See also
kindgi judgments list
Section titled “kindgi judgments list”Synopsis
kindgi judgments list [--run=<run-id>] [--agent=<id> [--agent-version=<v>]] [--flow=<id>] [--verdict=yes|no] [--class=<judge-class-id>] [--participant=<id>] [--project=<id>] [--limit=<n>] [--cursor=<c>]Signature
List judgments, newest first (replaced and removed ones are left out).
Options
| Option | Default | Description |
|---|---|---|
--agent <value> |
none | Only judgments of runs of this agent. |
--agent-version <value> |
none | Only judgments of runs of this agent version. Needs --agent. |
--class <value> |
none | Only judgments under this judge class id. |
--cursor <value> |
none | Resume after this cursor, from the previous page's nextCursor. |
--flow <value> |
none | Only judgments of runs of this flow. |
--limit <value> |
25 (at most 100) |
The most judgments to return (default 25, at most 100). |
--participant <value> |
none | Only judgments made for this end user's opaque id. |
--project <value> |
none | Only judgments in this project. |
--run <value> |
none | Only judgments of this run. |
--verdict <value> |
none | Only this verdict: yes or no. |
Every command also takes the global flags.
Examples
From Judge a run's output:
pnpm exec kindgi judgments list --run=<run-id> # newest first; replaced and removed ones are left outSee also
kindgi judgments show
Section titled “kindgi judgments show”Synopsis
kindgi judgments show <judgment-id>Signature
Show a judgment with the stored copies of the run's input and output, and the judged item's value.
Options
Every command also takes the global flags.
Examples
From Judge a run's output:
pnpm exec kindgi judgments show <judgment-id> # with copies of the run's input and output, and the item's valueSee also
kindgi judgments remove
Section titled “kindgi judgments remove”Synopsis
kindgi judgments remove <judgment-id>Signature
Remove a judgment.
Options
Every command also takes the global flags.
Examples
From Judge a run's output:
pnpm exec kindgi judgments remove <judgment-id>See also