# kindgi test-runs > Test runs: run a test set, or try an agent version on the test cases (start / show / list / cancel / rescore) ## `kindgi test-runs` **Synopsis** ```sh signature kindgi test-runs ``` Test runs: run a test set, or try an agent version on the test cases (start / show / list / cancel / rescore). **Commands** | Command | Description | | ---------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [`kindgi test-runs start`](https://docs.kindgi.com/v0.5.0/reference/cli/test-runs.md#kindgi-test-runs-start) | Start a test run. | | [`kindgi test-runs show`](https://docs.kindgi.com/v0.5.0/reference/cli/test-runs.md#kindgi-test-runs-show) | Show a test run: its status and, once done, its summary. | | [`kindgi test-runs list`](https://docs.kindgi.com/v0.5.0/reference/cli/test-runs.md#kindgi-test-runs-list) | List test runs, newest first. | | [`kindgi test-runs cancel`](https://docs.kindgi.com/v0.5.0/reference/cli/test-runs.md#kindgi-test-runs-cancel) | Cancel a test run that is pending or running. | | [`kindgi test-runs rescore`](https://docs.kindgi.com/v0.5.0/reference/cli/test-runs.md#kindgi-test-runs-rescore) | Rescore a completed run on the test cases: a new test run that replays nothing and scores the run's replays again, with what people corrected on them since (correct a changed answer on its replay, then rescore). | **See also** - [Try a version on the test cases](https://docs.kindgi.com/v0.5.0/guides/improve/compare-an-agent-version.md) ### `kindgi test-runs start` **Synopsis** ```sh signature kindgi test-runs start --project= (--agent= [--agent-version=] | --flow= [--flow-version=] [--with=@ ...]) [--baseline=recorded|@|live] [--baseline-project=] [--baseline-segment=: ...] [--reads=recorded|live] [--repetitions=] [--k=] [--group-weights=as-recorded|experts-only] [--dry-run] [--wait] ``` Start a test run. On test cases (a `corrections` test set), the run tries a version (`--agent` with `--agent-version`, or `--flow` with `--flow-version`) against the recorded runs: each case is replayed without doing anything the past run didn't, and the result's summary has the metrics. **Options** | Option | Default | Description | | ---------------------------- | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `--agent ` | required: one of `--agent`, `--flow` | Run this agent. | | `--agent-version ` | the latest version | The agent version to run. Needs `--agent`. Default: the latest. | | `--baseline ` | `recorded` | What a run on the test cases tries the agent version against: `recorded` (the default: each case's output as it was marked right or wrong), `@` (another version, replayed the same way) or `pinned` (the version pinned in a project or segment). Only `recorded` runs today. | | `--baseline-project ` | none | With `--baseline=pinned`: the project whose pinned version to try it against. | | `--baseline-segment ` | none | With `--baseline=pinned` and `--baseline-project`: the version pinned for this segment path, one `:` step per flag, coarse to fine (e.g. `--baseline-segment=company:acme --baseline-segment=role:buyer`). Repeatable. | | `--dry-run` | off | Check the request without running anything. | | `--flow ` | required: one of `--agent`, `--flow` | Run this flow. | | `--flow-version ` | the latest version | The flow version to run. Needs `--flow`. Default: the latest. | | `--group-weights ` | `as-recorded` | Which corrections count: `as-recorded` (the default: each at its corrector group's weight) or `experts-only` (only corrections of groups restricted to experts; a release gate with `onlyExpertGroups` needs it). | | `--k ` | `10` | How many ranked items weighted precision\@k looks at (1 to 100, default 10). | | `--project ` | required | The project the run belongs to. Required. | | `--reads ` | `recorded` | Whether replayed reads use the past run's results when it has them (`recorded`, the default) or run live. | | `--repetitions ` | `1` | Run each case this many times (1 to 10, default 1); the summary shows the spread. | | `--wait` | off | Wait until the run is done (at most 30 minutes), then print it. Exits 3 when it failed or was cancelled. | | `--with ` | none | With `--flow` and `--flow-version`: run one of the flow's agents or tools at another version, as `@`, without publishing a new flow version. Repeat the flag for several. Repeatable. | Every command also takes the [global flags](https://docs.kindgi.com/v0.5.0/reference/cli.md#global-flags). **Examples** From [Try a version on the test cases](https://docs.kindgi.com/v0.5.0/guides/improve/compare-an-agent-version.md): ```sh pnpm exec kindgi test-runs start agent.acme-ops.article-picker. --project= --agent=acme-ops.article-picker --agent-version=0.2.0 --wait pnpm exec kindgi test-runs start acme.flow-picks --project= --flow=acme-ops.suggest-articles --flow-version=0.1.0 --with=acme-ops.find-articles@0.2.0 --reads=live --wait ``` **See also** - [Try a version on the test cases](https://docs.kindgi.com/v0.5.0/guides/improve/compare-an-agent-version.md) - [`kindgi test-runs`](https://docs.kindgi.com/v0.5.0/reference/cli/test-runs.md#kindgi-test-runs) ### `kindgi test-runs show` **Synopsis** ```sh signature kindgi test-runs show ``` Show a test run: its status and, once done, its summary. **Options** Every command also takes the [global flags](https://docs.kindgi.com/v0.5.0/reference/cli.md#global-flags). **Examples** Show a test run's status and, once it's done, its summary. ```sh kindgi test-runs show ``` **See also** - [Try a version on the test cases](https://docs.kindgi.com/v0.5.0/guides/improve/compare-an-agent-version.md) - [`kindgi test-runs`](https://docs.kindgi.com/v0.5.0/reference/cli/test-runs.md#kindgi-test-runs) ### `kindgi test-runs list` **Synopsis** ```sh signature kindgi test-runs list [--test-set=] [--status=pending|running|completed|failed|cancelled] [--agent=] [--limit=] [--cursor=] ``` List test runs, newest first. **Options** | Option | Default | Description | | -------------------- | -------------------- | ----------------------------------------------------------------- | | `--agent ` | none | Only runs of this agent. | | `--cursor ` | none | Resume after this cursor, from the previous page's `nextCursor`. | | `--limit ` | `25` (at most `100`) | The most to return (default 25, at most 100). | | `--status ` | none | Only this status: pending, running, completed, failed, cancelled. | | `--test-set ` | none | Only runs of this test set. | Every command also takes the [global flags](https://docs.kindgi.com/v0.5.0/reference/cli.md#global-flags). **Examples** List the newest test runs of one test set. ```sh kindgi test-runs list --test-set=acme.article-picks ``` List failed runs of an agent, ten at a time. ```sh kindgi test-runs list --agent=acme-ops.article-picker --status=failed --limit=10 ``` **See also** - [Try a version on the test cases](https://docs.kindgi.com/v0.5.0/guides/improve/compare-an-agent-version.md) - [`kindgi test-runs`](https://docs.kindgi.com/v0.5.0/reference/cli/test-runs.md#kindgi-test-runs) ### `kindgi test-runs cancel` **Synopsis** ```sh signature kindgi test-runs cancel ``` Cancel a test run that is pending or running. **Options** Every command also takes the [global flags](https://docs.kindgi.com/v0.5.0/reference/cli.md#global-flags). **Examples** Cancel a pending or running test run. ```sh kindgi test-runs cancel ``` **See also** - [Try a version on the test cases](https://docs.kindgi.com/v0.5.0/guides/improve/compare-an-agent-version.md) - [`kindgi test-runs`](https://docs.kindgi.com/v0.5.0/reference/cli/test-runs.md#kindgi-test-runs) ### `kindgi test-runs rescore` **Synopsis** ```sh signature kindgi test-runs rescore [--project=] [--wait] ``` Rescore a completed run on the test cases: a new test run that replays nothing and scores the run's replays again, with what people corrected on them since (correct a changed answer on its replay, then rescore). The run rescored stays as it was. **Options** | Option | Default | Description | | ------------------- | ------- | -------------------------------------------------------------------------------------- | | `--project ` | none | The run's project, only for a runtime that doesn't record it on the run. | | `--wait` | off | Wait until the rescore finishes, and show it. Exits 3 when it failed or was cancelled. | Every command also takes the [global flags](https://docs.kindgi.com/v0.5.0/reference/cli.md#global-flags). **Examples** Rescore a completed run on the test cases and wait for the new run. ```sh kindgi test-runs rescore --wait ``` Rescore against a runtime that does not record the project on the run. ```sh kindgi test-runs rescore --project= ``` **See also** - [Try a version on the test cases](https://docs.kindgi.com/v0.5.0/guides/improve/compare-an-agent-version.md) - [`kindgi test-runs`](https://docs.kindgi.com/v0.5.0/reference/cli/test-runs.md#kindgi-test-runs)