# Add a case to an agent's test set

POST

/v1/agents/{agentId}/test-set/cases

```sh
curl --request POST \
  --url https://example.com/v1/agents/example/test-set/cases \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{ "runId": "example", "decisions": "right", "shouldCall": [ { "tool": "example", "args": {} } ], "mustNotCall": [ "example" ], "answer": "right", "shouldSay": "example", "why": "example", "items": [ { "key": "example", "pointer": "example", "rank": 1, "verdict": "right" } ] }'
```

A case by hand, from a correction’s body on one of the agent’s finished runs. It has no status, isn’t sorted, and nothing that counts corrections counts it (Improve’s trigger, the expert queue’s progress, a rule’s yes-rate). The run’s first correction or case stores a copy of its input and output. A comparison’s replay can’t be a case. Needs `publish` on the agent and `read` on the case’s project; for an agent more than one project serves, `write` on the case’s project.

## Authorizations

- **[bearerAuth](https://docs.kindgi.com/v0.4.0/reference/api.md#bearerauth)**

## Parameters

### Path Parameters

**agentId**

required

string

AgentId — dotted namespace (`acme.drafting`).

### Header Parameters

**Idempotency-Key**

string

\>= 1 characters

Caller-supplied idempotency key, scoped to the caller: the same key from someone else in the tenant is their own request (from 0.1.5). Retries with the same key return the original response byte-identical (per `docs/API-ROUTE-CONVENTIONS.md` §3.1). A retry sent while the first request still runs is answered `409 idempotency-key-in-flight` with `Retry-After`, on a runtime whose store holds keys (from 0.1.5); retry after it, and you get the first request’s answer. An answer that carries a secret (a new API key or public run token, a session token, a generated signing secret) isn’t kept: a retry gets `409 idempotency-key-replay-withheld`, with the first request’s status and when it succeeded (from 0.1.5). A request whose key store can’t be read before it runs is answered `503 idempotency-store-unavailable` with `Retry-After`: nothing ran, so retry with the same key (from 0.3.0).

## Request Bodyrequired

application/json

A correction’s body on one of the agent’s runs. At least one of `decisions`, `answer` and `items`. `decisions: "wrong"` needs an entry in `shouldCall` or `mustNotCall`; `answer: "wrong"` needs `shouldSay` or `why`.

object

**runId**

required

string

\>= 1 characters

**decisions**

string

Allowed values: right wrong

**shouldCall**

Only with `decisions: "wrong"`: the calls the agent should have made.

Array\<object>

<= 50 items

object

**tool**

required

string

\>= 1 characters <= 200 characters

**args**

Only when the inputs matter: the top-level argument fields to compare, by value. Left out, only the call is checked. At most 4000 characters in canonical JSON.

object

<= 100 properties

**mustNotCall**

Only with `decisions: "wrong"`: tools that shouldn’t have been called. A tool can’t be in both.

Array\<string>

<= 50 items

**answer**

string

Allowed values: right wrong

**shouldSay**

Only with `answer: "wrong"`: what it should have said.

string

<= 4000 characters

**why**

string

<= 4000 characters

**items**

For an answer that is a ranked list: each item, and whether it was right. Each key once.

Array\<object>

<= 100 items

object

**key**

required

The caller’s stable id for the item, such as a matched case’s id, never the item’s own text.

string

\>= 1 characters <= 200 characters

**pointer**

Where the item is in the run’s output, as a JSON Pointer (RFC 6901), e.g. `/matches/2`.

string

<= 500 characters

**rank**

The item’s position in the ranked list (0 = first).

integer

**verdict**

required

string

Allowed values: right wrong

## Responses

### 201

Case added.

application/json

object

**id**

required

A correction’s case has the correction’s id; a hand case’s starts `case-`.

string

**kind**

required

`correction`: a correction of the run. `hand`: added by hand.

string

Allowed values: correction hand

**projectId**

required

The run’s project.

string format: uuid

**runId**

required

string

**madeOn**

required

The agent version the run ran.

string

**expect**

required

One part for each part its correction gave a verdict on; at least one.

object

**decisions**

`right`: it calls the same tools the corrected run called, by name, in any order (args aren’t compared). `wrong`: it calls every `shouldCall` tool (with its `args` where given) and none of `mustNotCall`, checked exactly, without a model.

object

**verdict**

required

string

Allowed values: right wrong

**shouldCall**

Array\<object>

object

**tool**

required

string

\>= 1 characters <= 200 characters

**args**

required

The partial set of top-level argument fields to compare, by value. Fields left out are not compared; an empty object asks only that the tool was called. At most 4000 characters in canonical JSON.

object

<= 100 properties

**mustNotCall**

Array\<string>

**answer**

A judging model checks it. `right`: it says what the corrected run’s answer said. `wrong`: it does what `shouldSay` or `why` asks.

object

**verdict**

required

string

Allowed values: right wrong

**shouldSay**

string

**why**

string

**items**

A ranked answer’s items, scored as `weightedPrecisionAtK`.

Array\<object>

object

**key**

required

The caller’s stable id for the item, such as a matched case’s id, never the item’s own text.

string

\>= 1 characters

**pointer**

Where the item is in the run’s output, as a JSON Pointer (RFC 6901), e.g. `/matches/2`.

string

**rank**

The item’s position in the ranked list (0 = first).

integer

**verdict**

required

string

Allowed values: right wrong

**addedBy**

required

Who made the correction, or added the case.

object

**kind**

required

string

Allowed values: user service

**id**

required

A user id, or for a service token its token or session id.

string

**addedAt**

required

string format: date-time

**updatedAt**

required

When the case last changed: its `addedAt` until it’s retired.

string format: date-time

**retired**

Present on a retired case: it stays listed, and no test run checks it.

object

**reason**

required

string

**by**

required

Who asserted a correction: the authenticated caller, never a typed name.

object

**kind**

required

string

Allowed values: user service

**id**

required

A user id, or for a service token its token or session id.

string

**at**

required

string format: date-time

**expert**

Present (true) on an expert’s case: a correction’s case when the correction was recorded while its group was restricted, and every hand case, since adding one needs publish on the agent, or write on its project when more than one project serves the agent. A test run weighted `experts-only` grades only these. Absent from older servers.

boolean

##### Example

```json
{
  "kind": "correction",
  "expect": {
    "decisions": {
      "verdict": "right"
    },
    "answer": {
      "verdict": "right"
    },
    "items": [
      {
        "verdict": "right"
      }
    ]
  },
  "addedBy": {
    "kind": "user"
  },
  "retired": {
    "by": {
      "kind": "user"
    }
  },
  "expert": true
}
```

### 400

Malformed body; the run isn’t one of the agent’s, or is a comparison’s replay; or `item-not-found` (an item’s pointer resolves to nothing in the output).

application/json

object

**error**

required

object

**code**

required

Stable machine-readable discriminant. Values match domain error codes (see `docs/API-ROUTE-CONVENTIONS.md` §4.3).

string

**message**

required

string

**details**

Optional, kind-specific.

object

**_key_**

additional properties

any

**requestId**

required

Server-assigned request id; also echoed via `X-Request-Id` header.

string

##### Examplegenerated

```json
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}
```

### 401

Missing / malformed / expired / revoked bearer token.

application/json

object

**error**

required

object

**code**

required

Stable machine-readable discriminant. Values match domain error codes (see `docs/API-ROUTE-CONVENTIONS.md` §4.3).

string

**message**

required

string

**details**

Optional, kind-specific.

object

**_key_**

additional properties

any

**requestId**

required

Server-assigned request id; also echoed via `X-Request-Id` header.

string

##### Examplegenerated

```json
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}
```

### 404

`run-not-found`.

application/json

object

**error**

required

object

**code**

required

Stable machine-readable discriminant. Values match domain error codes (see `docs/API-ROUTE-CONVENTIONS.md` §4.3).

string

**message**

required

string

**details**

Optional, kind-specific.

object

**_key_**

additional properties

any

**requestId**

required

Server-assigned request id; also echoed via `X-Request-Id` header.

string

##### Examplegenerated

```json
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}
```

### 409

`run-not-finished`: the run has no output yet.

application/json

object

**error**

required

object

**code**

required

Stable machine-readable discriminant. Values match domain error codes (see `docs/API-ROUTE-CONVENTIONS.md` §4.3).

string

**message**

required

string

**details**

Optional, kind-specific.

object

**_key_**

additional properties

any

**requestId**

required

Server-assigned request id; also echoed via `X-Request-Id` header.

string

##### Examplegenerated

```json
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}
```

### 410

`run-erased`: an erasure cleared the run’s content.

application/json

object

**error**

required

object

**code**

required

Stable machine-readable discriminant. Values match domain error codes (see `docs/API-ROUTE-CONVENTIONS.md` §4.3).

string

**message**

required

string

**details**

Optional, kind-specific.

object

**_key_**

additional properties

any

**requestId**

required

Server-assigned request id; also echoed via `X-Request-Id` header.

string

##### Examplegenerated

```json
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}
```

### 413

The request body is over the API’s limit (`request-too-large`; `details.maxBytes` says how much).

application/json

object

**error**

required

object

**code**

required

Stable machine-readable discriminant. Values match domain error codes (see `docs/API-ROUTE-CONVENTIONS.md` §4.3).

string

**message**

required

string

**details**

Optional, kind-specific.

object

**_key_**

additional properties

any

**requestId**

required

Server-assigned request id; also echoed via `X-Request-Id` header.

string

##### Examplegenerated

```json
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}
```

### 500

Server error (unmapped domain code or framework crash).

application/json

object

**error**

required

object

**code**

required

Stable machine-readable discriminant. Values match domain error codes (see `docs/API-ROUTE-CONVENTIONS.md` §4.3).

string

**message**

required

string

**details**

Optional, kind-specific.

object

**_key_**

additional properties

any

**requestId**

required

Server-assigned request id; also echoed via `X-Request-Id` header.

string

##### Examplegenerated

```json
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}
```

### 503

The `Idempotency-Key` store couldn’t be read before the request ran (`idempotency-store-unavailable`), so nothing was changed; `Retry-After` (whole seconds) says when to try again with the same key.

application/json

object

**error**

required

object

**code**

required

Stable machine-readable discriminant. Values match domain error codes (see `docs/API-ROUTE-CONVENTIONS.md` §4.3).

string

**message**

required

string

**details**

Optional, kind-specific.

object

**_key_**

additional properties

any

**requestId**

required

Server-assigned request id; also echoed via `X-Request-Id` header.

string

##### Examplegenerated

```json
{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}
```

## See also

- [`GET /v1/agents/{agentId}/test-set/cases`: List a project's cases of an agent's test set](https://docs.kindgi.com/v0.4.0/reference/api/operations/agentstestsetlistcases.md)
- [`GET /v1/agents/{agentId}`: Fetch the highest version of an agent](https://docs.kindgi.com/v0.4.0/reference/api/operations/agentsget.md)
- [`GET /v1/agents/{agentId}/projects`: List the projects an agent runs in, and what runs in each](https://docs.kindgi.com/v0.4.0/reference/api/operations/agentsprojects.md)
- [`GET /v1/agents/{agentId}/versions`: List versions of an agent](https://docs.kindgi.com/v0.4.0/reference/api/operations/agentslistversions.md)
- [Error code `idempotency-store-unavailable`](https://docs.kindgi.com/v0.4.0/reference/errors.md#idempotency-store-unavailable)
- [Error code `item-not-found`](https://docs.kindgi.com/v0.4.0/reference/errors.md#item-not-found)
- [Error code `request-too-large`](https://docs.kindgi.com/v0.4.0/reference/errors.md#request-too-large)
- [Error code `run-erased`](https://docs.kindgi.com/v0.4.0/reference/errors.md#run-erased)
- [Error code `run-not-finished`](https://docs.kindgi.com/v0.4.0/reference/errors.md#run-not-finished)
- [Error code `run-not-found`](https://docs.kindgi.com/v0.4.0/reference/errors.md#run-not-found)
- [Every error code](https://docs.kindgi.com/v0.4.0/reference/errors.md)