Skip to content

Start an improvement pass

View as Markdown
POST
/v1/proposals/improve
curl --request POST \
--url https://example.com/v1/proposals/improve \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '{ "agentId": "example", "fromVersion": "example", "projectId": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0", "scope": { "kind": "tenant" }, "suiteId": "example", "tiers": [ "settings" ], "model": { "providerId": "example", "model": "example" }, "candidates": 3, "classWeights": "as-recorded", "objective": "weightedYesShare", "budget": { "maxCostUsd": 1, "maxCandidates": 1 } }'

The runtime looks for better values for the version’s tunable settings (keys its settings blocks’ schemas mark x-kindgi-tunable) on the test set, within the budget, and writes its best candidate as an improvement proposal, which waits for a reviewer when requested. It answers at once with the pass, running. The pass works in one project, as an improve schedule does: projectId, else the scope’s project, else (a tenant or org scope) the agent version’s. The test set must be one that project reaches, and the pass’s eval runs run there; its proposal’s version is made where the agent’s versions are. Checked first: the version is active and has something for the pass to work on (400 validation-failed, with details.reason): for a settings pass, a settings block with tunable keys (else no-tunable-keys), and for a prompt pass, instructions from a pinned prompt block (else no-pinned-prompt-block); the agent registry takes the versions a pass makes (409 registry-read-only; under kindgi dev, only for the whole tenant, since the pack’s files serve every scope), and the agent has a live version for the whole tenant (409 proposal-needs-pin). No pass starts in a suspended project (409 project-suspended), and a pass going when it is ends cancelled, with the reason in cancelReason. Needs publish on the agent, and write on the pass’s project when it isn’t the version’s. Without improvement passes in this runtime, 501 improve-unsupported.

Idempotency-Key
string
>= 1 characters

Caller-supplied idempotency key, scoped to the caller: the same key from someone else in the tenant is their own request (from 0.1.5). Retries with the same key return the original response byte-identical (per docs/API-ROUTE-CONVENTIONS.md §3.1). A retry sent while the first request still runs is answered 409 idempotency-key-in-flight with Retry-After, on a runtime whose store holds keys (from 0.1.5); retry after it, and you get the first request’s answer. An answer that carries a secret (a new API key or public run token, a session token, a generated signing secret) isn’t kept: a retry gets 409 idempotency-key-replay-withheld, with the first request’s status and when it succeeded (from 0.1.5). A request whose key store can’t be read before it runs is answered 503 idempotency-store-unavailable with Retry-After: nothing ran, so retry with the same key (from 0.3.0).

Media typeapplication/json

Start an improvement pass.

object
agentId
required
string
fromVersion

The version whose settings it tunes. Default: the one serving scope.

string
projectId

The project the pass works in: suiteId must be a test set it reaches, the version must reach it, and the pass’s eval runs run there. A project or segment scope must be in it. Default, as an improve schedule’s: the scope’s project, else (a tenant or org scope) the agent version’s. A project other than the version’s needs write there.

string format: uuid
scope
required
One of: discriminator: kind

The agent’s default for the whole tenant.

object
kind
required
string
Allowed values: tenant
suiteId
required

The test set (a judged eval suite). The pass splits it into a search part and a hold-out part, and proves its best candidate on the hold-out part.

string
tiers

['settings']: values for the tunable settings keys. ['prompt']: a model drafts templates for the prompt block (needs model); a template that reads or names anything the agent doesn’t have is refused, and a drafted proposal always waits for a reviewer.

Array<string>
default: settings >= 1 items <= 1 items
Allowed values: settings prompt
model

For a prompt pass: the tenant’s provider and model that drafts the templates.

object
providerId
required
string
model
required
string
candidates

For a prompt pass: how many templates it drafts.

integer
default: 3 >= 1 <= 5
classWeights

Which judgments the pass learns from. Omit it for the default: everyone’s (as-recorded) until the tenant, the project the pass reads its judgments from (its test set’s project, which can differ from where the agent is made), or the agent improved in that project has a restricted judge class, then only judgments recorded under one (restricted-only).

string
Allowed values: as-recorded restricted-only
objective
string
default: weightedYesShare
Allowed values: weightedYesShare weightedPrecisionAtK
budget

Default: $5 and 30 candidates.

object
maxCostUsd
number
> 0 <= 100
maxCandidates
integer
>= 1 <= 200

The pass, running.

Media typeapplication/json

An improvement pass: the runtime looking for better values for an agent version’s tunable settings (x-kindgi-tunable) on a test set, within a budget. Its best candidate becomes an improvement proposal.

object
id
required
string format: uuid
agentId
required
string
fromVersion
required

The version whose settings it tunes.

string
scope
required
One of: discriminator: kind

The agent’s default for the whole tenant.

object
kind
required
string
Allowed values: tenant
suiteId
required

The test set it searches and proves on.

string
tiers
required
Array<string>
Allowed values: settings prompt
objective
required
string
Allowed values: weightedYesShare weightedPrecisionAtK
classWeights

Which judgments its comparisons count. Absent from older servers: restricted-only.

string
Allowed values: as-recorded restricted-only
model

For a prompt pass: the provider and model that drafts the templates.

object
providerId
required
string
model
required
string
candidates

For a prompt pass: how many templates it drafts.

integer
>= 1 <= 5
budget
required
object
maxCostUsd
required

The most the pass’s comparisons may cost, in US dollars.

number
> 0 <= 100
maxCandidates
required

The most candidates it compares.

integer
>= 1 <= 200
requestedBy
required
string
status
required
string
Allowed values: running completed failed cancelled
candidatesEvaluated
required
integer
costUsd
required

What its comparisons have cost so far (US dollars).

string
outcome

What a finished pass found. proposed: its best candidate beat the current values on the test set’s hold-out part, so it wrote an improvement proposal (proposalId) for a reviewer to decide. nothing-found: no candidate beat them by more than the noise, or within the budget (reason; holdOut has the best candidate’s numbers when one got that far). failed: message says why.

object
kind
required
string
Allowed values: proposed nothing-found failed
proposalId
string format: uuid
reason
string
holdOut
object
baseline
required
number | null
candidate
required
number | null
delta
required
number | null
spread
number
message
string
cancelReason

Why a cancelled pass stopped when nobody cancelled it: a comparison’s eval run was cancelled for it (that run’s cancelReason, e.g. project-suspended). It wrote no proposal. Absent when it was cancelled.

string
comparisons

Its comparisons so far, each an eval run to open: reference (the version as it is, on the search part; for a model-settings change also on the hold-out part, what its proof is held to), each candidate (the block and the values it changed, on the search part), and the proof (the proposal, on the hold-out part). Absent from older servers.

Array<object>
object
evalRunId
string format: uuid
role
required
string
Allowed values: reference candidate proof
part
required
string
Allowed values: search hold-out
blockId
string
changed
object
key
additional properties
any
score
number | null
failed
string
refused

For a drafted template that was never compared: why the check refused it (what it reads or names that the agent doesn’t have, or its size).

Array<object>
object
path
required
string
message
required
string
hypothesis

For a drafted template: what the drafter meant it to change.

string
trigger

The improve schedule and fire that started it (GET /v1/schedules/{triggerId}/fires); absent for a pass a person started.

object
triggerId
required
string
fireId
required
string
createdAt
required
string format: date-time
updatedAt
required
string format: date-time
finishedAt
string format: date-time
Example
{
"scope": {
"kind": "tenant"
},
"tiers": [
"settings"
],
"objective": "weightedYesShare",
"classWeights": "as-recorded",
"status": "running",
"outcome": {
"kind": "proposed"
},
"comparisons": [
{
"role": "reference",
"part": "search"
}
]
}

Malformed request body, validation-failed (with details.reason when the version has nothing for the pass to work on: no-tunable-keys or no-pinned-prompt-block), or a project or segment scope that isn’t in projectId. Or reference-out-of-scope: a project or segment scope doesn’t reach the agent, the version doesn’t reach the pass’s project (/fromVersion), or the test set isn’t one that project reaches (details.issues).

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
"error": {
"code": "example",
"message": "example",
"details": {},
"requestId": "example"
}
}

Missing / malformed / expired / revoked bearer token.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
"error": {
"code": "example",
"message": "example",
"details": {},
"requestId": "example"
}
}

fromVersion (or the version serving the scope) is not an active version; or the body’s projectId names no project of this tenant (project-not-found).

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
"error": {
"code": "example",
"message": "example",
"details": {},
"requestId": "example"
}
}

registry-read-only (under kindgi dev, also a scope narrower than the whole tenant), proposal-needs-pin, or project-suspended: the pass’s project is suspended. Or Idempotency-Key was reused with a different body.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
"error": {
"code": "example",
"message": "example",
"details": {},
"requestId": "example"
}
}

The request body is over the API’s limit (request-too-large; details.maxBytes says how much).

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
"error": {
"code": "example",
"message": "example",
"details": {},
"requestId": "example"
}
}

Server error (unmapped domain code or framework crash).

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
"error": {
"code": "example",
"message": "example",
"details": {},
"requestId": "example"
}
}

improve-unsupported: this runtime runs no improvement passes.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
"error": {
"code": "example",
"message": "example",
"details": {},
"requestId": "example"
}
}

The Idempotency-Key store couldn’t be read before the request ran (idempotency-store-unavailable), so nothing was changed; Retry-After (whole seconds) says when to try again with the same key.

Media typeapplication/json
object
error
required
object
code
required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string
message
required
string
details

Optional, kind-specific.

object
key
additional properties
any
requestId
required

Server-assigned request id; also echoed via X-Request-Id header.

string
Examplegenerated
{
"error": {
"code": "example",
"message": "example",
"details": {},
"requestId": "example"
}
}

See also