An Improve run completed, failed or was cancelled

POST

Parameters

Header Parameters

webhook-id

required

string

The event id; the same on every retry. Deduplicate on it.

webhook-timestamp

required

string

/^[0-9]+$/

Unix seconds when the request was signed. Reject requests far from your clock.

webhook-signature

required

string

v1,<base64 HMAC-SHA256 of "{webhook-id}.{webhook-timestamp}.{body}"> under the endpoint’s secret; space-separated when two secrets sign during a rotation.

Request Body^required

Media typeapplication/json

An Improve run ended (completed, failed or cancelled): one a person started, or one an improve schedule did. Its outcome names the suggested change it wrote, if it wrote one.

object

id

required

Event id, also sent as the webhook-id header; the same on every retry.

string

type

required

string

Allowed value: improve-run.finished

createdAt

required

string format: date-time

data

required

object

improveRun

required

The Improve run, as GET /v1/improve-runs/{improveRunId} shows it.

object

id

required

string format: uuid

agentId

required

string

projectId

The project it works in. Absent from older servers, and on a run started for a workspace or an org before Improve runs worked in a project.

string format: uuid

startsFrom

required

The version an Improve run starts from: the one the project runs, pinned there (pin) or the agent’s latest (latest).

object

version

required

string

follows

required

string

Allowed values: pin latest

testSetVersion

The version of the project’s test set it searches and proves on; absent until it’s frozen.

string

setAside

The test set’s cases set aside for the proof: cases of of. None with fewer than five. Absent from older servers.

object

cases

required

integer

of

required

integer

tiers

required

The tiers it tries, the prompt tier first.

Array<string>

Allowed values: settings prompt

objective

required

string

Allowed values: passRate weightedPrecisionAtK

groupWeights

required

Which corrections its comparisons count.

string

Allowed values: as-recorded experts-only

model

For the prompt tier: the provider and model that drafts the templates.

object

providerId

required

string

model

required

string

candidates

For the prompt tier: how many templates it drafts.

integer

>= 1 <= 5

budget

required

object

maxCostUsd

required

The most the Improve run’s comparisons may cost, in US dollars.

number

> 0 <= 100

maxCandidates

required

The most candidates it compares.

integer

>= 1 <= 200

requestedBy

required

user:<id> or service:<id>.

string

status

required

string

Allowed values: running completed failed cancelled

tried

required

Candidates tried so far.

integer

costUsd

required

What its comparisons have cost so far (US dollars).

string

outcome

One of: discriminator: kind

object

A change passed more cases than the version the run started from, and failed none that passed: the run wrote it as a suggested change for a reviewer to decide.

object

kind

required

string

Allowed value: suggested

suggestedChangeId

required

string format: uuid

object

No change the run tried did better: reason says why, and holdOut has the best change’s numbers when one got that far.

object

kind

required

string

Allowed value: nothing-better

reason

required

string

holdOut

The best change’s proof numbers, when one got that far: on every case, or on the set-aside part for a run stored before that.

object

baseline

required

number | null

candidate

required

number | null

delta

required

number | null

spread

number

object

The run reached its budget with nothing kept.

object

kind

required

string

Allowed value: cost-cap

spentUsd

required

What it spent, in US dollars.

string

object

The run failed: message says why.

object

kind

required

string

Allowed value: failed

message

required

string

takes

The ids of the corrections it took when it started. Absent from older servers.

Array<string>

cancelReason

Why a cancelled Improve run stopped when nobody cancelled it: a comparison’s test run was cancelled for it (that run’s cancelReason, e.g. project-suspended). It wrote no suggested change. Absent when it was cancelled.

string

comparisons

Its comparisons so far, each a test run to open: reference (the version as it is, on each part: what the proof is held to, case by case), each candidate (the block and the values it changed, on the search part: what picks the change to suggest), and the proof (the suggested change, on all the cases at the reference’s repetitions: what keeps it or not; an Improve run from before 0.4 ran its proof on the hold-out part). Absent from older servers.

Array<object>

object

testRunId

string format: uuid

role

required

string

Allowed values: reference candidate proof

part

required

string

Allowed values: search hold-out all

blockId

string

changed

object

key

additional properties

any

score

number | null

failed

string

refused

For a drafted template that was never compared: why the check refused it (what it reads or names that the agent doesn’t have, or its size).

Array<object>

object

path

required

string

message

required

string

why

For a drafted template: what the drafter meant it to change.

string

trigger

The improve schedule and fire that started it (GET /v1/schedules/{triggerId}/fires); absent for an Improve run a person started.

object

triggerId

required

string

fireId

required

string

createdAt

required

string format: date-time

updatedAt

required

string format: date-time

finishedAt

string format: date-time

Responses

2XX

Received. Any other answer, or none within 10 seconds, is retried.