An improvement pass completed, failed or was cancelled

POST

Parameters

Header Parameters

webhook-id

required

string

The event id; the same on every retry. Deduplicate on it.

webhook-timestamp

required

string

/^[0-9]+$/

Unix seconds when the request was signed. Reject requests far from your clock.

webhook-signature

required

string

v1,<base64 HMAC-SHA256 of "{webhook-id}.{webhook-timestamp}.{body}"> under the endpoint’s secret; space-separated when two secrets sign during a rotation.

Request Body^required

Media typeapplication/json

An improvement pass ended (completed, failed or cancelled): one a person started, or one an improve schedule did. Its outcome names the proposal it wrote, if it wrote one.

object

id

required

Event id, also sent as the webhook-id header; the same on every retry.

string

type

required

string

Allowed value: improvement-pass.finished

createdAt

required

string format: date-time

data

required

object

pass

required

The pass, as GET /v1/improvement-passes/{passId} shows it.

object

id

required

string format: uuid

agentId

required

string

fromVersion

required

The version whose settings it tunes.

string

scope

required

One of: discriminator: kind

object

The agent’s default for the whole tenant.

object

kind

required

string

Allowed values: tenant

object

An org’s projects.

object

kind

required

string

Allowed values: org

orgId

required

string format: uuid

object

One project.

object

kind

required

string

Allowed values: project

projectId

required

string format: uuid

object

A segment path within a project: it covers every run whose path starts with it.

object

kind

required

string

Allowed values: segment

projectId

required

string format: uuid

path

required

Array<object>

>= 1 items <= 8 items

object

key

required

Lowercase, like an identifier: company, contact-role.

string

/^[a-z][a-z0-9_-]{0,63}$/

value

required

string

>= 1 characters <= 256 characters

suiteId

required

The test set it searches and proves on.

string

tiers

required

Array<string>

Allowed values: settings prompt

objective

required

string

Allowed values: weightedYesShare weightedPrecisionAtK

classWeights

Which judgments its comparisons count. Absent from older servers: restricted-only.

string

Allowed values: as-recorded restricted-only

model

For a prompt pass: the provider and model that drafts the templates.

object

providerId

required

string

model

required

string

candidates

For a prompt pass: how many templates it drafts.

integer

>= 1 <= 5

budget

required

object

maxCostUsd

required

The most the pass’s comparisons may cost, in US dollars.

number

> 0 <= 100

maxCandidates

required

The most candidates it compares.

integer

>= 1 <= 200

requestedBy

required

string

status

required

string

Allowed values: running completed failed cancelled

candidatesEvaluated

required

integer

costUsd

required

What its comparisons have cost so far (US dollars).

string

outcome

What a finished pass found. proposed: its best candidate beat the current values on the test set’s hold-out part, so it wrote an improvement proposal (proposalId) for a reviewer to decide. nothing-found: no candidate beat them by more than the noise, or within the budget (reason; holdOut has the best candidate’s numbers when one got that far). failed: message says why.

object

kind

required

string

Allowed values: proposed nothing-found failed

proposalId

string format: uuid

reason

string

holdOut

object

baseline

required

number | null

candidate

required

number | null

delta

required

number | null

spread

number

message

string

cancelReason

Why a cancelled pass stopped when nobody cancelled it: a comparison’s eval run was cancelled for it (that run’s cancelReason, e.g. project-suspended). It wrote no proposal. Absent when it was cancelled.

string

comparisons

Its comparisons so far, each an eval run to open: reference (the version as it is, on the search part; for a model-settings change also on the hold-out part, what its proof is held to), each candidate (the block and the values it changed, on the search part), and the proof (the proposal, on the hold-out part). Absent from older servers.

Array<object>

object

evalRunId

string format: uuid

role

required

string

Allowed values: reference candidate proof

part

required

string

Allowed values: search hold-out

blockId

string

changed

object

key

additional properties

any

score

number | null

failed

string

refused

For a drafted template that was never compared: why the check refused it (what it reads or names that the agent doesn’t have, or its size).

Array<object>

object

path

required

string

message

required

string

hypothesis

For a drafted template: what the drafter meant it to change.

string

trigger

The improve schedule and fire that started it (GET /v1/schedules/{triggerId}/fires); absent for a pass a person started.

object

triggerId

required

string

fireId

required

string

createdAt

required

string format: date-time

updatedAt

required

string format: date-time

finishedAt

string format: date-time

Responses

2XX

Received. Any other answer, or none within 10 seconds, is retried.