Rescore a proposal's latest evaluation

POST

/v1/proposals/{proposalId}/rescore

const url = 'https://example.com/v1/proposals/2489E9AD-2EE2-8E00-8EC9-32D5F69181C0/rescore';
const options = {method: 'POST', headers: {Authorization: 'Bearer <token>'}};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}

curl --request POST \
  --url https://example.com/v1/proposals/2489E9AD-2EE2-8E00-8EC9-32D5F69181C0/rescore \
  --header 'Authorization: Bearer <token>'

After people judge a comparison’s new answers on its replay runs (perCase[].changes.new, runIds), a new comparison eval run scores the same replays again, counting those judgments, as POST /v1/eval-runs/{runId}/rescore does: same test set version and settings, comparison.rescoreOf, nothing replayed. It becomes the proposal’s evaluation, so the proposal is evaluating, then evaluated or not-better as the rescore says; the run rescored stays as it was. Takes no body fields. Needs publish on the agent. Allowed from evaluated, not-better, refused, superseded and expired.

Authorizations

bearerAuth

Parameters

Path Parameters

proposalId

required

string format: uuid

The proposal id (a UUID).

Header Parameters

Idempotency-Key

string

>= 1 characters

Caller-supplied idempotency key, scoped to the caller: the same key from someone else in the tenant is their own request (from 0.1.5). Retries with the same key return the original response byte-identical (per docs/API-ROUTE-CONVENTIONS.md §3.1). A retry sent while the first request still runs is answered 409 idempotency-key-in-flight with Retry-After, on a runtime whose store holds keys (from 0.1.5); retry after it, and you get the first request’s answer. An answer that carries a secret (a new API key or public run token, a session token, a generated signing secret) isn’t kept: a retry gets 409 idempotency-key-replay-withheld, with the first request’s status and when it succeeded (from 0.1.5).

Responses

202

The proposal, evaluating.

application/json

An improvement proposal: a change to one data block an agent version pins, for one live scope, taken through the same comparison, gate and promotion as any other version.

object

required

string format: uuid

agentId

required

string

fromVersion

required

The agent version the change applies to.

string

scope

required

One of: discriminator: kind

The agent’s default for the whole tenant.

object

kind

required

string

Allowed values: tenant

A segment path within a project: it covers every run whose path starts with it.

object

kind

required

string

Allowed values: segment

projectId

required

string format: uuid

path

required

Array<object>

>= 1 items <= 8 items

object

key

required

Lowercase, like an identifier: company, contact-role.

string

/^[a-z][a-z0-9_-]{0,63}$/

value

required

string

>= 1 characters <= 256 characters

tier

required

What a proposal changes: a settings block (new values) or a prompt block (a new template) the agent version pins.

string

Allowed values: settings-block prompt-block

change

required

object

blockId

required

The data block the change is to.

string

fromVersion

required

The block version the agent version pins: what’s being changed.

string

content

required

The new content: { values } for a settings block (its schema carries over), { template } for a prompt block (its parameters carry over).

object

values

object

key

additional properties

any

template

string

hypothesis

required

What the change should improve, and why.

string

evidence

object

judgmentIds

Array<string>

drafter

required

Who wrote the proposal: a person, or one of the runtime’s drafters.

object

kind

required

string

Allowed values: person settings-optimizer prompt-drafter

For a person: user:<id> (or service:<id>).

string

version

For a drafter: the drafter’s version.

string

model

For a drafter that used a model: which.

object

providerId

required

string

model

required

string

passId

For a drafter: the improvement pass that drafted it (GET /v1/improvement-passes/{passId}).

string

status

required

Where a proposal stands, from its comparison and its promotion (never stored). draft: not evaluated yet. evaluating: its comparison is queued or running. evaluated: the candidate beat the recorded outputs on the objective metric by more than the noise (the spread, with more than one repetition). not-better: it didn’t. evaluation-failed: the comparison failed or was cancelled. in-review: requested; the gate passed and an approval is open. promoted: live for the scope (promotion.liveNow says whether it still serves it). refused: the gate refused it. rejected: the reviewer rejected it. expired: the approval expired undecided. superseded: approved after the scope’s live version or policy changed. rolled-back: rolled back through the proposal. withdrawn: withdrawn.

string

Allowed values: draft evaluating evaluated not-better evaluation-failed in-review promoted refused rejected expired superseded rolled-back withdrawn

candidate

The versions evaluating the proposal published. They serve no scope until a promotion makes the agent version live.

object

agentVersion

required

The derived agent version (its derivedFrom.proposalId names the proposal).

string

blockVersion

required

The block version published from the proposal’s content.

string

pinsDigest

required

string

evaluation

The comparison the proposal was evaluated with, and what it found on the objective metric (the full summary is the eval run’s).

object

evalRunId

required

string format: uuid

suiteId

required

The test set (a judged eval suite).

string

objective

required

string

Allowed values: weightedYesShare weightedPrecisionAtK

startedAt

required

string format: date-time

runStatus

The eval run’s status. Absent when the run can’t be read.

string

Allowed values: pending running completed failed cancelled

baseline

The recorded outputs’ score; null without judged evidence.

number | null

candidate

The candidate’s score.

number | null

delta

number | null

spread

With more than one repetition: the candidate’s max − min, the noise a delta must beat.

number

cases

integer

better

Set once the comparison finished.

boolean

promotion

The promotion the proposal’s request made.

object

required

string

status

required

string

Allowed values: promoted pending-approval refused superseded rejected expired

approvalId

The approval a request in review waits on.

string

liveNow

For a promoted proposal: whether its version still serves the scope.

boolean

rolledBack

object

required

string format: date-time

promotionId

required

The rollback’s own promotion row.

string

required

string

reason

string

withdrawn

object

required

string format: date-time

required

string

reason

required

string

createdAt

required

string format: date-time

updatedAt

required

string format: date-time

Example

{
  "scope": {
    "kind": "tenant"
  },
  "tier": "settings-block",
  "drafter": {
    "kind": "person"
  },
  "status": "draft",
  "evaluation": {
    "objective": "weightedYesShare",
    "runStatus": "pending"
  },
  "promotion": {
    "status": "promoted"
  }
}

400

Malformed request body.

application/json

object

error

required

object

code

required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string

message

required

string

details

Optional, kind-specific.

object

key

additional properties

any

requestId

required

Server-assigned request id; also echoed via X-Request-Id header.

string

Example^generated

{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

401

Missing / malformed / expired / revoked bearer token.

application/json

object

error

required

object

code

required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string

message

required

string

details

Optional, kind-specific.

object

key

additional properties

any

requestId

required

Server-assigned request id; also echoed via X-Request-Id header.

string

Example^generated

{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

404

No such proposal, or its candidate version is unregistered (agent-version-not-found).

application/json

object

error

required

object

code

required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string

message

required

string

details

Optional, kind-specific.

object

key

additional properties

any

requestId

required

Server-assigned request id; also echoed via X-Request-Id header.

string

Example^generated

{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

409

proposal-invalid-state-transition: the proposal’s status doesn’t allow it. eval-run-not-rescorable: the latest evaluation isn’t a completed comparison.

application/json

object

error

required

object

code

required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string

message

required

string

details

Optional, kind-specific.

object

key

additional properties

any

requestId

required

Server-assigned request id; also echoed via X-Request-Id header.

string

Example^generated

{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

500

Server error (unmapped domain code or framework crash).

application/json

object

error

required

object

code

required

Stable machine-readable discriminant. Values match domain error codes (see docs/API-ROUTE-CONVENTIONS.md §4.3).

string

message

required

string

details

Optional, kind-specific.

object

key

additional properties

any

requestId

required

Server-assigned request id; also echoed via X-Request-Id header.

string

Example^generated

{
  "error": {
    "code": "example",
    "message": "example",
    "details": {},
    "requestId": "example"
  }
}

Rescore a proposal's latest evaluation

Authorizations

Parameters

Path Parameters

Header Parameters

Responses

202

Example

400

Examplegenerated

401

Examplegenerated

404

Examplegenerated

409

Examplegenerated

500

Examplegenerated

Example^generated

Example^generated

Example^generated

Example^generated

Example^generated