client.eval_runs

client.eval_runs — the evalRuns operations.

On AsyncKindgi every method is the same, awaited.

client.eval_runs.start()

start(
    *,
    idempotency_key: str | None = None,
    timeout: float | None = None,
    **fields: Any,
) -> StartEvalRunResult

Start an eval run against a suite. POST /v1/eval-suites/{suiteId}/runs

Dispatches an eval run against the latest version of the suite. Exactly one of agentRef / flowRef is required — the subject the suite evaluates. Dispatch is per-kind: accuracy runs the subject once per case and grades via a configured judge; other kinds return 422 dispatcher-not-registered unless the deployment registers a dispatcher for them. Idempotency-Key applies.

client.eval_runs.list()

list(
    *,
    limit: int | None = None,
    cursor: str | None = None,
    suite_id: str | None = None,
    status: Literal['pending', 'running', 'completed', 'failed', 'cancelled'] | None = None,
    agent_id: str | None = None,
    flow_id: str | None = None,
    from_: str | None = None,
    to: str | None = None,
    scope_kind: Literal['tenant', 'org', 'project'] | None = None,
    scope_id: str | None = None,
    inherit: bool | None = None,
    timeout: float | None = None,
) -> EvalRunCollectionPage

List eval runs. GET /v1/eval-runs

Cursor-paginated. Filters: ?suiteId= / ?status= / ?agentId= / ?flowId= / ?from= / ?to=. Sort order: startedAt descending, runId descending as tie-breaker.

client.eval_runs.get()

get(*, timeout: float | None = None) -> EvalRun

Fetch an eval run. GET /v1/eval-runs/{runId}

client.eval_runs.cancel()

cancel(*, timeout: float | None = None) -> EvalRun

Cancel an eval run. POST /v1/eval-runs/{runId}/cancel

Best-effort cancel. Returns 409 when the run is already terminal.

client.eval_runs.events()

events(*, last_event_id: str | None = None, timeout: float | None = None) -> None

Stream eval-run events (SSE). GET /v1/eval-runs/{runId}/events

Server-Sent Events. Per-case progress events (eval-run.case-completed / eval-run.case-failed) followed by a terminal event (eval-run.completed / eval-run.failed / eval-run.cancelled) carrying the aggregate result. Mirrors /v1/runs/:runId/stream structurally.