client.projects.judging_rules

client.projects.judging_rules — the projects.judgingRules operations.

On AsyncKindgi every method is the same, awaited.

client.projects.judging_rules.list()

list(
    project_id: str | UUID,
    /,
    *,
    limit: int | None = None,
    cursor: str | None = None,
    timeout: float | None = None,
) -> JudgingRulePage

Signature

List a project's judging rules. GET /v1/projects/{projectId}/judging-rules

The live rules (their latest versions), oldest first. A rule says which of the project's runs to queue for a person's judgment when they end. Needs read on the project.

client.projects.judging_rules.create()

create(
    project_id: str | UUID,
    body: JudgingRuleSpec | Mapping[str, Any] | None = None,
    /,
    *,
    idempotency_key: str | None = None,
    timeout: float | None = None,
    **fields: Any,
) -> JudgingRule

Signature

Create a judging rule. POST /v1/projects/{projectId}/judging-rules

Version 1 of a rule. From then on, a top-level run of the project that matches when as it ends (and falls in sample, while the rule has fewer than maxOpen open items) is queued for a person's judgment; replays never are. A rule only lists runs: it never starts a model. Needs write on the project.

client.projects.judging_rules.preview()

preview(
    project_id: str | UUID,
    /,
    *,
    agent_ids: str | None = None,
    flow_ids: str | None = None,
    versions: str | None = None,
    status: str | None = None,
    include_dry_runs: bool | None = None,
    sample: float | None = None,
    last: int | None = None,
    rule_id: str | UUID | None = None,
    timeout: float | None = None,
) -> JudgingRulePreview

Signature

What a rule would have queued. GET /v1/projects/{projectId}/judging-rules/preview

Among the project's last last top-level runs (not replays), how many a rule with these fields (and its sample) would have queued. The rule editor's "would have added N of the last 100". Lists are comma-separated. Needs read on the project.

client.projects.judging_rules.get()

get(
    project_id: str | UUID,
    rule_id: str | UUID,
    /,
    *,
    timeout: float | None = None,
) -> JudgingRule

Signature

Fetch a judging rule. GET /v1/projects/{projectId}/judging-rules/{ruleId}

Its latest version. Needs read on the project.

client.projects.judging_rules.update()

update(
    project_id: str | UUID,
    rule_id: str | UUID,
    body: JudgingRulePatch | Mapping[str, Any] | None = None,
    /,
    *,
    timeout: float | None = None,
    **fields: Any,
) -> JudgingRule

Signature

Change a judging rule. PATCH /v1/projects/{projectId}/judging-rules/{ruleId}

A new version with these fields changed (when is replaced whole). Earlier versions are kept: …/versions shows who changed what, when. Needs write on the project.

client.projects.judging_rules.unregister()

unregister(
    project_id: str | UUID,
    rule_id: str | UUID,
    /,
    *,
    timeout: float | None = None,
) -> JudgingRuleUnregisterResult

Signature

Stop a judging rule. POST /v1/projects/{projectId}/judging-rules/{ruleId}/unregister

It queues nothing more; its versions and the items it queued stay. unregistered: false when it was already stopped. Needs write on the project.

client.projects.judging_rules.versions()

versions(
    project_id: str | UUID,
    rule_id: str | UUID,
    /,
    *,
    limit: int | None = None,
    cursor: str | None = None,
    timeout: float | None = None,
) -> JudgingRulePage

Signature

A judging rule's versions. GET /v1/projects/{projectId}/judging-rules/{ruleId}/versions

Every version, newest first: who wrote it, when, and what it said. Needs read on the project.

client.projects.judging_rules.results()

results(
    project_id: str | UUID,
    rule_id: str | UUID,
    /,
    *,
    since: str | None = None,
    timeout: float | None = None,
) -> JudgingRuleResults

Signature

A judging rule's results. GET /v1/projects/{projectId}/judging-rules/{ruleId}/results

By the rule's version and the agent's: the runs it queued (open, judged, dismissed, erased), the ones its maxOpen skipped, and the live judgments on them, with their class-weighted yes share and a count by class. Never pooled with another rule's, or across a rule's versions: each is a sample of its own when. Needs read on the project.

client.projects.judging_rules.queue_recent()

queue_recent(
    project_id: str | UUID,
    rule_id: str | UUID,
    body: JudgingQueueRecentInput | Mapping[str, Any] | None = None,
    /,
    *,
    timeout: float | None = None,
    **fields: Any,
) -> JudgingRecentQueued

Signature

Queue a rule's recent matches. POST /v1/projects/{projectId}/judging-rules/{ruleId}/queue-recent

Catch a rule up on runs that ended before it was there: among the project's recent runs — last of them (newest first) and/or those completed at or after since — queue the ones it matches, sampled as the rule samples, deduped per (rule, run) as run-end matching is. maxOpen isn't applied. The answer splits the matches into queued (newly added), alreadyQueued (this rule already had them) and alreadyReviewed (their run was already judged or dismissed, so they're left as they are); an erased run is no longer a match. At least one of last, since is given. Needs write on the project.