Skip to content

Models: J

View as Markdown
  • sequence: int
  • kind: str
  • node_id: str | None (nodeId on the wire) (optional)
  • payload: Any | None (optional)
  • timestamp: AwareDatetime | str
  • root: str
  • id: str
  • tenant_id: str (tenantId on the wire)
  • scope: kindgi.client.JudgeClassScope1 | kindgi.client.JudgeClassScope2 | kindgi.client.JudgeClassScope3
  • name: str
  • weight: float
  • description: str | None (optional)
  • assertable_by: kindgi.client.JudgeClassAssertableByView | None (assertableBy on the wire) (optional)
  • created_at: AwareDatetime (createdAt on the wire)
  • updated_at: AwareDatetime (updatedAt on the wire)
  • unregistered_at: AwareDatetime | None (unregisteredAt on the wire) (optional)

Who may assert the class: every part that is set must hold. A class with one is restricted: a judgment recorded while it is carries restricted, and a comparison weighted restricted-only counts only those.

  • min_reviewer_role: Optional[Literal['standard', 'senior', 'admin']] (minReviewerRole on the wire) (optional)
  • principal_kinds: list[Literal['user', 'service']] | None (principalKinds on the wire) (optional)
  • principal_ids: list[kindgi.client.PrincipalId] | None (principalIds on the wire) (optional)

Who may assert the class, as the caller sees it: every part that is set must hold. principalIds is sent only to an admin on the class's scope; principalCount to everyone who reads it.

  • min_reviewer_role: Optional[Literal['standard', 'senior', 'admin']] (minReviewerRole on the wire) (optional)
  • principal_kinds: list[Literal['user', 'service']] | None (principalKinds on the wire) (optional)
  • principal_ids: list[kindgi.client.PrincipalId] | None (principalIds on the wire) (optional)
  • principal_count: int | None (principalCount on the wire) (optional)
  • data: list[kindgi.client.JudgeClass]
  • next_cursor: str | None (nextCursor on the wire) (optional)
  • has_more: bool (hasMore on the wire)
  • root: str

Where a judge class applies: the whole tenant, one project, or one agent in a project.

  • kind: Literal['tenant']

Where a judge class applies: the whole tenant, one project, or one agent in a project.

  • kind: Literal['project']
  • project_id: str (projectId on the wire)

Where a judge class applies: the whole tenant, one project, or one agent in a project.

  • kind: Literal['agent']
  • project_id: str (projectId on the wire)
  • agent_id: str (agentId on the wire)

What people said about this item on the replay itself, once they judged it there.

  • yes_weight: float (yesWeight on the wire)
  • total_weight: float (totalWeight on the wire)

A comparison's result (a judged eval run's): the summary and each case. EvalRun.result stays an open object, since each kind has its own; the clients read it as this (TS comparisonOf(run), Python comparison_of(run)).

  • summary: kindgi.client.JudgedComparisonSummary
  • per_case: list[kindgi.client.ComparisonCaseResult] (perCase on the wire)

What a comparison concluded: the candidate beside the recorded runs, the case counts, and the metrics. What a promotion gate reads.

  • eval_run_id: str (evalRunId on the wire)
  • status: Literal['completed', 'partial', 'failed']
  • completed_at: AwareDatetime (completedAt on the wire)
  • suite: kindgi.client.Suite1
  • candidate: kindgi.client.ComparisonCandidate1 | kindgi.client.ComparisonCandidate2
  • baseline: kindgi.client.Baseline | kindgi.client.Baseline1
  • scope: kindgi.client.Scope
  • cases: int
  • diverged: int
  • refused_writes: int (refusedWrites on the wire)
  • errors: int
  • erased: int | None (optional)
  • rescore_of: str | None (rescoreOf on the wire) (optional)
  • not_rescored: int | None (notRescored on the wire) (optional)
  • stopped: int
  • reads: Literal['recorded', 'live']
  • class_weights: Optional[Literal['as-recorded', 'restricted-only']] (classWeights on the wire) (optional)
  • sample: kindgi.client.EvalSample | None (optional)
  • sampling: kindgi.client.Sampling
  • repetitions: int
  • metrics: kindgi.client.Metrics

One case of a judged eval suite: a copy of a judged run with its items' judgments summed up.

  • case_id: str (caseId on the wire)
  • subject: kindgi.client.JudgedSubject
  • input: Any
  • context: kindgi.client.JudgedRunContext | None (optional)
  • output: Any
  • items: list[kindgi.client.JudgedItemSummary]
  • erased: Optional[Literal[True]] (optional)
  • data: list[kindgi.client.JudgedEvalCase]
  • next_cursor: str | None (nextCursor on the wire) (optional)
  • has_more: bool (hasMore on the wire)

The judged item of a run's output.

  • key: str
  • pointer: str | None (optional)
  • rank: int | None (optional)

The judgments of one item of a case's output, summed up.

  • key: str
  • pointer: str | None (optional)
  • rank: int | None (optional)
  • yes: int
  • no: int
  • yes_weight: float (yesWeight on the wire)
  • total_weight: float (totalWeight on the wire)
  • restricted: kindgi.client.Restricted | None (optional)
  • reasons: list[kindgi.client.Reason]

What a judged run needs besides its input to be replayed, captured when it was first judged. For an agent turn: the conversation before it, what its retrievals returned, and the decision at its session approval gate. For a flow run: its tool calls with their results. For both: the env values its tools were sent.

  • history: list[Any] | None (optional)
  • history_truncated: bool | None (historyTruncated on the wire) (optional)
  • retrieved: Any | None (optional)
  • recalled: Any | None (optional)
  • session_approval: kindgi.client.SessionApproval | None (sessionApproval on the wire) (optional)
  • flow: kindgi.client.Flow | None (optional)
  • tool_env: dict[str, dict[str, str]] | None (toolEnv on the wire) (optional)
  • replay_of: str | None (replayOf on the wire) (optional)

The stored copy of a judged run's input and output, taken when it was first judged.

  • run_id: str (runId on the wire)
  • subject: kindgi.client.JudgedSubject
  • input: Any
  • context: kindgi.client.JudgedRunContext | None (optional)
  • output: Any
  • segments: list[kindgi.client.ScopeSegment] | None (optional)
  • captured_at: AwareDatetime (capturedAt on the wire)

What a judged run ran: an agent at a version, or a flow at a version.

  • kind: Literal['agent', 'flow']
  • id: str
  • version: str
  • judge_class_id: str | None (judgeClassId on the wire)
  • count: int
  • judge_class_id: str | None (judgeClassId on the wire)
  • judgments: int
  • yes: int
  • reason: str | None (optional)

What the caller may do with the item, by the check the routes make (write on the project, as judging the run needs).

  • dismiss: bool
  • reopen: bool
  • rule_id: str (ruleId on the wire)
  • version: int

The live judgments on the run so far, by class.

  • total: int
  • by_class: list[kindgi.client.JudgingClassCount] (byClass on the wire)

A queued run: never its content, only what it was and where it stands.

  • run_id: str (runId on the wire)
  • project_id: str (projectId on the wire)
  • agent_id: str | None (agentId on the wire) (optional)
  • agent_version: str | None (agentVersion on the wire) (optional)
  • flow_id: str (flowId on the wire)
  • run_status: Literal['completed', 'failed', 'cancelled'] (runStatus on the wire)
  • completed_at: AwareDatetime (completedAt on the wire)
  • rules: list[kindgi.client.JudgingItemRule]
  • wanted_class_ids: list[str] (wantedClassIds on the wire)
  • any_judgment: bool (anyJudgment on the wire)
  • progress: kindgi.client.JudgingProgress
  • added_at: AwareDatetime (addedAt on the wire)
  • state: Literal['open', 'judged', 'dismissed', 'erased']
  • closed_at: AwareDatetime | None (closedAt on the wire) (optional)
  • closed_by: str | None (closedBy on the wire) (optional)
  • reason: str | None (optional)
  • can: kindgi.client.JudgingItemCan
  • data: list[kindgi.client.JudgingQueueItem]
  • has_more: bool (hasMore on the wire)
  • next_cursor: str | None (nextCursor on the wire) (optional)
  • total: int

One rule version's runs of one agent version. added = open + judged + dismissed + erased.

  • rule_version: int (ruleVersion on the wire)
  • agent_version: str | None (agentVersion on the wire)
  • added: int
  • open: int
  • judged: int
  • dismissed: int
  • erased: int
  • skipped_by_cap: int (skippedByCap on the wire)
  • judgments: int
  • runs_with_judgments: int | None (runsWithJudgments on the wire) (optional)
  • yes_share: float | None (yesShare on the wire)
  • by_class: list[kindgi.client.JudgingClassResult] (byClass on the wire)

One version of a judging rule; the latest live one applies.

  • rule_id: str (ruleId on the wire)
  • project_id: str (projectId on the wire)
  • version: int
  • name: str
  • when: kindgi.client.JudgingRuleWhen
  • sample: float
  • max_open: int | None (maxOpen on the wire) (optional)
  • judge_class_id: str | None (judgeClassId on the wire) (optional)
  • enabled: bool
  • created_by: str | None (createdBy on the wire) (optional)
  • created_at: AwareDatetime (createdAt on the wire)
  • unregistered_at: AwareDatetime | None (unregisteredAt on the wire) (optional)
  • data: list[kindgi.client.JudgingRule]
  • has_more: bool (hasMore on the wire)
  • next_cursor: str | None (nextCursor on the wire) (optional)

The fields to change; when is replaced whole.

  • name: str | None (optional)
  • when: kindgi.client.JudgingRuleWhen | None (optional)
  • sample: float | None (optional)
  • max_open: int | None (maxOpen on the wire) (optional)
  • judge_class_id: str | None (judgeClassId on the wire) (optional)
  • enabled: bool | None (optional)
  • considered: int
  • matched: int
  • rule_id: str (ruleId on the wire)
  • since: AwareDatetime | None (optional)
  • groups: list[kindgi.client.JudgingResultGroup]

A judging rule as written. It only lists runs: nothing here starts a model.

  • name: str
  • when: kindgi.client.JudgingRuleWhen
  • sample: float | None (optional)
  • max_open: int | None (maxOpen on the wire) (optional)
  • judge_class_id: str | None (judgeClassId on the wire) (optional)
  • enabled: bool | None (optional)
  • rule_id: str (ruleId on the wire)
  • unregistered: bool

Which of a project's runs a rule matches as they end. Every field narrows; absent fields don't. agentIds or flowIds, not both. Top-level runs only (an agent's own runs, not its turns as a flow's step); replays never match.

  • agent_ids: list[str] | None (agentIds on the wire) (optional)
  • flow_ids: list[str] | None (flowIds on the wire) (optional)
  • versions: list[str] | None (optional)
  • status: list[Literal['completed', 'failed', 'cancelled']] | None (optional)
  • include_dry_runs: bool | None (includeDryRuns on the wire) (optional)
  • id: str
  • tenant_id: str (tenantId on the wire)
  • project_id: str (projectId on the wire)
  • run_id: str (runId on the wire)
  • subject: kindgi.client.JudgedSubject
  • item: kindgi.client.JudgedItem
  • verdict: Literal['yes', 'no']
  • reason: str | None (optional)
  • judge_class_id: str | None (judgeClassId on the wire) (optional)
  • restricted: Optional[Literal[True]] (optional)
  • asserted_by: kindgi.client.JudgmentAssertedBy (assertedBy on the wire)
  • participant_id: str | None (participantId on the wire) (optional)
  • created_at: AwareDatetime (createdAt on the wire)
  • unregistered_at: AwareDatetime | None (unregisteredAt on the wire) (optional)
  • superseded_by: str | None (supersededBy on the wire) (optional)

Who asserted a judgment: the authenticated caller, never a typed name.

  • kind: Literal['user', 'service']
  • id: str
  • data: list[kindgi.client.Judgment]
  • next_cursor: str | None (nextCursor on the wire) (optional)
  • has_more: bool (hasMore on the wire)

A judgment with the stored copies of what was judged.

  • id: str
  • tenant_id: str (tenantId on the wire)
  • project_id: str (projectId on the wire)
  • run_id: str (runId on the wire)
  • subject: kindgi.client.JudgedSubject
  • item: kindgi.client.JudgedItem
  • verdict: Literal['yes', 'no']
  • reason: str | None (optional)
  • judge_class_id: str | None (judgeClassId on the wire) (optional)
  • restricted: Optional[Literal[True]] (optional)
  • asserted_by: kindgi.client.JudgmentAssertedBy (assertedBy on the wire)
  • participant_id: str | None (participantId on the wire) (optional)
  • created_at: AwareDatetime (createdAt on the wire)
  • unregistered_at: AwareDatetime | None (unregisteredAt on the wire) (optional)
  • superseded_by: str | None (supersededBy on the wire) (optional)
  • run: kindgi.client.JudgedRunCopy
  • item_value: Any | None (itemValue on the wire) (optional)