Skip to content

Models: J

View as Markdown
  • sequence: int
  • kind: str
  • node_id: str | None (nodeId on the wire) (optional)
  • payload: Any | None (optional)
  • timestamp: AwareDatetime | str
  • root: str
  • id: str
  • tenant_id: str (tenantId on the wire)
  • scope: kindgi.client.JudgeClassScopeTenant | kindgi.client.JudgeClassScopeProject | kindgi.client.JudgeClassScopeAgent | kindgi.client.JudgeClassScopeUnrecognized
  • name: str
  • weight: float
  • description: str | None (optional)
  • assertable_by: kindgi.client.JudgeClassAssertableByView | None (assertableBy on the wire) (optional)
  • created_at: AwareDatetime (createdAt on the wire)
  • updated_at: AwareDatetime (updatedAt on the wire)
  • unregistered_at: AwareDatetime | None (unregisteredAt on the wire) (optional)

Who may assert the class: every part that is set must hold. A class with one is restricted: a judgment recorded while it is carries restricted, and a comparison weighted restricted-only counts only those.

  • min_reviewer_role: Optional[Literal['standard', 'senior', 'admin']] (minReviewerRole on the wire) (optional)
  • principal_kinds: list[Literal['user', 'service']] | None (principalKinds on the wire) (optional)
  • principal_ids: list[kindgi.client.PrincipalId] | None (principalIds on the wire) (optional)

Who may assert the class, as the caller sees it: every part that is set must hold. principalIds is sent only to an admin on the class's scope; principalCount to everyone who reads it.

  • min_reviewer_role: Optional[Annotated[Union[Literal['standard', 'senior', 'admin'], str], <kindgi.client._open.OpenEnum object at 0x10700bbd0>]] (minReviewerRole on the wire) (optional)
  • principal_kinds: list[Annotated[Union[Literal['user', 'service'], str], <kindgi.client._open.OpenEnum object at 0x10700bbd0>]] | None (principalKinds on the wire) (optional)
  • principal_ids: list[kindgi.client.PrincipalId] | None (principalIds on the wire) (optional)
  • principal_count: int | None (principalCount on the wire) (optional)
  • data: list[kindgi.client.JudgeClass]
  • next_cursor: str | None (nextCursor on the wire) (optional)
  • has_more: bool (hasMore on the wire)
  • root: str

Where a judge class applies: the whole tenant, one project, or one agent in a project.

  • kind: Literal['agent']
  • project_id: str (projectId on the wire)
  • agent_id: str (agentId on the wire)

Where a judge class applies: the whole tenant, one project, or one agent in a project.

  • kind: Literal['project']
  • project_id: str (projectId on the wire)

Where a judge class applies: the whole tenant, one project, or one agent in a project.

  • kind: Literal['tenant']

A kind this client doesn't know yet, from a newer runtime: the object as it came, every field kept.

  • kind: str

What people said about this item on the replay itself, once they judged it there.

  • yes_weight: float (yesWeight on the wire)
  • total_weight: float (totalWeight on the wire)

A comparison's result (a judged eval run's): the summary and each case. EvalRun.result stays an open object, since each kind has its own; the clients read it as this (TS comparisonOf(run), Python comparison_of(run)).

  • summary: kindgi.client.JudgedComparisonSummary
  • per_case: list[kindgi.client.ComparisonCaseResult] (perCase on the wire)

What a comparison concluded: the candidate beside the recorded runs, the case counts, and the metrics. What a promotion gate reads.

  • eval_run_id: str (evalRunId on the wire)
  • status: Literal['completed', 'partial', 'failed']
  • completed_at: AwareDatetime (completedAt on the wire)
  • suite: kindgi.client.Suite1
  • candidate: kindgi.client.ComparisonCandidateAgent | kindgi.client.ComparisonCandidateFlow
  • baseline: kindgi.client.Baseline | kindgi.client.Baseline1
  • scope: kindgi.client.JudgedComparisonSummaryScope
  • cases: int
  • diverged: int
  • refused_writes: int (refusedWrites on the wire)
  • errors: int
  • erased: int | None (optional)
  • rescore_of: str | None (rescoreOf on the wire) (optional)
  • not_rescored: int | None (notRescored on the wire) (optional)
  • stopped: int
  • reads: Literal['recorded', 'live']
  • class_weights: Optional[Literal['as-recorded', 'restricted-only']] (classWeights on the wire) (optional)
  • sample: kindgi.client.EvalSample | None (optional)
  • sampling: kindgi.client.Sampling
  • repetitions: int
  • metrics: kindgi.client.Metrics

Where the test set's judgments came from.

  • project_id: str | None (projectId on the wire) (optional)

One case of a judged eval suite: a copy of a judged run with its items' judgments summed up.

  • case_id: str (caseId on the wire)
  • subject: kindgi.client.JudgedSubject
  • input: Any
  • context: kindgi.client.JudgedRunContext | None (optional)
  • output: Any
  • items: list[kindgi.client.JudgedItemSummary]
  • erased: Optional[Literal[True]] (optional)
  • data: list[kindgi.client.JudgedEvalCase]
  • next_cursor: str | None (nextCursor on the wire) (optional)
  • has_more: bool (hasMore on the wire)

The judged item of a run's output.

  • key: str
  • pointer: str | None (optional)
  • rank: int | None (optional)

The judgments of one item of a case's output, summed up.

  • key: str
  • pointer: str | None (optional)
  • rank: int | None (optional)
  • yes: int
  • no: int
  • yes_weight: float (yesWeight on the wire)
  • total_weight: float (totalWeight on the wire)
  • restricted: kindgi.client.Restricted | None (optional)
  • reasons: list[kindgi.client.Reason]

What a judged run needs besides its input to be replayed, captured when it was first judged. For an agent turn: the conversation before it, what its retrievals returned, and the decision at its session approval gate. For a flow run: its tool calls with their results. For both: the env values its tools were sent.

  • history: list[Any] | None (optional)
  • history_truncated: bool | None (historyTruncated on the wire) (optional)
  • retrieved: Any | None (optional)
  • recalled: Any | None (optional)
  • session_approval: kindgi.client.SessionApproval | None (sessionApproval on the wire) (optional)
  • flow: kindgi.client.JudgedRunContextFlow | None (optional)
  • tool_env: dict[str, dict[str, str]] | None (toolEnv on the wire) (optional)
  • replay_of: str | None (replayOf on the wire) (optional)

For a flow run: what it did, kept at its first judgment so it can be replayed. Every tool call it made with its result (at its tool nodes, in its agent steps' turns and in its sub-flows), at most 500, and its agent steps.

  • calls: list[kindgi.client.Call]
  • steps: list[kindgi.client.Step]
  • truncated: bool | None (optional)

The stored copy of a judged run's input and output, taken when it was first judged.

  • run_id: str (runId on the wire)
  • subject: kindgi.client.JudgedSubject
  • input: Any
  • context: kindgi.client.JudgedRunContext | None (optional)
  • output: Any
  • segments: list[kindgi.client.ScopeSegment] | None (optional)
  • captured_at: AwareDatetime (capturedAt on the wire)

What a judged run ran: an agent at a version, or a flow at a version.

  • kind: Union[Literal['agent', 'flow'], str]
  • id: str
  • version: str
  • judge_class_id: str | None (judgeClassId on the wire)
  • count: int
  • judge_class_id: str | None (judgeClassId on the wire)
  • judgments: int
  • yes: int
  • reason: str | None (optional)

What the caller may do with the item, by the check the routes make (write on the project, as judging the run needs).

  • dismiss: bool
  • reopen: bool
  • rule_id: str (ruleId on the wire)
  • version: int

The live judgments on the run so far, by class.

  • total: int
  • by_class: list[kindgi.client.JudgingClassCount] (byClass on the wire)

A queued run: never its content, only what it was and where it stands.

  • run_id: str (runId on the wire)
  • project_id: str (projectId on the wire)
  • agent_id: str | None (agentId on the wire) (optional)
  • agent_version: str | None (agentVersion on the wire) (optional)
  • flow_id: str (flowId on the wire)
  • run_status: Union[Literal['completed', 'failed', 'cancelled'], str] (runStatus on the wire)
  • completed_at: AwareDatetime (completedAt on the wire)
  • rules: list[kindgi.client.JudgingItemRule]
  • wanted_class_ids: list[str] (wantedClassIds on the wire)
  • any_judgment: bool (anyJudgment on the wire)
  • progress: kindgi.client.JudgingProgress
  • added_at: AwareDatetime (addedAt on the wire)
  • state: Union[Literal['open', 'judged', 'dismissed', 'erased'], str]
  • closed_at: AwareDatetime | None (closedAt on the wire) (optional)
  • closed_by: str | None (closedBy on the wire) (optional)
  • reason: str | None (optional)
  • can: kindgi.client.JudgingItemCan
  • data: list[kindgi.client.JudgingQueueItem]
  • has_more: bool (hasMore on the wire)
  • next_cursor: str | None (nextCursor on the wire) (optional)
  • total: int

One rule version's runs of one agent version. added = open + judged + dismissed + erased.

  • rule_version: int (ruleVersion on the wire)
  • agent_version: str | None (agentVersion on the wire)
  • added: int
  • open: int
  • judged: int
  • dismissed: int
  • erased: int
  • skipped_by_cap: int (skippedByCap on the wire)
  • judgments: int
  • runs_with_judgments: int | None (runsWithJudgments on the wire) (optional)
  • yes_share: float | None (yesShare on the wire)
  • by_class: list[kindgi.client.JudgingClassResult] (byClass on the wire)

One version of a judging rule; the latest live one applies.

  • rule_id: str (ruleId on the wire)
  • project_id: str (projectId on the wire)
  • version: int
  • name: str
  • when: kindgi.client.JudgingRuleWhen
  • sample: float
  • max_open: int | None (maxOpen on the wire) (optional)
  • judge_class_id: str | None (judgeClassId on the wire) (optional)
  • enabled: bool
  • created_by: str | None (createdBy on the wire) (optional)
  • created_at: AwareDatetime (createdAt on the wire)
  • unregistered_at: AwareDatetime | None (unregisteredAt on the wire) (optional)
  • data: list[kindgi.client.JudgingRule]
  • has_more: bool (hasMore on the wire)
  • next_cursor: str | None (nextCursor on the wire) (optional)

The fields to change; when is replaced whole.

  • name: str | None (optional)
  • when: kindgi.client.JudgingRuleWhen | None (optional)
  • sample: float | None (optional)
  • max_open: int | None (maxOpen on the wire) (optional)
  • judge_class_id: str | None (judgeClassId on the wire) (optional)
  • enabled: bool | None (optional)
  • considered: int
  • matched: int
  • rule_id: str (ruleId on the wire)
  • since: AwareDatetime | None (optional)
  • groups: list[kindgi.client.JudgingResultGroup]

A judging rule as written. It only lists runs: nothing here starts a model.

  • name: str
  • when: kindgi.client.JudgingRuleWhen
  • sample: float | None (optional)
  • max_open: int | None (maxOpen on the wire) (optional)
  • judge_class_id: str | None (judgeClassId on the wire) (optional)
  • enabled: bool | None (optional)
  • rule_id: str (ruleId on the wire)
  • unregistered: bool

Which of a project's runs a rule matches as they end. Every field narrows; absent fields don't. agentIds or flowIds, not both. Top-level runs only (an agent's own runs, not its turns as a flow's step); replays never match.

  • agent_ids: list[str] | None (agentIds on the wire) (optional)
  • flow_ids: list[str] | None (flowIds on the wire) (optional)
  • versions: list[str] | None (optional)
  • status: list[Annotated[Union[Literal['completed', 'failed', 'cancelled'], str], <kindgi.client._open.OpenEnum object at 0x10700bbd0>]] | None (optional)
  • include_dry_runs: bool | None (includeDryRuns on the wire) (optional)
  • id: str
  • tenant_id: str (tenantId on the wire)
  • project_id: str (projectId on the wire)
  • run_id: str (runId on the wire)
  • subject: kindgi.client.JudgedSubject
  • item: kindgi.client.JudgedItem
  • verdict: Union[Literal['yes', 'no'], str]
  • reason: str | None (optional)
  • judge_class_id: str | None (judgeClassId on the wire) (optional)
  • restricted: Optional[Literal[True]] (optional)
  • asserted_by: kindgi.client.JudgmentAssertedBy (assertedBy on the wire)
  • participant_id: str | None (participantId on the wire) (optional)
  • created_at: AwareDatetime (createdAt on the wire)
  • unregistered_at: AwareDatetime | None (unregisteredAt on the wire) (optional)
  • superseded_by: str | None (supersededBy on the wire) (optional)

Who asserted a judgment: the authenticated caller, never a typed name.

  • kind: Union[Literal['user', 'service'], str]
  • id: str
  • data: list[kindgi.client.Judgment]
  • next_cursor: str | None (nextCursor on the wire) (optional)
  • has_more: bool (hasMore on the wire)

A judgment with the stored copies of what was judged.

  • id: str
  • tenant_id: str (tenantId on the wire)
  • project_id: str (projectId on the wire)
  • run_id: str (runId on the wire)
  • subject: kindgi.client.JudgedSubject
  • item: kindgi.client.JudgedItem
  • verdict: Union[Literal['yes', 'no'], str]
  • reason: str | None (optional)
  • judge_class_id: str | None (judgeClassId on the wire) (optional)
  • restricted: Optional[Literal[True]] (optional)
  • asserted_by: kindgi.client.JudgmentAssertedBy (assertedBy on the wire)
  • participant_id: str | None (participantId on the wire) (optional)
  • created_at: AwareDatetime (createdAt on the wire)
  • unregistered_at: AwareDatetime | None (unregisteredAt on the wire) (optional)
  • superseded_by: str | None (supersededBy on the wire) (optional)
  • run: kindgi.client.JudgedRunCopy
  • item_value: Any | None (itemValue on the wire) (optional)