Trace an answer and its cost

This guide reads a turn's provenance record, to see where an answer came from and what it cost.

Every agent turn leaves a provenance record: a graph of the turn, from the user's message through each model call and tool call to the answer. It names the model of each call and the version of each tool, and, beside the graph, each call's tokens and cost. Use it to answer "where did this answer come from?" and "what did it cost?".

Before you start

A runtime, such as kindgi dev, with Kindgi 0.1's CLI.
The id of an agent turn's run. kindgi runs list shows it.
KINDGI_API_URL and KINDGI_API_TOKEN set in your shell, for the HTTP requests. With kindgi dev, they're the URL and token it prints. They're also in the pack's .kindgirc.json.

Read a turn's record

Fetch it by the run's id:

pnpm exec kindgi provenance get a64ce71c-9cfc-4263-87b1-b5562f09079c

uv run kindgi provenance get a64ce71c-9cfc-4263-87b1-b5562f09079c

./kindgiw provenance get a64ce71c-9cfc-4263-87b1-b5562f09079c

./kindgiw provenance get a64ce71c-9cfc-4263-87b1-b5562f09079c

Over HTTP it's GET /v1/provenance/<run-id>:

curl "$KINDGI_API_URL/v1/provenance/a64ce71c-9cfc-4263-87b1-b5562f09079c" \
  -H "Authorization: Bearer $KINDGI_API_TOKEN"

Both answer the record:

{
  "id": "9f0f7385-5270-484d-9ee4-13716c4edb6e",
  "runId": "a64ce71c-9cfc-4263-87b1-b5562f09079c",
  …
  "dag": {
    "nodes": [
      {"id": "input:0", "kind": "input", "timestamp": "2026-10-08T15:13:29.826Z"},
      {"id": "model-call:1", "kind": "model-call", "timestamp": "2026-10-08T15:13:31.345Z", "modelVersion": "anthropic/claude-sonnet-5-5", "attributes": {"step": 1, "model": "claude-sonnet-5-5", "callId": "975d170b-d14f-4fbf-b662-85bcf26b9431", "providerId": "anthropic", "finishReason": "tool-use"}},
      {"id": "tool-call:toolu_01LiLE7Xk9BK16Hkr2tLBEq4", "kind": "tool-call", "timestamp": "2026-10-08T15:13:31.369Z", "attributes": {"toolId": "acme.echo", "toolVersion": "0.1.0", "invocationId": "toolu_01LiLE7Xk9BK16Hkr2tLBEq4", "toolVersionRange": "0.1.0"}},
      {"id": "tool-result:toolu_01LiLE7Xk9BK16Hkr2tLBEq4", "kind": "tool-result", "timestamp": "2026-10-08T15:13:31.369Z"},
      {"id": "model-call:2", "kind": "model-call", "timestamp": "2026-10-08T15:13:32.418Z", "modelVersion": "anthropic/claude-sonnet-5-5", "attributes": {"step": 2, "model": "claude-sonnet-5-5", "callId": "99929d2e-05e2-4452-853e-b6c6f0deed6e", "providerId": "anthropic", "finishReason": "stop"}},
      {"id": "model-output:2", "kind": "model-output", "timestamp": "2026-10-08T15:13:32.442Z", "modelVersion": "anthropic/claude-sonnet-5-5"}
    ],
    "edges": [
      {"from": "model-call:1", "to": "input:0", "kind": "caused-by"},
      {"from": "tool-call:toolu_01LiLE7Xk9BK16Hkr2tLBEq4", "to": "model-call:1", "kind": "invoked"},
      {"from": "tool-result:toolu_01LiLE7Xk9BK16Hkr2tLBEq4", "to": "tool-call:toolu_01LiLE7Xk9BK16Hkr2tLBEq4", "kind": "produced"},
      {"from": "model-call:2", "to": "input:0", "kind": "caused-by"},
      {"from": "model-call:2", "to": "tool-result:toolu_01LiLE7Xk9BK16Hkr2tLBEq4", "kind": "influenced-by"},
      {"from": "model-output:2", "to": "model-call:2", "kind": "produced"}
    ]
  },
  "callUsage": {
    "975d170b-d14f-4fbf-b662-85bcf26b9431": {"usage": {"promptTokens": 645, "completionTokens": 132, "cacheReadTokens": 0, "cacheWriteTokens": 0}, "costUsd": 0.0026100000000000003, "durationMs": 1460, "servedModel": "claude-sonnet-5-5"},
    "99929d2e-05e2-4452-853e-b6c6f0deed6e": {"usage": {"promptTokens": 740, "completionTokens": 42, "cacheReadTokens": 0, "cacheWriteTokens": 0}, "costUsd": 0.0019, "durationMs": 1012, "servedModel": "claude-sonnet-5-5"}
  }
}

Excerpt

Read it from the answer backwards:

The answer, model-output:2, was produced by the second model call.
That call answered the user's message: it's caused-by input:0.
It also read the echo tool's result: it's influenced-by that tool-result.
The tool call that produced the result was invoked by the first model call.

Node	What it is
`input`	The message the turn answered.
`model-call`	One call to a model: `modelVersion` is `provider/model`; `attributes` has its `step`, `model`, `providerId`, why it stopped (`finishReason`) and its `callId`.
`tool-call`, `tool-result`	A tool the model called, with the exact tool version that ran, and what it returned.
`model-output`	The answer.
`wait`, `resume`	A tool call that waited for an approval: the wait, and the decision that ended it.

Edges point from an effect to its cause: caused-by, invoked, produced, and influenced-by from a model call to each tool result it read.

callUsage holds each model call's tokens, cost in USD, duration and the model version that answered, servedModel, by the call's callId. A token count the provider reports is there, 0 included, as in cacheReadTokens: 0. One it doesn't report is absent.

A turn that waited for an approval is recorded whole, before and after the wait. The wait sits between the tool call and its result: the tool-call waited-on a wait node, which was resumed-from a resume node, and the tool-result was caused-by the resume. The resume node's actor is who decided, and its attributes have the decision, the rationale and the approval's id. A call a reviewer rejected:

{"id": "tool-hitl-gate-resume:toolu_01MAfC8Fb7e24iY12Cenoh11", "kind": "resume", "timestamp": "2026-10-05T16:26:25.504Z", "actor": "user:26fc414a-b49f-4e1f-a022-162865770178", "attributes": {"gate": "tool-call", "decision": "reject", "rationale": "Not confirmed yet", "approvalId": "12c38101-4cac-4241-8fc7-57323e4f63b9"}}

Its tool-call is there too, and its tool-result is the rejection the model read.

kindgi runs get <run-id> shows the same graph, under output.provenance, without callUsage.

From your app

Read the record in your app's code. Ask your coding agent: "I want to see which model calls and tool calls produced a run's answer, and what each model call cost." For reference, the code looks like this. Each script takes the run's id as its first argument:

import { createClient } from '@kindgi/sdk/client';
import type { RunId } from '@kindgi/sdk/types';

const kindgi = createClient(); // KINDGI_API_URL, KINDGI_API_TOKEN, or the running kindgi dev

const record = await kindgi.provenance.get(process.argv[2] as RunId);
for (const node of record.dag.nodes) {
  const { callId } = (node.attributes ?? {}) as { callId?: string };
  const call = typeof callId === 'string' ? record.callUsage?.[callId] : undefined;
  console.log(node.kind, node.modelVersion ?? '', call ? `${call.usage.promptTokens}+${call.usage.completionTokens} tokens, ${call.costUsd} USD` : '');
}

import sys

from kindgi.client import Kindgi

kindgi = Kindgi()  # KINDGI_API_URL, KINDGI_API_TOKEN, or the running kindgi dev

record = kindgi.provenance.get(sys.argv[1])
for node in record.dag.nodes:
    call_id = (node.attributes or {}).get("callId")
    call = (record.call_usage or {}).get(call_id) if call_id else None
    usage = f"{call.usage.prompt_tokens}+{call.usage.completion_tokens} tokens, {call.cost_usd} USD" if call else ""
    print(node.kind, node.model_version or "", usage)

package com.acme.app;

import com.kindgi.client.Kindgi;
import com.kindgi.client.models.ProvenanceRecord;
import java.util.Map;
import java.util.Objects;
import java.util.UUID;

public final class Trace {
  public static void main(String[] args) {
    Kindgi kindgi = Kindgi.create(); // KINDGI_API_URL, KINDGI_API_TOKEN, or the running kindgi dev

    ProvenanceRecord record = kindgi.provenance().get(UUID.fromString(args[0]));
    Map<String, ProvenanceRecord.CallUsageValue> usage =
        Objects.requireNonNullElse(record.callUsage(), Map.of());
    for (ProvenanceRecord.Dag.NodesItem node : record.dag().nodes()) {
      Object callId = node.attributes() == null ? null : node.attributes().get("callId");
      ProvenanceRecord.CallUsageValue call = callId instanceof String id ? usage.get(id) : null;
      System.out.println(node.kind() + " " + Objects.toString(node.modelVersion(), "") + " "
          + (call == null ? ""
              : call.usage().promptTokens() + "+" + call.usage().completionTokens() + " tokens, "
                  + call.costUsd() + " USD"));
    }
  }
}

package acme.app

import com.kindgi.client.Kindgi
import com.kindgi.client.models.ProvenanceRecord
import java.util.UUID
import scala.jdk.CollectionConverters._

object Trace {
  def main(args: Array[String]): Unit = {
    val kindgi = Kindgi.create() // KINDGI_API_URL, KINDGI_API_TOKEN, or the running kindgi dev

    val record = kindgi.provenance().get(UUID.fromString(args(0)))
    val usage: Map[String, ProvenanceRecord.CallUsageValue] =
      Option(record.callUsage).map(_.asScala.toMap).getOrElse(Map.empty)
    for (node <- record.dag.nodes.asScala) {
      val callId = Option(node.attributes).flatMap(a => Option(a.get("callId"))).collect { case id: String => id }
      val call = callId.flatMap(usage.get)
      val cost = call.map(c => s"${c.usage.promptTokens}+${c.usage.completionTokens} tokens, ${c.costUsd} USD").getOrElse("")
      println(s"${node.kind} ${Option(node.modelVersion).getOrElse("")} $cost")
    }
  }
}

Each prints every node, with the tokens and cost of each model call:

input
model-call anthropic/claude-sonnet-5-5 645+132 tokens, 0.0026100000000000003 USD
tool-call
tool-result
model-call anthropic/claude-sonnet-5-5 740+42 tokens, 0.0019 USD
model-output anthropic/claude-sonnet-5-5

What a turn cost

Each model call's cost, in callUsage, is its tokens times the prices its provider was registered with, as Models describes. The whole turn's total is in the run's output, under usage. Get the run. <run-id> is its id:

pnpm exec kindgi runs get <run-id>

uv run kindgi runs get <run-id>

./kindgiw runs get <run-id>

./kindgiw runs get <run-id>

    "usage": {
      "steps": 2,
      "durationMs": 4971,
      "promptTokens": 1671,
      "totalCostUsd": 0.002156,
      "completionTokens": 97
    },

Excerpt

For the cost of several runs, a flow's whole run, or one customer's month, read the cost API, as Cost per run and per customer shows.

A flow's answers

A flow run has no record of its own:

{"error":{"code":"provenance-not-found","message":"No provenance record for run \"41027166-71ec-4bd6-b516-cc47c0787da4\"","details":{"runId":"41027166-71ec-4bd6-b516-cc47c0787da4"},"requestId":"req-997f000a-22f6-470a-b52b-acdd21d6e228"}}

Each agent step is a run of its own, with its own record. The step's output in the flow's journal has that run's id. <flow-run-id> is the flow run's id:

pnpm exec kindgi runs journal <flow-run-id>

uv run kindgi runs journal <flow-run-id>

./kindgiw runs journal <flow-run-id>

./kindgiw runs journal <flow-run-id>

  {
    "sequence": 9,
    "kind": "step.completed",
    "nodeId": "respond",
    "payload": {
      "output": {
        "text": "⚠ dev-echo isn't a real model: …\n\nTool responded: …",
        "runId": "4bdae2f9-ff67-48a3-9549-49028d290cc1",
        …

Excerpt

Fetch the record for that run: kindgi provenance get 4bdae2f9-….

Find records

kindgi provenance list lists records, newest first, without their graphs. --created-after=<time> keeps the newer ones, --run=<run-id> one run's, --agent=<agent-id> one agent's turns at any version, and --project=<id> or --org=<id> one project's or one org's. --table prints a row per record:

pnpm exec kindgi provenance list --created-after=2026-10-06T17:00:00Z --limit=3 --table

uv run kindgi provenance list --created-after=2026-10-06T17:00:00Z --limit=3 --table

./kindgiw provenance list --created-after=2026-10-06T17:00:00Z --limit=3 --table

./kindgiw provenance list --created-after=2026-10-06T17:00:00Z --limit=3 --table

RUN                                   CREATED                   FLOW  SIGNED
────────────────────────────────────  ────────────────────────  ────  ──────
e4cb91b4-d0b1-499a-a282-c75b83c7693c  2026-10-06T17:12:02.090Z        no

Over HTTP it's GET /v1/provenance, filtered by runId, agentId or createdAfter:

curl "$KINDGI_API_URL/v1/provenance?createdAfter=2026-10-05T16:25:00Z&limit=3" \
  -H "Authorization: Bearer $KINDGI_API_TOKEN"

{"data":[{"id":"e87e13ae-472f-404b-8dfe-94ec738980cd","runId":"f91ff793-b029-48e8-9ce4-f9fa70bd0c40","tenantId":"9126df1e-094b-496d-9ea8-7daab93c3f4f","version":"1.0.0","createdAt":"2026-10-05T16:26:30.533Z","signed":false,"projectId":"13b7cfca-8b77-4983-858f-149c16afe56d"},…]}

Excerpt

Each record carries its run's projectId. scopeKind=project&scopeId=<id> lists one project's records, and scopeKind=org&scopeId=<id> every project's in an org. The clients call it scope. A record with no project is listed only without a scope, and one whose run doesn't name its agent isn't matched by --agent.

signed is false: the records themselves aren't signed as they're written. To hand a run's provenance to someone, export it signed by the deployment, with kindgi provenance export <run-id> > provenance.json. Check it with kindgi exports verify provenance.json, as Export signed evidence shows.

Provenance in the HTTP API has every operation.

Trace an answer and its cost

Before you start

Read a turn's record

From your app

What a turn cost

A flow's answers

Find records

Next