Skip to content
Kindgi

Ask before a long conversation continues

View as Markdown

This guide makes each turn of a long conversation wait for a reviewer, once the conversation has run a number of turns.

Some conversations should get a person's attention once they run long: a support chat that hasn't been resolved in a few turns, say. hitl.afterTurns lets the first turns of a conversation run, then makes each turn after them wait for a reviewer before it starts.

Before you start

Section titled “Before you start”
  • An agent, in a pack on a runtime such as kindgi dev, with Kindgi 0.1's CLI. This guide uses acme-ops.echo-agent, from a sample pack named acme-ops (kindgi init acme-ops --template=sample).
  • A reviewer's role. Under kindgi dev, you're an admin reviewer. Decide an approval shows how to become one elsewhere.

Set the limit

Section titled “Set the limit”

Give the agent's conversation policy a limit. Ask your coding agent: "Once a conversation runs long, make my agent wait for a person before each turn." For reference, it adds a limit like this:

// in agents/echo-agent/index.ts
  conversationPolicy: { historyLimit: 10, hitl: { afterTurns: 2 } },
Excerpt

afterTurns: 2 lets two turns run as usual. From the third on, each turn of the same conversation waits for an approval before it runs.

Try it

Section titled “Try it”

Start a conversation, then continue it twice. --agent is the agent, and --input its input. <conversation-id> is the conversationId the first turn returns, in its output:

Terminal window
pnpm exec kindgi runs start --agent=acme-ops.echo-agent --input='{"userMessage":"My order is late."}'
pnpm exec kindgi runs start --agent=acme-ops.echo-agent --input='{"userMessage":"It was due Monday.","conversationId":"<conversation-id>"}'
pnpm exec kindgi runs start --agent=acme-ops.echo-agent --input='{"userMessage":"Can you refund it?","conversationId":"<conversation-id>"}'

The first two runs complete. The third is suspended:

{
  "id": "0c424165-fc78-4563-965f-5ca9ba0ec4a9",
  …
  "status": "suspended",
  …
}
Excerpt

Its approval says why. List the pending approvals. --status=pending lists only the ones still pending:

Terminal window
pnpm exec kindgi approvals list --status=pending
 
{
  "data": [
    {
      "id": "ff98fdd3-5d1d-4245-bdab-2ab9971f3962",
      …
      "subjectKind": "agent-turn:session-hitl-gate",
      "subjectRef": {
        "agentId": "acme-ops.echo-agent",
        "threshold": 2,
        "turnCount": 2,
        "agentVersion": "0.1.0",
        "conversationId": "5d72d0da-188f-4ea0-b923-766be818891e"
      },
      "requiredRole": "standard",
      "status": "pending",
      "title": "HITL review required for Echo Agent",
      "description": "Conversation reached HITL threshold: 2 turns >= 2",
      "provenanceRef": {
        "runId": "0c424165-fc78-4563-965f-5ca9ba0ec4a9"
      },
      …
    }
  ],
  "hasMore": false
}
Excerpt

The turn hasn't run yet: the model hasn't been called. kindgi runs resume <run-id> names the approval the run waits for, with the command that decides it, as What a run waits for shows.

Approve or reject

Section titled “Approve or reject”

Decide the approval to let the turn run or stop it. <approval-id> is the approval's id from the list:

Terminal window
pnpm exec kindgi approvals complete <approval-id> --decision=approve

It prints the recorded decision as JSON: the approval, the decision, and what it did to the run. The turn runs, and it's the conversation's third: the run's output has "turnNumber": 3. The next turn asks again: every turn past the limit needs its own approval.

Reject, and the run fails without calling the model. --rationale is kept with the decision. Then get the run by its id:

Terminal window
pnpm exec kindgi approvals complete ff98fdd3-5d1d-4245-bdab-2ab9971f3962 --decision=reject --rationale="Hand this one to a person"
pnpm exec kindgi runs get 0c424165-fc78-4563-965f-5ca9ba0ec4a9
 
{
  "id": "0c424165-fc78-4563-965f-5ca9ba0ec4a9",
  …
  "status": "failed",
  …
  "failureMessage": "{\"__agent_turn_failure__\":true,\"error\":{\"code\":\"hitl-rejected\",\"message\":\"Reviewer rejected the session-HITL gate: Hand this one to a person\",\"rationale\":\"Hand this one to a person\"}}"
}
Excerpt

A rejected turn doesn't count: the conversation still has two turns, and the next message asks again.

You need to be a reviewer to see and decide these, as Decide an approval shows.

Next

Section titled “Next”