Skip to content
Kindgi

Configure a guardrail

View as Markdown

This guide gives a guardrail's check its settings, so you change a limit without touching the check's code. Then it chooses where in the agent's turn the guardrail runs.

Before you start

Section titled “Before you start”
  • The answer-length guardrail from Write a guardrail, in the pack my-pack, with kindgi dev running and Kindgi 0.2's CLI.

Declare the settings

Section titled “Declare the settings”

A check's settings don't belong in its code: a limit, a list of words or a tool's id, for example. The guardrail declares them as config, and the check runs with them. Ask your coding agent, for example: "I want to change my guardrail's limit without touching its check." Kindgi's skills in your pack tell it how (How it knows Kindgi).

For reference, the answer-length guardrail has one setting, maxChars:

// in guardrails/answer-length/index.ts
export const check = defineCheck({
  id: 'my-pack.checks.answer-length',
  kind: 'zero-llm',
  configSchema: z.object({ maxChars: z.number().int().positive().default(500) }),
  evaluate: async (config, trace) => {
    // config.maxChars: 60 here, or 500 when the declaration gives none
    // …
  },
});


export default {
  id: 'my-pack.answer-length',
  // …
  check,
  config: { maxChars: 60 },
  // …
};
Excerpt

configSchema describes the settings. config in the declaration gives their values. The check's evaluate gets those values, typed from the schema.

Change a setting

Section titled “Change a setting”

A new value takes effect without a restart. Edit the value and save. The next turn uses it. With maxChars: 300, the dev-echo answer from Write a guardrail passes:

Terminal window
pnpm exec kindgi runs start --agent=my-pack.echo-agent --input='{"userMessage":"hi"}'

--agent is the agent the guardrail is on, and --input its input.

  "status": "completed",
  …
    "violations": [],
Excerpt

The registered guardrail shows the values it runs with. Get it by its id:

Terminal window
pnpm exec kindgi guardrails get my-pack.answer-length
 
  "config": {
    "maxChars": 300
  },
Excerpt

What the check receives

Section titled “What the check receives”

The check gets config resolved by its schema:

  • Defaults are filled in. A guardrail without config runs with the schema's defaults; here, a limit of 500.
  • A value that doesn't fit is refused, and the message names the setting. In TypeScript and Python that happens before any turn runs; in Java and Scala, on the turns the guardrail checks.

Zod's .default() and a JSON Schema's default both apply. When the pack is indexed, each guardrail's config is checked against its configSchema, so kindgi dev reports the file and kindgi build refuses the pack. With config: { maxChars: -5 }:

guardrails/answer-length/index.ts: guardrail my-pack.answer-length's config doesn't fit its configSchema at /maxChars: must be > 0

A setting the schema requires, with no default, must be in config:

guardrails/answer-length/index.ts: guardrail my-pack.answer-length's config doesn't fit its configSchema: must have required property 'maxChars'

In a deployment, a guardrail registered through the API, with kindgi guardrails register, names the pack's check with its own config. A config that doesn't fit the check's configSchema is refused with 422 guardrail-config-invalid, and nothing is registered:

Error [invalid-request]: Guardrail "acme.strict-length" doesn't fit check "my-pack.checks.answer-length": config.maxChars must be > 0.
  ✗ /config/maxChars: config.maxChars must be > 0.

Under kindgi dev the indexer has refused such a config already.

The pack service also checks the config on every call, as sent, against the indexed configSchema. A registered guardrail whose config doesn't fit fails every turn it checks, whatever its action: the check doesn't run. kindgi runs start exits 1 with:

Error [run-failed]: input-validation-failed: Check "my-pack.checks.answer-length" config failed validation at /maxChars: must be > 0

To fix it, unregister the guardrail and register it again with a config that fits.

Where in the turn it runs

Section titled “Where in the turn it runs”

A guardrail judges the agent's final answer unless it says otherwise. at says where in the turn it runs:

  • 'final-answer', the default: the answer the turn would store.
  • 'before-tool': a tool call the model asked for, before it runs. It runs after the call's access check, and before any approval is asked. A call it judges doesn't start until the step's calls before it have finished, so its check sees them as calls that ran (Calls that run together).
  • 'after-tool': what a call returned, or the error it threw, before the model sees it.
  • 'model-step': a model reply that asks for tool calls, before any of them runs.

What halt, retry and the other actions do at each point is in At a tool call or a model reply.

A guardrail runs only where its check can. A check names its points with points (Check a tool call or a reply). A check that names none runs at the final answer only. The built-in checks' points are in Where each check runs.

At 'before-tool' and 'after-tool', scope.tools limits the guardrail to the tools it names, by id. A trailing * matches every id with that prefix, such as acme.crm.*. Left out, the guardrail checks every tool's calls.

This guardrail withholds any result of an acme.crm tool that contains SECRET, before the model sees it:

guardrails/no-secret-in-results/index.ts
export default {
  id: 'my-pack.no-secret-in-results',
  kind: 'zero-llm',
  check: 'forbidden-substring',
  config: { patterns: ['SECRET'] },
  at: 'after-tool',
  scope: { tools: ['acme.crm.*'] },
  action: { 'on-violation': 'retry' },
  severity: 'error',
};

When the pack is indexed, kindgi dev reports the file and kindgi build refuses the pack if:

  • at isn't one of the values above;
  • scope.tools is empty, has a * anywhere but the end, or is set at 'model-step' or 'final-answer';
  • the action is 'escalate' or 'compensate' away from the final answer;
  • the pack's own check doesn't run at at.

The answer-length check names no points, so with at: 'before-tool':

guardrails/answer-length/index.ts: guardrail my-pack.answer-length runs at 'before-tool', which isn't one of its check's points (final-answer)

Deploying the pack refuses the same, and also a built-in check at a point it doesn't run at. A deploy fails with deployment-validation-failed, each problem named with its guardrail and its path: /at, /action/on-violation or /scope/tools. Registering the guardrail through the API answers 422 guardrail-config-invalid for a built-in check that doesn't run at at, and 400 validation-failed for what a point can't do. A pack check's points aren't checked when it's registered: at a point its check doesn't run at, the guardrail is a check that can't run (check-point-unsupported) each time a turn reaches that point.

How long its check may take

Section titled “How long its check may take”

At 'before-tool', 'after-tool' and 'model-step', a check may take as long as its guardrail's budget.maxLatencyMs, such as budget: { maxLatencyMs: 2_000 } for two seconds. Past it, the agent loop stops waiting for the check and cancels it. A guardrail without one gets a limit by its kind: 10 seconds for zero-llm, 60 for llm-judge and 30 for any other. A check cancelled this way fails with check-timed-out, and the guardrail's action handles it as a check that couldn't run (When the check can't run). A pack's check already running on its service can still finish there; its result is dropped.

At the final answer, a check has no time limit. budget.maxCostUsd isn't enforced at any point.

Next

Section titled “Next”