Skip to content
Kindgi

Write a guardrail

View as Markdown

This guide writes a guardrail that keeps an agent's answers short, puts it on an agent, and tests its check.

Before you start

Section titled “Before you start”
  • A pack with an agent, on a runtime such as kindgi dev, with Kindgi 0.1's CLI. This guide uses the quickstart's pack, my-pack, and its my-pack.echo-agent, from the TypeScript, Python, Java or Scala quickstart.

The guardrail

Section titled “The guardrail”

The guardrail fails a turn whose answer is longer than a limit. It's one file in the pack's guardrails folder. Ask your coding agent: "I want a guardrail that keeps my agent's answers short." Kindgi's skills in your pack tell it how (How it knows Kindgi). For reference, the guardrail looks like this. This guide calls it my-pack.answer-length:

guardrails/answer-length/index.ts
import { defineCheck } from '@kindgi/sdk/define';
import { z } from 'zod';


export const check = defineCheck({
  id: 'my-pack.checks.answer-length',
  kind: 'zero-llm',
  configSchema: z.object({ maxChars: z.number().int().positive().default(500) }),
  evaluate: async (config, trace) => {
    const length = trace.output?.length ?? 0;
    if (length > config.maxChars) {
      return {
        passed: false,
        reason: `The answer is ${length} characters; the limit is ${config.maxChars}.`,
      };
    }
    return { passed: true };
  },
});


export default {
  id: 'my-pack.answer-length',
  name: 'Answer is short',
  kind: 'zero-llm',
  check,
  config: { maxChars: 60 },
  action: { 'on-violation': 'halt' },
  severity: 'error',
};

The file has two parts:

  • The check, from defineCheck, exported as check. Its evaluate gets the guardrail's config and the turn's trace, and returns { passed, reason? }.
  • The declaration, the default export: the guardrail's id, the check it runs, the check's config, and the action on a failure.

  • The id is <pack-id>.<name>. Name the rule as what should hold: answer-length, response-not-empty.

  • kind: 'zero-llm': the check is your code, a function of the trace. It can't call a model.

  • The reason is what the violation reports. Say what was wrong. It shows wherever the failure does, such as the run's error and its page in the console. So describe what you blocked, never quote it: "it contains a card number", not the number.

  • The trace is the turn. RunTrace in the Python reference lists its fields. Its names are camelCase in TypeScript and snake_case in Python:

    • output: the answer it judges: the turn's final answer. When the turn has a retry guardrail at the final answer, a halt or retry guardrail there judges each answer the model writes;
    • toolCalls, or tool_calls: its tool calls, each with the tool's name and arguments. At the final answer, that's every call the model asked for. A TypeScript check can also run at a tool call or a model reply, where the trace holds other fields: Check a tool call or a reply;
    • the tools' results, its model calls, the user's input and its cost;
    • projectId, or project_id: the run's project;
    • orgId, or org_id: the project's org, when it has one.

    The runtime sets the project and org, never the input.

Put it on an agent

Section titled “Put it on an agent”

A guardrail checks only the agents that list it. Ask your coding agent: "Put the answer length guardrail on my agent." For reference, it adds it to the agent's guardrails:

// in agents/echo-agent/index.ts
guardrails: ['my-pack.response-not-empty' as GuardrailId, 'my-pack.answer-length' as GuardrailId],
Excerpt

The agent names the guardrail's id, not the check's.

Once it's saved, kindgi dev registers the guardrail with the rest of the pack. An agent that names a guardrail that isn't registered doesn't run. Its turn fails with:

Error [unresolved-guardrail]: Agent "my-pack.gated" references guardrails not in the registry: my-pack.no-such-guardrail

Run it

Section titled “Run it”

Start a turn to see the guardrail stop it. dev-echo's answer, its "isn't a real model" line and the tool's result, is longer than 60 characters, so the turn stops. --agent is the agent, and --input its input:

Terminal window
pnpm exec kindgi runs start --agent=my-pack.echo-agent --input='{"userMessage":"hi"}'
Error [guardrail-violation]: Turn blocked by guardrail 'my-pack.answer-length': The answer is 243 characters; the limit is 60.

The command exits with status 1. The run's status is failed, and it has no answer. Stop a turn or record a violation shows what the run records, and how to let the turn finish instead.

A check that throws can't run, and its action decides what happens next: with halt, the turn fails (When the check can't run). The error carries the exception's message: handler-throw: Check "…" evaluate() threw: …. Return passed: false for a rule that isn't met. Throw only when the check itself can't run.

Check a tool call or a reply

Section titled “Check a tool call or a reply”

A check can judge a step of the turn before it takes effect, instead of the final answer: a tool call before it runs, a call's result before the model sees it, or a model reply before its calls run. The check names the points it can run at, and each guardrail that runs it picks one of them with at (Where in the turn it runs).

defineCheck({ points }) names the points: 'before-tool', 'after-tool', 'model-step' and 'final-answer'. A check without points runs at the final answer only. This check refuses a refund over a limit, before the refund tool runs:

guardrails/refund-limit/index.ts
import { defineCheck } from '@kindgi/sdk/define';
import { z } from 'zod';


export const check = defineCheck({
  id: 'my-pack.checks.refund-limit',
  kind: 'zero-llm',
  points: ['before-tool'],
  configSchema: z.object({ maxAmount: z.number().positive().default(100) }),
  evaluate: async (config, trace) => {
    const amount = trace.call?.arguments.amount;
    if (typeof amount === 'number' && amount > config.maxAmount) {
      return {
        passed: false,
        reason: `The refund is ${amount}; the limit is ${config.maxAmount}.`,
      };
    }
    return { passed: true };
  },
});


export default {
  id: 'my-pack.refund-limit',
  name: 'Refunds stay under a limit',
  kind: 'zero-llm',
  check,
  config: { maxAmount: 100 },
  at: 'before-tool',
  scope: { tools: ['acme.orders.refund'] },
  action: { 'on-violation': 'halt' },
  severity: 'error',
};

At each point, the trace holds what the check judges:

  • trace.at: the point it runs at. Absent at the final answer.
  • trace.call, at 'before-tool' and 'after-tool': the call, with its callId, toolId, toolName, toolVersion and arguments.
  • trace.result, at 'after-tool': what the call came to. { output } when it returned, or { error: { code?, message } } when it threw.
  • trace.step, at 'model-step': the reply, with its text and the toolCalls it asks for. text is empty when the reply only asks for calls.

Away from the final answer, the trace has no output. Its toolCalls and toolResults hold only the calls that ran, never the one the check judges.

defineCheck throws invalid-check-definition when points is empty, names a point that doesn't exist, or names one twice.

With halt, a refund over the limit stops the turn before the call runs:

Error [guardrail-violation]: Turn blocked by guardrail 'my-pack.refund-limit' before calling acme.orders.refund: The refund is 250; the limit is 100.

With retry, the call doesn't run, and the model gets the reason as its result, so it can go on without the refund or ask for a smaller one. At a tool call or a model reply describes each action at each point.

Test the check

Section titled “Test the check”

The check is a function of the config and the trace, so a unit test calls it directly, without a run. Ask your coding agent: "Write tests for my answer length guardrail." For reference, the tests look like this, with the command that runs the pack's tests:

guardrails/answer-length/index.test.ts
import type { RunId, TenantId } from '@kindgi/sdk/types';
import { expect, test } from 'vitest';


import { check } from './index.js';


const trace = (output: string) => ({
  runId: 'run-1' as RunId,
  tenantId: 'test-tenant' as TenantId,
  output,
  toolCalls: [],
  toolResults: [],
  modelCalls: [],
  mode: 'runtime' as const,
});


test('passes a short answer', async () => {
  expect(await check.evaluate({ maxChars: 60 }, trace('Shipped.'), {})).toEqual({ passed: true });
});


test('fails a long answer, with a reason', async () => {
  const result = await check.evaluate({ maxChars: 5 }, trace('Shipped yesterday.'), {});
  expect(result.passed).toBe(false);
  expect(result.reason).toBe('The answer is 18 characters; the limit is 5.');
});
Terminal window
pnpm exec kindgi test
 ✓ guardrails/answer-length/index.test.ts (2 tests) 2ms
…
      Tests  7 passed (7)
Excerpt

In the console

Section titled “In the console”

A project's Guardrails page lists its guardrails: "Checks on each agent answer: how each checks, what happens when it fires, which agents it covers." Each row has:

  • How it checks: "A rule in code, no model call", "A model judges each answer, a model call each time" or "Another system checks it", with its check, such as "Check: forbidden substring (built in)";
  • When it fires: "The turn fails", or "It is recorded; nothing stops";
  • Applies: "In runs and in CI", or "In runs only, not in CI";
  • Named by: how many agents name it, "1 agent", or "No agent".

The list has a search box, sorts by a column's header, and Columns chooses which columns it shows.

The Guardrails page: "Checks on each agent answer: how each checks, what happens when it fires, which agents it covers.", a search box and the Columns menu, then No secrets in answers, Says thanks, No refund promises, No shouting and No placeholder text, each a rule in code with its built-in check, when it fires, where it applies and how many agents name it: 1 agent each, and No agent for No placeholder text.

A guardrail's page has:

  • What it checks: how, the check, its settings, when it fires, its severity and where it applies.
  • Agents it checks: each agent version that names it and can run here, marked "(latest)", "(live)" or "(live and latest)".
  • What it caught: its outcomes by agent version, as Stop a turn or record a violation shows.

No secrets in answers' page, with Remove: a rule in code, forbidden substring (built in), Patterns: "password", the turn fails, Critical, in runs and in CI; the agent version that names it, Guarded agent 1.0.0 (latest); and What it caught: 8 answers checked in the last 7 days, 6 passed and 2 blocked.

A flow step runs the agent version its flow version pins, so that version is checked too: "Drafts agent 1.0.0 (pinned by flow acme.drafting 1.0.0)".

No shouting's page, with Remove, whose agent version is one a flow pins: Drafts agent 1.0.0 (pinned by flow acme.drafting 1.0.0).

Guardrail admins also see Remove. It asks first, naming the agent versions whose turns then fail, and how they come back:

Remove No secrets in answers?
Guarded agent 1.0.0 (latest) names it: its turns fail, saying it isn't registered, until it's registered again or a version that doesn't name it goes live.
There's no undo: registering it again brings it back.
  • When a flow pins one of those versions, it adds: "The flow version that pins it fails at that step too: …".
  • Keep it closes the dialog with nothing changed.
  • Once it's removed, the dialog says "Removed. Its definition, to register it again:", with the definition.
  • Past 100 agents or flows, the page and the dialog say which weren't checked: "Only the first 100 agents were checked." They say the same of the first 100 flows' steps.
  • Under kindgi dev, Remove is off: "the pack's files are its guardrails. Take it out of the pack, and kindgi dev reloads it."

The Remove No secrets in answers? dialog: Guarded agent 1.0.0 (latest) names it, its turns fail until it's registered again or a version that doesn't name it goes live, and there's no undo; then Keep it and Remove.

Next

Section titled “Next”