Skip to content
Kindgi

Write a guardrail

View as Markdown

This guide writes a guardrail that keeps an agent's answers short, puts it on an agent, and tests its check.

Before you start

Section titled “Before you start”
  • A pack with an agent, on a runtime such as kindgi dev, with Kindgi 0.1's CLI. This guide uses the quickstart's pack, my-pack, and its my-pack.echo-agent, from the TypeScript, Python, Java or Scala quickstart.

The guardrail

Section titled “The guardrail”

The guardrail fails a turn whose answer is longer than a limit. It's one file in the pack's guardrails folder. Ask your coding agent: "I want a guardrail that keeps my agent's answers short." Kindgi's skills in your pack tell it how (How it knows Kindgi). For reference, the guardrail looks like this. This guide calls it my-pack.answer-length:

guardrails/answer-length/index.ts
import { defineCheck } from '@kindgi/sdk/define';
import { z } from 'zod';


export const check = defineCheck({
  id: 'my-pack.checks.answer-length',
  kind: 'zero-llm',
  configSchema: z.object({ maxChars: z.number().int().positive().default(500) }),
  evaluate: async (config, trace) => {
    const length = trace.output?.length ?? 0;
    if (length > config.maxChars) {
      return {
        passed: false,
        reason: `The answer is ${length} characters; the limit is ${config.maxChars}.`,
      };
    }
    return { passed: true };
  },
});


export default {
  id: 'my-pack.answer-length',
  name: 'Answer is short',
  kind: 'zero-llm',
  check,
  config: { maxChars: 60 },
  action: { 'on-violation': 'halt' },
  severity: 'error',
};

The file has two parts:

  • The check, from defineCheck, exported as check. Its evaluate gets the guardrail's config and the turn's trace, and returns { passed, reason? }.
  • The declaration, the default export: the guardrail's id, the check it runs, the check's config, and the action on a failure.

  • The id is <pack-id>.<name>. Name the rule as what should hold: answer-length, response-not-empty.

  • kind: 'zero-llm': the check is your code, a function of the trace. It can't call a model.

  • The reason is what the violation reports. Say what was wrong. It shows wherever the failure does, such as the run's error and its page in the console. So describe what you blocked, never quote it: "it contains a card number", not the number.

  • The trace is the turn. RunTrace in the Python reference lists its fields. Its names are camelCase in TypeScript and snake_case in Python:

    • output: the turn's final answer;
    • toolCalls, or tool_calls: its tool calls, each with the tool's name and arguments;
    • the tools' results, its model calls, the user's input and its cost;
    • projectId, or project_id: the run's project;
    • orgId, or org_id: the project's org, when it has one.

    The runtime sets the project and org, never the input.

Put it on an agent

Section titled “Put it on an agent”

A guardrail checks only the agents that list it. Ask your coding agent: "Put the answer length guardrail on my agent." For reference, it adds it to the agent's guardrails:

// in agents/echo-agent/index.ts
guardrails: ['my-pack.response-not-empty' as GuardrailId, 'my-pack.answer-length' as GuardrailId],
Excerpt

The agent names the guardrail's id, not the check's.

Once it's saved, kindgi dev registers the guardrail with the rest of the pack. An agent that names a guardrail that isn't registered doesn't run. Its turn fails with:

Error [unresolved-guardrail]: Agent "my-pack.gated" references guardrails not in the registry: my-pack.no-such-guardrail

Run it

Section titled “Run it”

Start a turn to see the guardrail stop it. dev-echo's answer, its "isn't a real model" line and the tool's result, is longer than 60 characters, so the turn stops. --agent is the agent, and --input its input:

Terminal window
pnpm exec kindgi runs start --agent=my-pack.echo-agent --input='{"userMessage":"hi"}'
Error [guardrail-violation]: Turn blocked by guardrail 'my-pack.answer-length': The answer is 243 characters; the limit is 60.

The command exits with status 1. The run's status is failed, and it has no answer. Stop a turn or record a violation shows what the run records, and how to let the turn finish instead.

A check that throws fails the turn whatever its action. The error carries the exception's message: handler-throw: Check "…" evaluate() threw: …. Return passed: false for a rule that isn't met. Throw only when the check itself can't run.

Test the check

Section titled “Test the check”

The check is a function of the config and the trace, so a unit test calls it directly, without a run. Ask your coding agent: "Write tests for my answer length guardrail." For reference, the tests look like this, with the command that runs the pack's tests:

guardrails/answer-length/index.test.ts
import type { RunId, TenantId } from '@kindgi/sdk/types';
import { expect, test } from 'vitest';


import { check } from './index.js';


const trace = (output: string) => ({
  runId: 'run-1' as RunId,
  tenantId: 'test-tenant' as TenantId,
  output,
  toolCalls: [],
  toolResults: [],
  modelCalls: [],
  mode: 'runtime' as const,
});


test('passes a short answer', async () => {
  expect(await check.evaluate({ maxChars: 60 }, trace('Shipped.'), {})).toEqual({ passed: true });
});


test('fails a long answer, with a reason', async () => {
  const result = await check.evaluate({ maxChars: 5 }, trace('Shipped yesterday.'), {});
  expect(result.passed).toBe(false);
  expect(result.reason).toBe('The answer is 18 characters; the limit is 5.');
});
Terminal window
pnpm exec kindgi test
 ✓ guardrails/answer-length/index.test.ts (2 tests) 2ms
…
      Tests  7 passed (7)
Excerpt

In the console

Section titled “In the console”

A project's Guardrails page lists its guardrails: "Checks on agents' answers: what each checks, what happens when an answer breaks it, and the agents it checks. A run checks an agent only with the guardrails its version names." Each row has:

  • Guardrail: its name and description;
  • Check: its check, such as "Forbidden substring", and how it checks: "A rule in code", "A model judges it, a call per answer" or "Another system checks it";
  • If an answer breaks it: "The turn fails" ("Its answer isn't kept"), or "Recorded only" ("The answer goes out"). A guardrail that asks for a retry, a reviewer or a compensation says "Recorded only" too, with a note that it doesn't happen yet (Stop a turn or record a violation);
  • Agents it checks: the agents it checks here, by name, or "None yet", which says why when you point at it. Under them, "Not in runs: in CI only" for a guardrail only CI runs, and "Only for …" or "Only in flows …" for one limited to some agents or flows.

The list has a search box, sorts by a column's header, and Columns chooses which columns it shows.

The Guardrails page: "Checks on agents' answers: what each checks, what happens when an answer breaks it, and the agents it checks. A run checks an agent only with the guardrails its version names.", a search box and the Columns menu, then No secrets in answers ("An answer never says a password."), Says thanks, No refund promises, No shouting and No placeholder text, each a rule in code with its check, a forbidden substring or, for Says thanks, a required one; what happens if an answer breaks it, the turn fails and its answer isn't kept or, for Says thanks, it's recorded only and the answer goes out; and the agents it checks: Guarded agent for the first two, Billing agent, Drafts agent, and None yet for No placeholder text.

A guardrail's page has:

  • What it checks: how, the check, its settings, what happens if an answer breaks it ("The turn fails: its answer isn't kept"), its severity, and where it applies: "In every run", "In every run, not in CI" or "Not in runs: in CI only".
  • Agents it checks: each agent version that names it and can run here, marked "(latest)", "(pinned)" or "(pinned and latest)".
  • What it caught: its outcomes by agent version, as Stop a turn or record a violation shows.

No secrets in answers' page, with Remove: What it checks, a rule in code with no model call, forbidden substring (built in), Patterns: "password", the turn fails and its answer isn't kept, Critical, in every run; Agents it checks: "Each agent version below names it. It applies in every run." and Guarded agent 1.0.0 (latest); What it caught: in the last 7 days it checked 8 answers, 6 passed and 2 blocked, with Guarded agent 1.0.0's row and the two runs it blocked; and Details, folded.

A flow step runs the agent version its flow version pins, so that version is checked too: "Drafts agent 1.0.0 (pinned by flow acme.drafting 1.0.0)".

No shouting's page, with Remove, whose agent version is one a flow pins: Drafts agent 1.0.0 (pinned by flow acme.drafting 1.0.0).

Guardrail admins also see Remove. It asks first, naming the agent versions whose turns then fail, and how they come back:

Remove No secrets in answers?
Guarded agent 1.0.0 (latest) names it: its turns fail, saying it isn't registered, until it's registered again or a version that doesn't name it takes over.
There's no undo: registering it again brings it back.
  • When a flow pins one of those versions, it adds: "The flow version that pins it fails at that step too: …".
  • Keep it closes the dialog with nothing changed.
  • Once it's removed, the dialog says "Removed. Its definition, to register it again:", with the definition.
  • Past 100 agents or flows, the page and the dialog say which weren't checked: "Only the first 100 agents were checked." They say the same of the first 100 flows' steps.
  • Under kindgi dev, Remove is off: "the pack's files are its guardrails. Take it out of the pack, and kindgi dev reloads it."

The Remove No secrets in answers? dialog: "Guarded agent 1.0.0 (latest) names it: its turns fail, saying it isn't registered, until it's registered again or a version that doesn't name it takes over." and "There's no undo: registering it again brings it back."; then Keep it and Remove.

Next

Section titled “Next”