# Write a guardrail > Write a check over an agent's turn, declare it as a guardrail, and put it on an agent. This guide writes a guardrail that keeps an agent's answers short, puts it on an agent, and tests its check. ## Before you start - **A pack with an agent,** on a runtime such as `kindgi dev`, with Kindgi 0.1's CLI. This guide uses the quickstart's pack, `my-pack`, and its `my-pack.echo-agent`, from the [TypeScript](https://docs.kindgi.com/v0.1.8/start/quickstart-typescript.md), [Python](https://docs.kindgi.com/v0.1.8/start/quickstart-python.md), [Java](https://docs.kindgi.com/v0.1.8/start/quickstart-java.md) or [Scala](https://docs.kindgi.com/v0.1.8/start/quickstart-scala.md) quickstart. ## The guardrail The guardrail fails a turn whose answer is longer than a limit. It's one file in the pack's `guardrails` folder. Ask your coding agent: "I want a guardrail that keeps my agent's answers short." Kindgi's skills in your pack tell it how ([How it knows Kindgi](https://docs.kindgi.com/v0.1.8/start/coding-agents.md)). For reference, the guardrail looks like this. This guide calls it `my-pack.answer-length`: - **TypeScript** `guardrails/answer-length/index.ts` ```ts import { defineCheck } from '@kindgi/sdk/define'; import { z } from 'zod'; export const check = defineCheck({ id: 'my-pack.checks.answer-length', kind: 'zero-llm', configSchema: z.object({ maxChars: z.number().int().positive().default(500) }), evaluate: async (config, trace) => { const length = trace.output?.length ?? 0; if (length > config.maxChars) { return { passed: false, reason: `The answer is ${length} characters; the limit is ${config.maxChars}.`, }; } return { passed: true }; }, }); export default { id: 'my-pack.answer-length', name: 'Answer is short', kind: 'zero-llm', check, config: { maxChars: 60 }, action: { 'on-violation': 'halt' }, severity: 'error', }; ``` The file has two parts: - **The check**, from `defineCheck`, exported as `check`. Its `evaluate` gets the guardrail's config and the turn's trace, and returns `{ passed, reason? }`. - **The declaration**, the default export: the guardrail's id, the check it runs, the check's `config`, and the `action` on a failure. - **Python** `guardrails/answer_length.py` ```python from pydantic import BaseModel, Field from kindgi import CheckResult, RunTrace, guardrail class Config(BaseModel): max_chars: int = Field(500, alias="maxChars", gt=0) @guardrail( id="my-pack.answer-length", name="Answer is short", on_violation="halt", severity="error", config={"maxChars": 60}, ) def answer_length(config: Config, trace: RunTrace) -> CheckResult: length = len(trace.output or "") if length > config.max_chars: return CheckResult( passed=False, reason=f"The answer is {length} characters; the limit is {config.max_chars}.", ) return CheckResult(passed=True) ``` `@guardrail` declares both parts: - **The check** is the function. It takes `(config, trace)`, and can be a `def` or an `async def`. - **The declaration** is the decorator's arguments. The first parameter's type is the config's schema. - **Java** `src/main/java/mypack/guardrails/AnswerLength.java` ```java package mypack.guardrails; import com.fasterxml.jackson.annotation.JsonProperty; import com.kindgi.pack.CheckResult; import com.kindgi.pack.Guardrail; import jakarta.validation.constraints.Positive; import java.util.Map; public final class AnswerLength { public record Config(@JsonProperty(defaultValue = "500") @Positive int maxChars) {} public static final Guardrail GUARDRAIL = Guardrail.define("my-pack.answer-length") .name("Answer is short") .checkId("my-pack.checks.answer-length") .onViolation("halt") .severity("error") .config(Config.class) .set("config", Map.of("maxChars", 60)) .check((config, trace) -> { int length = trace.output() == null ? 0 : trace.output().length(); if (length > config.maxChars()) { return CheckResult.fail( "The answer is " + length + " characters; the limit is " + config.maxChars() + "."); } return CheckResult.pass(); }); private AnswerLength() {} } ``` The class has both parts: - **The check** is the lambda in `check(...)`. It gets the guardrail's config, as a `Config`, and the turn's trace, and returns `CheckResult.pass()` or `CheckResult.fail(reason)`. - **The declaration** is the builder: the guardrail's id, its check's id (`checkId`), the config's type (`config(Config.class)`), the config's values (`set("config", …)`, keyed as on the wire) and the action on a failure. - **Scala** `src/main/scala/mypack/guardrails/AnswerLength.scala` ```scala package mypack.guardrails import com.kindgi.pack.scaladsl._ import jakarta.validation.constraints.Positive object AnswerLength { final case class Config(@Positive maxChars: Int = 500) val guardrail: Guardrail[Config] = Guardrail[Config]("my-pack.answer-length") .name("Answer is short") .checkId("my-pack.checks.answer-length") .onViolation("halt") .severity("error") .set("config", Map("maxChars" -> 60)) .check { (config, trace) => val length = Option(trace.output).map(_.length).getOrElse(0) if (length > config.maxChars) CheckResult.fail(s"The answer is $length characters; the limit is ${config.maxChars}.") else CheckResult.pass } } ``` `Guardrail[Config]` declares both parts: the function in `check` is the check, `(config, trace)`, and the builder is the declaration. `Config`, a case class, is the config's schema, and `set("config", …)` gives its values. * **The id** is `.`. Name the rule as what should hold: `answer-length`, `response-not-empty`. * **`kind: 'zero-llm'`**: the check is your code, a function of the trace. It can't call a model. * **The reason** is what the violation reports. Say what was wrong. It shows wherever the failure does, such as the run's error and its page in the console. So describe what you blocked, never quote it: "it contains a card number", not the number. * **The trace** is the turn. `RunTrace` in the [Python reference](https://docs.kindgi.com/v0.1.8/reference/python/authoring.md#runtrace) lists its fields. Its names are camelCase in TypeScript and snake_case in Python: - `output`: the turn's final answer; - `toolCalls`, or `tool_calls`: its tool calls, each with the tool's name and arguments; - the tools' results, its model calls, the user's input and its cost; - `projectId`, or `project_id`: the run's project; - `orgId`, or `org_id`: the project's org, when it has one. The runtime sets the project and org, never the input. ## Put it on an agent A guardrail checks only the agents that list it. Ask your coding agent: "Put the answer length guardrail on my agent." For reference, it adds it to the agent's guardrails: - **TypeScript** ```ts excerpt // in agents/echo-agent/index.ts guardrails: ['my-pack.response-not-empty' as GuardrailId, 'my-pack.answer-length' as GuardrailId], ``` The agent names the guardrail's id, not the check's. - **Python** ```python excerpt # in agents/echo_agent.py from ..guardrails.answer_length import answer_length from ..guardrails.response_not_empty import response_not_empty # … guardrails=[response_not_empty, answer_length], ``` - **Java** ```java excerpt // in src/main/java/mypack/agents/EchoAgent.java import mypack.guardrails.AnswerLength; // … .guardrail(ResponseNotEmpty.GUARDRAIL) .guardrail(AnswerLength.GUARDRAIL) ``` - **Scala** ```scala excerpt // in src/main/scala/mypack/agents/EchoAgent.scala import mypack.guardrails.{AnswerLength, ResponseNotEmpty} // … .guardrail(ResponseNotEmpty.guardrail) .guardrail(AnswerLength.guardrail) ``` Once it's saved, `kindgi dev` registers the guardrail with the rest of the pack. An agent that names a guardrail that isn't registered doesn't run. Its turn fails with: ```text Error [unresolved-guardrail]: Agent "my-pack.gated" references guardrails not in the registry: my-pack.no-such-guardrail ``` ## Run it Start a turn to see the guardrail stop it. `dev-echo`'s answer, its "isn't a real model" line and the tool's result, is longer than 60 characters, so the turn stops. `--agent` is the agent, and `--input` its input: - **TypeScript** ```sh pnpm exec kindgi runs start --agent=my-pack.echo-agent --input='{"userMessage":"hi"}' ``` ```text Error [guardrail-violation]: Turn blocked by guardrail 'my-pack.answer-length': The answer is 243 characters; the limit is 60. ``` - **Python** ```sh uv run kindgi runs start --agent=my-pack.echo-agent --input='{"userMessage":"Ada"}' ``` ```text Error [guardrail-violation]: Turn blocked by guardrail 'my-pack.answer-length': The answer is 252 characters; the limit is 60. ``` - **Java** ```sh ./kindgiw runs start --agent=my-pack.echo-agent --input='{"userMessage":"hi"}' ``` ```text Error [guardrail-violation]: Turn blocked by guardrail 'my-pack.answer-length': The answer is 246 characters; the limit is 60. ``` - **Scala** ```sh ./kindgiw runs start --agent=my-pack.echo-agent --input='{"userMessage":"hi"}' ``` ```text Error [guardrail-violation]: Turn blocked by guardrail 'my-pack.answer-length': The answer is 246 characters; the limit is 60. ``` The command exits with status 1. The run's status is `failed`, and it has no answer. [Stop a turn or record a violation](https://docs.kindgi.com/v0.1.8/guides/guardrails/halt-or-record.md) shows what the run records, and how to let the turn finish instead. A check that throws fails the turn whatever its action. The error carries the exception's message: `handler-throw: Check "…" evaluate() threw: …`. Return `passed: false` for a rule that isn't met. Throw only when the check itself can't run. ## Test the check The check is a function of the config and the trace, so a unit test calls it directly, without a run. Ask your coding agent: "Write tests for my answer length guardrail." For reference, the tests look like this, with the command that runs the pack's tests: - **TypeScript** `guardrails/answer-length/index.test.ts` ```ts import type { RunId, TenantId } from '@kindgi/sdk/types'; import { expect, test } from 'vitest'; import { check } from './index.js'; const trace = (output: string) => ({ runId: 'run-1' as RunId, tenantId: 'test-tenant' as TenantId, output, toolCalls: [], toolResults: [], modelCalls: [], mode: 'runtime' as const, }); test('passes a short answer', async () => { expect(await check.evaluate({ maxChars: 60 }, trace('Shipped.'), {})).toEqual({ passed: true }); }); test('fails a long answer, with a reason', async () => { const result = await check.evaluate({ maxChars: 5 }, trace('Shipped yesterday.'), {}); expect(result.passed).toBe(false); expect(result.reason).toBe('The answer is 18 characters; the limit is 5.'); }); ``` ```sh pnpm exec kindgi test ``` ```text excerpt ✓ guardrails/answer-length/index.test.ts (2 tests) 2ms … Tests 7 passed (7) ``` - **Python** `tests/test_answer_length.py` ```python from kindgi import RunTrace from guardrails.answer_length import Config, answer_length def test_a_short_answer_passes(): trace = RunTrace(run_id="r", tenant_id="t", output="Shipped.") assert answer_length(Config(maxChars=60), trace).passed def test_a_long_answer_fails_with_a_reason(): trace = RunTrace(run_id="r", tenant_id="t", output="Shipped yesterday.") result = answer_length(Config(maxChars=5), trace) assert not result.passed assert result.reason == "The answer is 18 characters; the limit is 5." ``` ```sh uv run pytest ``` ```text ........ [100%] 8 passed in 0.19s ``` - **Java** `src/test/java/mypack/guardrails/AnswerLengthTest.java` ```java package mypack.guardrails; import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertTrue; import com.kindgi.pack.CheckResult; import com.kindgi.pack.RunTrace; import java.util.Map; import org.junit.jupiter.api.Test; class AnswerLengthTest { private static RunTrace trace(String output) { return new RunTrace(Map.of("runId", "run-1", "tenantId", "test-tenant", "output", output)); } @Test void passesAShortAnswer() throws Exception { assertTrue(AnswerLength.GUARDRAIL.evaluate(new AnswerLength.Config(60), trace("Shipped.")).passed()); } @Test void failsALongAnswerWithAReason() throws Exception { CheckResult result = AnswerLength.GUARDRAIL.evaluate(new AnswerLength.Config(5), trace("Shipped yesterday.")); assertFalse(result.passed()); assertEquals("The answer is 18 characters; the limit is 5.", result.reason()); } } ``` ```sh ./mvnw test ``` ```text excerpt [INFO] Running mypack.guardrails.AnswerLengthTest [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.444 s -- in mypack.guardrails.AnswerLengthTest … [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS ``` - **Scala** `src/test/scala/mypack/guardrails/AnswerLengthSuite.scala` ```scala package mypack.guardrails import com.kindgi.pack.RunTrace import scala.jdk.CollectionConverters._ class AnswerLengthSuite extends munit.FunSuite { private def trace(output: String) = new RunTrace(Map[String, AnyRef]("runId" -> "run-1", "tenantId" -> "test-tenant", "output" -> output).asJava) test("passes a short answer") { assert(AnswerLength.guardrail.evaluate(AnswerLength.Config(60), trace("Shipped.")).passed) } test("fails a long answer, with a reason") { val result = AnswerLength.guardrail.evaluate(AnswerLength.Config(5), trace("Shipped yesterday.")) assert(!result.passed) assertEquals(result.reason, "The answer is 18 characters; the limit is 5.") } } ``` ```sh sbt test ``` ```text excerpt … Test run mypack.guardrails.AnswerLengthSuite finished: 0 failed, 0 ignored, 2 total 0.525s … [info] Passed: Total 5, Failed 0, Errors 0, Passed 5 ``` ## In the console A project's **Guardrails** page lists its guardrails, with a headline such as "5 guardrails; 1 isn't named by any agent here." Each row says: - **how it checks:** "A rule: no model call", with the check; - **what happens when it fires:** "The turn fails", or "It is recorded; nothing stops"; - **when it applies:** "In runs and in CI", or "In runs only, not in CI"; - **how many agents name it:** "1 agent", or "No agent". The list has a search box, sorts by a column's header, and **Columns** chooses which columns it shows. ![The Guardrails page: "5 guardrails; 1 isn't named by any agent here.", a search box and the Columns menu, with No secrets in answers, Says thanks, No refund promises, No shouting and No placeholder text, each with how it checks, when it fires, where it applies and how many agents name it.](https://docs.kindgi.com/v0.1.8/_astro/guardrails-list.DmPTcfmV_16ih0j.webp) A guardrail's page has: - **What it checks:** how, the check, its settings, when it fires, its severity and where it applies. - **Agents it checks:** each agent version that names it and can run here, marked "(latest)" or "(live)". - **What it caught:** its outcomes by agent version, as [Stop a turn or record a violation](https://docs.kindgi.com/v0.1.8/guides/guardrails/halt-or-record.md#in-the-console) shows. ![No secrets in answers' page: a rule, forbidden-substring (built in), patterns \["password"\], the turn fails, critical, in runs and in CI; the agent version that names it, Guarded agent 1.0.0 (latest); and What it caught.](https://docs.kindgi.com/v0.1.8/_astro/guardrail-page.BP_QkskW_Z1QJGxF.webp) A flow step runs the agent version its flow version pins, so that version is checked too: "Drafts agent 1.0.0 (pinned by flow acme.drafting 1.0.0)". ![No shouting's page, whose agent version is one a flow pins: Drafts agent 1.0.0 (pinned by flow acme.drafting 1.0.0).](https://docs.kindgi.com/v0.1.8/_astro/guardrail-flow-pinned.BxIBtSRu_ZkjLfG.webp) Guardrail admins also see **Unregister**. Its dialog names the agent versions whose turns then fail, and how they come back: ```text Unregister No secrets in answers? Guarded agent 1.0.0 (latest) names it: its turns fail, saying it isn't registered, until it's registered again or a version that doesn't name it goes live. There's no reinstate: registering it again brings it back. ``` - **When a flow pins one of those versions,** it adds: "The flow version that pins it fails at that step too: …". - **Once it's done,** the dialog shows the guardrail's definition, to register it again. - **Past 100 agents or flows,** the page and the dialog say which weren't checked: "Only the first 100 agents were checked." They say the same of the first 100 flows' steps. ![The Unregister No secrets in answers? dialog: Guarded agent 1.0.0 (latest) names it, its turns fail until it's registered again or a version that doesn't name it goes live, and there's no reinstate.](https://docs.kindgi.com/v0.1.8/_astro/guardrail-unregister.DGTAGpY5_ZmlFCm.webp) ## Next - [Configure a guardrail](https://docs.kindgi.com/v0.1.8/guides/guardrails/configure-a-guardrail.md) - [Stop a turn or record a violation](https://docs.kindgi.com/v0.1.8/guides/guardrails/halt-or-record.md) - [Use a built-in check](https://docs.kindgi.com/v0.1.8/guides/guardrails/use-a-built-in-check.md)