# Retry a failing step

> Retry a step that fails now and then, with backoff, and give it a time limit.

This guide retries a flow step that fails now and then, with backoff, and gives it a time limit.

A step that calls something flaky can be retried before its failure fails the run. The retry policy goes on the **edge into the step**.

## Before you start

- **A pack on a runtime such as `kindgi dev`,** with Kindgi 0.1's CLI.
- **A step that's safe to run twice,** as the caution in [The policy](https://docs.kindgi.com/v0.1.8/guides/flows/retry-a-step.md#the-policy) explains.

## Retry a step

Ask your coding agent: "I want a flow that reserves an order's stock at the warehouse, and tries again while the warehouse is busy." Kindgi's skills in your pack tell it how ([How it knows Kindgi](https://docs.kindgi.com/v0.1.8/start/coding-agents.md)). For reference, the tool and the flow look like this. `acme.reserve-stock` calls a warehouse that is busy for the first two calls per order, and the flow's edge into the step retries it:

- **TypeScript**

  `tools/reserve-stock/index.ts`

  ```ts
  import { defineTool } from '@kindgi/sdk/define';
  import type { ToolId } from '@kindgi/sdk/types';
  import { z } from 'zod';

  // The warehouse API is busy for the first two calls per order.
  const calls = new Map<string, number>();

  const defined = defineTool({
    id: 'acme.reserve-stock' as ToolId,
    description: 'Reserves the stock for an order at the warehouse.',
    version: '0.1.0',
    input: z.object({ orderId: z.string() }),
    output: z.object({ orderId: z.string(), reserved: z.boolean(), attempt: z.number().int() }),
    effects: [],
    handler: async ({ orderId }) => {
      const attempt = (calls.get(orderId) ?? 0) + 1;
      calls.set(orderId, attempt);
      if (attempt < 3) throw new Error('Warehouse busy (503)');
      return { orderId, reserved: true, attempt };
    },
  });

  if (defined.kind === 'err') throw new Error(defined.error.message);

  export default defined.value;
  ```

  `flows/reserve-order/index.ts`

  ```ts
  import { defineFlow } from '@kindgi/sdk/define';

  const defined = defineFlow({
    id: 'acme.reserve-order',
    version: '0.1.0',
    name: 'Reserve an order',
    description: 'Reserves the stock for an order, retrying while the warehouse is busy.',
    nodes: [{ id: 'reserve', kind: 'tool', ref: 'acme.reserve-stock' }],
    edges: [
      {
        id: 'e1',
        from: '$start',
        to: 'reserve',
        policy: {
          retry: { maxAttempts: 4, delayMs: 500, backoff: 'exponential', maxDelayMs: 5_000 },
          timeoutMs: 10_000,
        },
      },
      { id: 'e2', from: 'reserve', to: '$end' },
    ],
  });

  if (defined.kind === 'err') throw new Error(defined.error.message);

  export default defined.value;
  ```

- **Python**

  `tools/reserve_stock.py`

  ```python
  from pydantic import BaseModel, Field

  from kindgi import tool

  # The warehouse API is busy for the first two calls per order.
  calls: dict[str, int] = {}


  class ReserveInput(BaseModel):
      order_id: str = Field(alias="orderId")


  class Reserved(BaseModel):
      order_id: str = Field(alias="orderId")
      reserved: bool
      attempt: int


  @tool(id="acme.reserve-stock")
  def reserve_stock(input: ReserveInput) -> Reserved:
      """Reserves the stock for an order at the warehouse."""
      attempt = calls.get(input.order_id, 0) + 1
      calls[input.order_id] = attempt
      if attempt < 3:
          raise RuntimeError("Warehouse busy (503)")
      return Reserved(orderId=input.order_id, reserved=True, attempt=attempt)
  ```

  `flows/reserve_order.py`

  ```python
  from kindgi import Flow

  reserve_order = Flow(
      id="acme.reserve-order",
      version="0.1.0",
      name="Reserve an order",
      description="Reserves the stock for an order, retrying while the warehouse is busy.",
      nodes=[{"id": "reserve", "kind": "tool", "ref": "acme.reserve-stock"}],
      edges=[
          {
              "id": "e1",
              "from": "$start",
              "to": "reserve",
              "policy": {
                  "retry": {"maxAttempts": 4, "delayMs": 500, "backoff": "exponential", "maxDelayMs": 5_000},
                  "timeoutMs": 10_000,
              },
          },
          {"id": "e2", "from": "reserve", "to": "$end"},
      ],
  )
  ```

- **Java**

  `src/main/java/acme/tools/ReserveStock.java`

  ```java
  package acme.tools;

  import com.kindgi.pack.Tool;
  import java.util.Map;
  import java.util.concurrent.ConcurrentHashMap;

  public final class ReserveStock {
    public record Input(String orderId) {}

    public record Output(String orderId, boolean reserved, int attempt) {}

    // The warehouse API is busy for the first two calls per order.
    private static final Map<String, Integer> CALLS = new ConcurrentHashMap<>();

    public static final Tool<Input, Output> TOOL = Tool.define("acme.reserve-stock")
        .description("Reserves the stock for an order at the warehouse.")
        .version("0.1.0")
        .input(Input.class)
        .output(Output.class)
        .handler((input, ctx) -> {
          int attempt = CALLS.merge(input.orderId(), 1, Integer::sum);
          if (attempt < 3) throw new IllegalStateException("Warehouse busy (503)");
          return new Output(input.orderId(), true, attempt);
        });

    private ReserveStock() {}
  }
  ```

  `src/main/java/acme/flows/ReserveOrder.java`

  ```java
  package acme.flows;

  import com.kindgi.pack.Flow;
  import java.util.Map;

  public final class ReserveOrder {
    public static final Flow FLOW = Flow.define("acme.reserve-order")
        .version("0.1.0")
        .set("name", "Reserve an order")
        .set("description", "Reserves the stock for an order, retrying while the warehouse is busy.")
        .node(Map.of("id", "reserve", "kind", "tool", "ref", "acme.reserve-stock"))
        .edge(Map.of("id", "e1", "from", "$start", "to", "reserve",
            "policy", Map.of(
                "retry", Map.of("maxAttempts", 4, "delayMs", 500, "backoff", "exponential", "maxDelayMs", 5_000),
                "timeoutMs", 10_000)))
        .edge("e2", "reserve", "$end")
        .build();

    private ReserveOrder() {}
  }
  ```

- **Scala**

  `src/main/scala/acme/tools/ReserveStock.scala`

  ```scala
  package acme.tools

  import com.kindgi.pack.scaladsl._
  import java.util.concurrent.ConcurrentHashMap

  object ReserveStock {
    final case class Input(orderId: String)
    final case class Output(orderId: String, reserved: Boolean, attempt: Int)

    // The warehouse API is busy for the first two calls per order.
    private val calls = new ConcurrentHashMap[String, Integer]()

    val tool: Tool[Input, Output] = Tool[Input, Output]("acme.reserve-stock")
      .description("Reserves the stock for an order at the warehouse.")
      .version("0.1.0")
      .handler { (in, _) =>
        val attempt: Int = calls.merge(in.orderId, 1, (a, b) => a + b)
        if (attempt < 3) throw new IllegalStateException("Warehouse busy (503)")
        Output(in.orderId, reserved = true, attempt)
      }
  }
  ```

  `src/main/scala/acme/flows/ReserveOrder.scala`

  ```scala
  package acme.flows

  import com.kindgi.pack.scaladsl._

  object ReserveOrder {
    val flow: Flow = Flow("acme.reserve-order")
      .version("0.1.0")
      .set("name", "Reserve an order")
      .set("description", "Reserves the stock for an order, retrying while the warehouse is busy.")
      .node(Map("id" -> "reserve", "kind" -> "tool", "ref" -> "acme.reserve-stock"))
      .edge(Map("id" -> "e1", "from" -> "$start", "to" -> "reserve",
        "policy" -> Map(
          "retry" -> Map("maxAttempts" -> 4, "delayMs" -> 500, "backoff" -> "exponential", "maxDelayMs" -> 5000),
          "timeoutMs" -> 10000)))
      .edge("e2", "reserve", "$end")
      .build()
  }
  ```

Run it:

- **TypeScript**

  ```sh
  pnpm exec kindgi runs start --flow=acme.reserve-order --input='{"orderId":"A-200"}'
  ```

- **Python**

  ```sh
  uv run kindgi runs start --flow=acme.reserve-order --input='{"orderId":"A-200"}'
  ```

- **Java**

  ```sh
  ./kindgiw runs start --flow=acme.reserve-order --input='{"orderId":"A-200"}'
  ```

- **Scala**

  ```sh
  ./kindgiw runs start --flow=acme.reserve-order --input='{"orderId":"A-200"}'
  ```

`--flow` is the flow, and `--input` its input. It prints the run, which completed at the third attempt:

```json excerpt
{
  …
  "status": "completed",
  …
  "output": {
    "attempt": 3,
    "orderId": "A-200",
    "reserved": true
  },
  …
}
```

The journal, which `kindgi runs journal <run-id>` prints, has each retry. It shows the error that caused it and the wait before the next attempt:

```json excerpt
{"sequence": 2, "kind": "step.started", "nodeId": "reserve", "payload": {"input": {"orderId": "A-200"}}, …}
{"sequence": 3, "kind": "step.retry-scheduled", "nodeId": "reserve", "payload": {"nodeId": "reserve", "attempt": 1, "nextDelayMs": 500, "previousError": "handler-error: Tool \"acme.reserve-stock\" handler threw: … Warehouse busy (503)"}, …}
{"sequence": 4, "kind": "step.retry-scheduled", "nodeId": "reserve", "payload": {"nodeId": "reserve", "attempt": 2, "nextDelayMs": 1000, "previousError": "handler-error: Tool \"acme.reserve-stock\" handler threw: … Warehouse busy (503)"}, …}
{"sequence": 5, "kind": "step.completed", "nodeId": "reserve", "payload": {"output": {"attempt": 3, "orderId": "A-200", "reserved": true}}, …}
```

## The policy

`policy.retry`:

- **`maxAttempts`:** attempts in all, the first one included, at most 10.

- **`delayMs`:** the wait before the first retry, 0 by default.

- **`backoff`:** how the wait grows.

  - `fixed`, the default: the same wait each time.
  - `linear`: `delayMs` × the retry's number.
  - `exponential`: doubling each time, 500, 1000, 2000 ms here. It needs **`maxDelayMs`**, the longest wait.

When the last attempt fails, the step fails and so does the run, with the last error.

`policy.timeoutMs` limits each attempt. An attempt that takes longer is stopped and counts as a failed attempt. Without retries, the step fails with `reason: "timeout"`:

```json excerpt
{"sequence": 3, "kind": "step.failed", "nodeId": "reserve", "payload": {"reason": "timeout", "limitMs": 5, "message": "Handler for node \"reserve\" exceeded timeout of 5ms", "elapsedMs": 68}, …}
```

That entry came from `timeoutMs: 5`, set that low to show it.

> **Caution**
>
> Retry only a step that is safe to run twice. A tool that failed may have done part of its work: a payment that timed out may still have gone through. Make the tool idempotent, such as by passing the order id to the API as its idempotency key, or don't retry it.

## Where a policy applies

A policy applies to the step its edge leads to, when that step has exactly one incoming edge. A step that joins branches, with several incoming edges, ignores the policies on them. The other policy fields, `concurrencyKey` and `priority`, are in the [flow schema](https://docs.kindgi.com/v0.1.8/reference/schemas/flow.md).

## Next

- [Run steps in parallel](https://docs.kindgi.com/v0.1.8/guides/flows/run-steps-in-parallel.md)
- [Retry a start safely](https://docs.kindgi.com/v0.1.8/guides/runs/retry-a-start-safely.md)
- [See why runs failed](https://docs.kindgi.com/v0.1.8/guides/runs/see-why-runs-failed.md)