# @kindgi/adapter-model-in-process

> In-process ModelProvider for @kindgi/capabilities. Runs small instruction-tuned LLMs inside the Node.js process via @huggingface/transformers (ONNX runtime; no external processes, no API keys). SmolLM2-360M by default (~273MB q4f16, Apache 2.0, tuned for function calling); SmolLM2-135M and Qwen3-0.6B selectable via the `models` option. prepareInProcessModel downloads a model ahead of first use and streams progress. Positioned as the cheapest tier for routing/classification/smoke-test workloads — not a replacement for hosted providers on real reasoning tasks.

`npm install @kindgi/adapter-model-in-process` · [source](https://github.com/kindgi/kindgi-sdk/tree/@kindgi/sdk@0.1.8/packages/adapters/model-in-process)

In-process `ModelProvider` for [`@kindgi/capabilities`](https://docs.kindgi.com/v0.1.8/reference/packages/capabilities.md). Runs small instruction-tuned models inside the Node.js process through `@huggingface/transformers` (ONNX runtime): no API key, no external process, no network once the weights are cached, and zero USD cost. Intended as the cheapest routing tier — classification, extraction, routing, smoke tests — not as a replacement for hosted models on reasoning tasks.

## Purpose

Give the capability router a local provider it can match like any other. Every model reports `cost` of `0`, the provider's `region` is `'in-process'`, and `attributes` carry `in-process` plus each model's tier and `suitableFor` hints (`routing`, `classification`, `local`, `lower-cost`, …) for preference matching. `prepareInProcessModel` downloads weights ahead of the first call and streams progress in the `PrepareEvent` shape used by `AdapterFactoryEntry.prepare`.

## Installing

The ONNX runtime it runs on (`onnxruntime-node`, through `@huggingface/transformers`) ships its CPU binaries in the package. On Linux x64 its install script also downloads the CUDA and TensorRT providers from NuGet by default; this adapter runs on the CPU, so skip that download: with npm or yarn, `ONNXRUNTIME_NODE_INSTALL=skip npm install …` (the package's own variable). pnpm 10 and later don't run a dependency's install script unless you allow it, so leave `onnxruntime-node` off your allowed builds.

## Exports

- **`inProcessAdapterEntry`** (`IN_PROCESS_ADAPTER_ID`) — the `AdapterFactoryEntry` a runtime registers: **`inProcessAdapterFactory`** (a registration lists models by their `MODEL_SPECS` key in `metadata.models[].name`), `prepare` (`prepareInProcessModel`), and **`inProcessCheckConfig(input)`**, which reports a model that isn't one of `MODEL_SPECS` (`metadata.models`) with the factory's own message.

- **`createInProcessModelProvider(options?)`** — returns a `ModelProvider`. Construction is cheap: each model's pipeline loads on the first `invoke()` that names it (concurrent first calls share one load), downloading weights if they are not cached yet. With `@huggingface/transformers` 4.x the default cache is a `.cache` directory inside that package's install location; `cacheDir` redirects the model files (transformers.js 4.3 can still write a model's `config.json` to its default cache). Throws when `models` is empty; `invoke()` throws for a model the provider does not expose.

  - `invoke()` passes each message to the model's chat template as `{ role, content }`, uses `maxOutputTokens` as `max_new_tokens` (default `512`), samples only when `temperature > 0` (greedy otherwise), and returns `finishReason: 'stop'` and `costUsd: 0`. Token counts come from the model's tokenizer, falling back to about four characters per token. `abortSignal` is not observed.

- **`InProcessProviderOptions`**:

  - `models?: readonly LocalModel[]` — models to expose; default `[DEFAULT_LOCAL_MODEL]`.
  - `providerId?: string` — `metadata.id`; default `in-process/<models joined with '+'>`. Lets the same model set be registered twice under different ids.
  - `cacheDir?: string` — directory for downloaded model files, passed to transformers.js as `cache_dir`.

- **`LocalModel`**, **`MODEL_SPECS`**, **`ModelSpec`**, **`DEFAULT_LOCAL_MODEL`** — the closed model set and its per-model spec (`hfName`, `approxDownloadMb`, `contextWindow`, `toolUse`, `tier`, `suitableFor`, `dtype`). All use `q4f16` weights:

  | `LocalModel`             | Hugging Face repo                     | Download | Context | Tier        |
  | ------------------------ | ------------------------------------- | -------- | ------- | ----------- |
  | `smollm2-135m`           | `HuggingFaceTB/SmolLM2-135M-Instruct` | \~118 MB | 8,192   | ultra-light |
  | `smollm2-360m` (default) | `HuggingFaceTB/SmolLM2-360M-Instruct` | \~273 MB | 8,192   | small       |
  | `qwen3-0.6b`             | `onnx-community/Qwen3-0.6B-ONNX`      | \~550 MB | 32,768  | medium      |

- **`prepareInProcessModel(params?)`** / **`PrepareInProcessParams`** — `AsyncIterable<PrepareEvent>` for `{ model?, cacheDir? }` (model defaults to `DEFAULT_LOCAL_MODEL`). Loads the model once, yielding `progress` events (with `ratio`, `loadedBytes`, `totalBytes`, `file` when known) and then a single `ready` or `error`. The loaded pipeline is not shared: a provider loads its own copy from the cache on first `invoke()`.

- **`ModelProvider`**, **`PrepareEvent`** — type re-exports from `@kindgi/capabilities`.

## Example

```ts excerpt
import { createProviderRegistry } from '@kindgi/capabilities';
import {
  createInProcessModelProvider,
  prepareInProcessModel,
} from '@kindgi/adapter-model-in-process';

// Download (or verify the cached copy of) the weights before serving traffic.
for await (const event of prepareInProcessModel({ model: 'smollm2-360m' })) {
  if (event.kind === 'error') throw new Error(event.message);
  if (event.kind === 'progress' && event.ratio !== undefined) {
    console.log(`${event.file ?? 'model'} ${Math.round(event.ratio * 100)}%`);
  }
}

const provider = createInProcessModelProvider({ models: ['smollm2-360m'] });
const { registry } = createProviderRegistry();
registry.register(tenantId, provider); // metadata.id: 'in-process/smollm2-360m'

const result = await provider.invoke({
  model: 'smollm2-360m',
  messages: [
    { role: 'system', content: 'Classify the ticket as billing, bug or other. Reply with one word.' },
    { role: 'user', content: 'I was charged twice for the acme plan this month.' },
  ],
  maxOutputTokens: 8,
});
console.log(result.message.content, result.usage); // costUsd is always 0
```

## Non-goals

- **Streaming.** `invoke()` resolves with the complete response.
- **Tool calling and structured output.** `tools`, `structuredOutput`, and message `toolCalls` / `toolCallId` are not passed to the model; responses are plain text.
- **Arbitrary Hugging Face models.** Only the three `LocalModel` keys are selectable.

## Related

- [`@kindgi/capabilities`](https://docs.kindgi.com/v0.1.8/reference/packages/capabilities.md) — `ModelProvider`, the provider registry, and `AdapterFactoryEntry`.
- [`@kindgi/adapter-model-openai-compat`](https://docs.kindgi.com/v0.1.8/reference/packages/adapter-model-openai-compat.md) — for local runners that expose an OpenAI-compatible HTTP endpoint instead.
- [`@kindgi/dev-echo-provider`](https://docs.kindgi.com/v0.1.8/reference/packages/dev-echo-provider.md) — deterministic scripted provider for tests.