# skein-js — full documentation > The open-source alternative to LangGraph Platform for TypeScript: a self-hosted Agent Protocol > runtime for LangGraph.js servers or local SQLite applications, and a drop-in replacement for the LangGraph CLI. > This file concatenates the user-facing docs for wholesale ingestion. It is GENERATED by > scripts/generate-llms-full.mjs — run `pnpm docs:llms` to regenerate. The curated index is llms.txt. # skein-js [![npm](https://img.shields.io/npm/v/skein-js?logo=npm&color=cb3837&label=skein-js)](https://www.npmjs.com/package/skein-js) [![downloads](https://img.shields.io/npm/dm/skein-js?color=blue)](https://www.npmjs.com/package/skein-js) [![license](https://img.shields.io/npm/l/skein-js?color=green)](./LICENSE) [![CI](https://github.com/skein-js/skein-js/actions/workflows/ci.yml/badge.svg)](https://github.com/skein-js/skein-js/actions/workflows/ci.yml) [![docs](https://img.shields.io/badge/docs-skein.js.org-3b82f6)](https://skein.js.org/) **Run LangGraph.js agents on your own infrastructure.** skein-js is an open-source TypeScript runtime for hosting [LangGraph.js](https://docs.langchain.com/oss/javascript/langgraph/overview) agents. LangGraph defines and executes your graphs; Skein provides the API, persistence, streaming, background work, and console around them. ## Why skein-js? A graph that works in a script still needs a backend before people can use it. Conversations need state, long-running work needs a queue, and people need a way to see what happened or approve the next step. Building and maintaining that infrastructure takes time away from your application. Skein supplies that backend. Keep your graph code, models, and tools, then run it as a service on your infrastructure or inside an application you already ship. It is an Apache-2.0 alternative to LangGraph Platform (now LangSmith Deployment), with no commercial license key required. Build a customer-facing assistant, a research tool, or a workflow that waits for human approval. Your existing LangGraph SDK clients and Agent Chat UI connect through the same Agent Protocol: the HTTP and streaming API around your graphs. ## What you get - **Conversation state and long-term memory.** Keep conversation history and recall information across threads. Use Postgres for shared servers or SQLite for local persistence. [State and persistence](./docs/state-and-context.md) · [Memory](./docs/memory.md) - **Streaming with standard LangGraph clients.** Deliver tokens, tool activity, and graph updates as they happen. Connect with the LangGraph SDK and its frontend bindings, including React's `useStream`. [Streaming](./docs/streaming.md) · [Client integration](./docs/react-sdk.md) - **Human approvals and checkpoint history.** Expose LangGraph interrupts to your users, resume after a decision, and explore a different outcome from a saved checkpoint. [Human-in-the-loop](./docs/human-in-the-loop.md) · [Time travel](./docs/threads.md#time-travel-re-run-a-turn-a-different-way) - **Background jobs and schedules.** Let work continue after a request ends, run agents on a schedule, and receive a callback when a run finishes. [Background jobs](./docs/background-jobs.md) · [Schedules](./docs/crons.md) - **Channels and durable callbacks.** Connect authenticated events from your systems to a graph and deliver its outcome to an allowed destination. Inspect and replay webhook deliveries. [Channels](./docs/channels.md) · [Webhooks](./docs/webhooks.md) · [Native access and BullMQ](./docs/native-access.md) - **Console and observability.** Test graphs, inspect conversations, and manage approvals in your own console. Connect LangSmith, PostHog, or OpenTelemetry for tracing and analytics. [Console](./docs/console.md) · [Observability](./docs/observability.md) ## Quick start > 🚧 **Status: pre-alpha, but end-to-end.** Development and self-hosted production work today, with > Fetch, Express, Fastify, NestJS, and Next.js adapters. See the [roadmap](./docs/roadmap.md). Use **Node 24** (recommended), then create a project: ```bash npm create skein-js@latest my-agent cd my-agent npm run dev ``` Accept the scaffolder's defaults to install dependencies and start with an echo graph that needs no API key, database service, or Docker. Open , create a thread, and send a message. The server runs at . [Your first agent](./docs/your-first-agent.md) walks through adding a model and tools, then deploying. For workspace setup and generator options, see [scaffolding](./docs/scaffolding.md). ### Already using LangGraph? Install the CLI in your existing project: ```bash npm install -D skein-js ``` Then change your development script: ```diff - "dev": "langgraph dev" + "dev": "skein dev" ``` Keep your `langgraph.json`, graph exports, and clients. See [LangGraph CLI compatibility](./docs/langgraph-cli-compat.md) for supported commands, fields, and limitations. ## Run it your way Each path uses the same Agent Protocol engine. Choose where the runtime lives: | Path | Use it for | Guide | | --------------------- | ------------------------------------------------------------------------------- | ----------------------------------------------------------------- | | **Standalone server** | A shared agent service with Postgres and Redis, deployed as Node or a container | [Server deployment](./docs/deploy.md) | | **Existing backend** | Graphs mounted alongside your Express, Fastify, NestJS, or Next.js routes | [Embedding](./docs/embedding.md) · [Adapters](./docs/adapters.md) | | **Local application** | A desktop app, CLI, or daemon with one Node process owning a SQLite file | [Local deployment](./docs/local-deployment.md) | The default development server uses memory drivers. Select durable storage when you need state to survive beyond development. Your application chooses its models, credentials, and tools in every path. ## The console The console is served by your own process at `/console`. Send a message in the playground, follow live runs, review pending approvals, and browse saved memory and schedules. ![Testing a graph in the skein console Playground](./docs/public/images/console/playground-light.png) It is enabled during development and **off by default in production**. The [console guide](./docs/console.md) walks through testing your graph and diagnosing a result. Use the [setup reference](./docs/console-reference.md) for authentication and mounting, then turn important cases into [automated project tests](./docs/testing-your-project.md). ## Documentation [Read the docs](https://skein.js.org/) for the complete guides and reference. - [Getting started](./docs/getting-started.md) — adopt Skein with a new project or an existing graph. - [Testing your project](./docs/testing-your-project.md) — graph assertions and a server smoke test. - [Features](./docs/features.md) — find a capability and its guide. - [Agent Protocol](./docs/agent-protocol.md) — API resources and authentication. - [Recipes](./docs/recipes/index.md) — practical patterns for serving and running agents. - [Agent skills](./docs/agent-skills.md) — setup guidance for AI coding assistants. LangChain Deep Agents support is **experimental**: see the [Deep Agents recipe](./docs/recipes/deep-agents.md). For AI tools, [llms.txt](./llms.txt) indexes the docs and [llms-full.txt](./llms-full.txt) bundles them. ## Examples Start with [triage-agent](./examples/triage-agent) for background work, schedules, and approvals without an API key, or [chat-app](./examples/chat-app) for a research assistant with a Next.js UI. The [examples directory](./examples) also includes framework integrations, migration, and local apps. For durable embedded calls and external command queues, use the [native/BullMQ approval example](./examples/native-transports) and [security guide](./docs/native-access-security.md). ## Contributing & feedback [Open an issue](https://github.com/skein-js/skein-js/issues) for bugs and compatibility reports, or see [CONTRIBUTING.md](./CONTRIBUTING.md) to contribute. Changes and releases are in the [changelog](./CHANGELOG.md). ## Sponsorship Sponsored by [Garatropic Studios](https://garatropic.com), supporting engineering and ongoing maintenance. To support development, [get in touch](https://github.com/skein-js/skein-js/issues). ## License [Apache-2.0](./LICENSE) --- ## Why skein-js? {#why-skein-js} skein-js is an open-source TypeScript runtime for hosting LangGraph.js agents. LangGraph defines and executes your graphs; Skein provides the API, persistence, streaming, background work, and console around them. A working graph still needs a backend before people can use it. Skein supplies that infrastructure so you can ship an assistant, research tool, or approval workflow while keeping your graph code, models, and tools. It is an Apache-2.0 alternative to LangGraph Platform (now LangSmith Deployment) that you run on your own infrastructure. ## Quick start {#start} Use **Node 24** (recommended): ```bash npm create skein-js@latest my-agent cd my-agent npm run dev ``` Accept the scaffolder's defaults to install dependencies and try an echo graph that needs no API key, database service, or Docker. Open `http://localhost:2024/console` and send a message. [Your first agent](./your-first-agent.md) takes you through adding a model and tools, then deploying. Skein is **pre-alpha**; see the [roadmap](./roadmap.md) for support and known gaps. ### Already using LangGraph? Install the CLI in your project with `npm install -D skein-js`, then replace `langgraph dev` with `skein dev`. Keep your `langgraph.json`, graph exports, and LangGraph SDK clients. [CLI compatibility](./langgraph-cli-compat.md) documents supported commands, fields, and limitations. ## Run it your way | Path | Use it for | Guide | | --------------------- | ------------------------------------------------------------------------ | ----------------------------------------- | | **Standalone server** | A shared service with Postgres and Redis | [Server deployment](./deploy.md) | | **Existing backend** | Graphs alongside your Express, Fastify, NestJS, or Next.js routes | [Embedding](./embedding.md) | | **Local application** | A desktop app, CLI, or daemon with one Node process owning a SQLite file | [Local deployment](./local-deployment.md) | Each path uses the same Agent Protocol engine. The default development server uses memory drivers; choose durable storage when you need persistence beyond development. ## See it running {#see-it} The console is served by your own process at `/console`. Test a graph, follow live runs, review approvals, and browse memory and schedules. It is enabled in development and off by default in production. [Test your graph in the console](./console.md), then save important cases as [automated project tests](./testing-your-project.md).
![Testing a graph in the skein console Playground](/images/console/playground-light.png)
![Testing a graph in the skein console Playground in dark mode](/images/console/playground-dark.png)
## Where to go next Browse [features](./features.md) for capabilities, [recipes](./recipes/index.md) for practical patterns, or [examples](https://github.com/skein-js/skein-js/tree/main/examples) for runnable projects. Connect traces and analytics with [observability](./observability.md). LangChain Deep Agents support is **experimental**: see the [Deep Agents recipe](./recipes/deep-agents.md). --- # Why skein-js Use skein-js when you have LangGraph.js agents and want to run them on infrastructure you control, with conversations, streaming, background execution, persistence, and a console available together. Your application continues to choose its models, tools, prompts, and access policy. For a support workflow, that might mean keeping a ticket's conversation, looking up its order, waiting for a refund approval, and continuing tomorrow without losing the request. Start with the parts your application needs; you do not have to use every capability. ## Is it a fit? {#who-this-is-for} | You need | What skein-js provides | | ------------------------------------------- | ----------------------------------------------------------------------- | | A server for existing TypeScript graphs | Load your `langgraph.json` or embed compiled graphs in your backend | | Conversations and human approval | Threads, checkpoints, interrupt/resume, and history | | Work after an HTTP request ends | Background runs, schedules, and completion delivery | | A way to test before building a frontend | A local [console](./console.md) for inputs, output, state, and failures | | Durable shared services | Postgres storage, with Redis for cross-instance queues and streaming | | Agents inside a desktop app, CLI, or daemon | SQLite embedding with one Node process owning the file | Keep TypeScript graphs and application code together when that fits your team. If a tool or model requires another language, your graph can call that service; skein-js does not require you to rewrite it. ## Choose how much to adopt {#your-four-options} You can run a LangGraph graph directly when you only need graph execution. Add skein-js when your application needs a hosted protocol, shared state, background work, or an inspection UI. You can also use a [plain graph endpoint](./serving-a-single-graph.md) when threads and runs are unnecessary. Choose an adoption path: - **Existing `langgraph.json`:** replace `langgraph dev` with `skein dev`; follow [Getting started](./getting-started.md#path-b--i-have-a-langgraphjson-drop-in). - **Existing backend:** [embed your graph](./embedding.md) alongside your own application routes. - **New project:** [create your first agent](./your-first-agent.md) and test it without a model key. - **Local application:** [embed with SQLite](./local-deployment.md) and let your host own process lifetime. ## What you operate {#the-same-stack-in-dev-and-production} You deploy and maintain the server or local application, storage, credentials, backups, and application access policy. The default development server needs no database; choose durable storage for deployment. SQLite suits a single local Node owner; Postgres and Redis suit shared services. See [deployment choices](./deploy.md#choose-a-deployment). skein-js is [Apache-2.0 licensed](https://github.com/skein-js/skein-js/blob/main/LICENSE). It requires no skein-js account or license key. You still pay for the infrastructure and model services you choose. If you prefer a provider to operate your deployment, compare managed offerings against your hosting, security, and support requirements using their current terms. ## Keep your existing clients {#we-want-leaving-to-be-easy} Your graphs remain LangGraph.js graphs, and clients use the standard LangChain SDKs. Test an existing client against skein-js before migration; check the [compatibility reference](./langgraph-compatibility.md) for verified versions, supported behavior, and intentional differences. Skein-specific options such as durable delivery need consideration when moving to another host. ## When to choose something else {#probably-not-for-you} If you only need to invoke a graph inside one request, direct LangGraph execution may be enough. If you want a managed control plane, billing system, model gateway, or evaluation service, those remain separate choices. A frozen serverless function cannot continue background work after its request ends; use a long-lived backend for that [deployment pattern](./deploy-serverless.md). ## Try it with your project {#convinced-or-curious} Follow [Getting started](./getting-started.md), run a known input in the [console](./console.md), and add a [repeatable project test](./testing-your-project.md). Verify the behavior your application relies on before choosing a deployment. --- # Your first agent From an empty directory to a deployed agent server. No LangGraph experience assumed — the parts of it you need are explained here, where you need them. If you already have a LangGraph.js graph or a `langgraph.json`, you want [getting-started.md](./getting-started.md) instead: there is nothing to scaffold, and adopting skein is a one-line change. **Prerequisites:** Node 24 is recommended, with Node 22.12+ within 22.x also supported, and a package manager. Nothing in this guide needs an API key until you choose a model, and nothing needs Docker until deployment. For a workspace, start with the [monorepo guidance](./scaffolding.md#nx-and-other-monorepos). ## 1. Create the project ```bash npm create skein-js@latest my-agent ``` `pnpm create skein-js@latest my-agent` and `yarn create skein-js@latest my-agent` do the same thing. Keep `@latest` to request the latest published scaffolder. Interactive runs ask about the model provider, ESLint and Prettier, local storage, dependency installation, and git initialization. **Pick "None" for the provider**, keep memory storage and ESLint/Prettier enabled, and accept dependency installation. The directory is already supplied above; it is prompted when omitted. `--yes` accepts defaults without prompting. You can add a model in a minute. The generated echo graph runs without credentials. Then: ```bash cd my-agent npm run dev ``` You now have an agent server on `http://localhost:2024`. ## 2. Look at it before you read any code Open `http://localhost:2024/console`. That's the **skein console**, served by your own process — no account, no hosted service, no tunnel. In Playground, choose the **echo** Assistant, send "hello", and watch the run execute. You will get `echo: hello` back, because the graph you just created echoes its input. This is worth doing before anything else: everything below is about changing what happens between your message and that reply, and it helps to have seen the loop close. Run the generated graph tests in a second terminal: ```bash npm test ``` Both echo cases should pass without a model key or a running server. As you change the graph, update its behavior assertions; [Testing your project](./testing-your-project.md) adds an SDK smoke test and checks for multi-turn state, persistence, and approval flows. ## 3. What you actually got The project includes tests, default ESLint/Prettier tooling, and a `.nvmrc` selecting a supported Node version. The three files to understand first: **`src/echo-graph.ts`** — your agent. We'll come back to it. **`langgraph.json`** — how skein finds your graphs: ```json { "node_version": "24", "graphs": { "echo": "./src/echo-graph.ts:graph" }, "env": ".env" } ``` `"./src/echo-graph.ts:graph"` means _the export named `graph` in that file_. The key — `echo` — is what clients ask for by name. This is the same file format the LangGraph CLI reads, which is why skein is a drop-in for it. **`package.json`** — the commands that are the whole lifecycle: | | | | ---------------------- | ------------------------------------------------------------- | | `npm run dev` | what you're running: hot reload, in-memory state, zero setup | | `npm run dev:services` | Postgres + Redis in Docker, for the durable `dev` and `start` | | `npm run dev:postgres` | the same hot reload, against those services instead | | `npm run build` | compile your graphs to plain JavaScript in `.skein/` | | `npm start` | serve that build — this is what production runs | (If you picked Postgres at the storage prompt, `dev` is already the durable one and the in-memory spelling is `dev:memory` instead — both are always there.) The rest: `.env` (ready to use — the model key is commented out, the service URIs are not) and `.env.example` (a committed reference copy of it), `compose.dev.yaml` (the services for `start`), `tsconfig.json`, a test, and a README. ## 4. Understanding the graph Here is the whole agent: ```ts import { AIMessage, type BaseMessage } from "@langchain/core/messages"; import { MessagesAnnotation, StateGraph } from "@langchain/langgraph"; function echo(state: typeof MessagesAnnotation.State): { messages: BaseMessage[] } { const last = state.messages.at(-1); const text = typeof last?.content === "string" ? last.content : ""; return { messages: [new AIMessage(`echo: ${text}`)] }; } export const graph = new StateGraph(MessagesAnnotation) .addNode("echo", echo) .addEdge("__start__", "echo") .addEdge("echo", "__end__") .compile(); ``` Four ideas, and they are the only LangGraph concepts you need to get started: **State** is the data flowing through your agent. `MessagesAnnotation` is a ready-made state that holds a list of chat messages and knows to _append_ new ones rather than replace the list. Most conversational agents want exactly this. **A node** is a function. It receives the current state and returns only the part that changed — here, one new message. It is a plain function: you can call it, test it, and step through it in a debugger. **Edges** say what runs next. `__start__` and `__end__` are the built-in entry and exit points, so this graph is "start → echo → done." Real agents branch here — that is where the "graph" part earns its name. **`.compile()`** turns the definition into something runnable. What it returns is what skein serves. Try it: change `echo:` to `you said:` in `src/echo-graph.ts` and save. The server hot-reloads and **keeps your existing threads** — send another message in the console and you will see the new reply in the same conversation. Those four ideas are enough to finish this guide. When you want the rest — custom state, branching, subgraphs, `interrupt()` — [LangGraph essentials](./langgraph-essentials.md) is the short version, with a link out to the LangChain docs on each one. ## 5. Give it a real model Now swap the echo for an LLM. Install a provider: ```bash npm install @langchain/anthropic langchain zod ``` Create `src/agent-graph.ts`: ```ts import { ChatAnthropic } from "@langchain/anthropic"; import { createAgent } from "langchain"; const model = new ChatAnthropic({ model: "claude-sonnet-5", temperature: 0 }); export const graph = createAgent({ model, tools: [] }); ``` `createAgent` is LangChain's prebuilt agent loop: call the model, and if it asks for a tool, run the tool and call the model again. You did not have to build that loop. It lives in the `langchain` package — LangGraph v1 deprecated its own `createReactAgent` and moved it there. Register it in `langgraph.json`: ```json { "graphs": { "echo": "./src/echo-graph.ts:graph", "agent": "./src/agent-graph.ts:graph" }, "env": ".env" } ``` Add your [Anthropic API key](https://console.anthropic.com/settings/keys) to the existing `.env` file: ```bash ANTHROPIC_API_KEY=your-api-key ``` The packages installed above provide the model, `createAgent`, and the tool schema used in the next step. Keep the existing `env` setting in `langgraph.json` when adding the new graph. Restart, pick `agent` in the console, and you are talking to a real model. Tokens stream as they are generated. > Prefer to skip this assembly? `npm create skein-js@latest my-agent -- --provider anthropic` scaffolds > all of it, plus a working tool, in one step. ## 6. Give it a tool Talking is half of it. Tools are what let an agent actually do things: ```ts import { tool } from "@langchain/core/tools"; import { z } from "zod"; const getWeather = tool( async ({ city }: { city: string }) => { const geo = await fetch( `https://geocoding-api.open-meteo.com/v1/search?name=${encodeURIComponent(city)}&count=1`, ).then((response) => response.json()); const place = geo.results?.[0]; if (!place) return `I couldn't find ${city}.`; const forecast = await fetch( `https://api.open-meteo.com/v1/forecast?latitude=${place.latitude}` + `&longitude=${place.longitude}¤t=temperature_2m`, ).then((response) => response.json()); return `It's ${forecast.current.temperature_2m}°C in ${place.name}.`; }, { name: "get_weather", description: "Get the current weather for a city.", schema: z.object({ city: z.string().describe("City name, e.g. 'Nairobi'") }), }, ); ``` Pass it in — `createAgent({ model, tools: [getWeather] })` — and ask "what's the weather in Nairobi?". The console shows the model deciding to call the tool, the result coming back, and the final answer. (Open-Meteo needs no API key of its own.) The `description` and `schema` are how the model knows when to use it. Write them for a reader who has no other context, because that is exactly the model's situation. ## 7. Talk to it from your own code The server speaks the standard Agent Protocol, so the official SDK works — there is no skein client: ```ts import { Client } from "@langchain/langgraph-sdk"; const client = new Client({ apiUrl: "http://localhost:2024" }); const thread = await client.threads.create(); for await (const event of client.runs.stream(thread.thread_id, "agent", { input: { messages: [{ role: "user", content: "what's the weather in Nairobi?" }] }, })) { console.log(event.event, event.data); } ``` For a UI, the `useStream` React hook talks to the same server with only a URL change — see [react-sdk.md](./react-sdk.md). Vue, Svelte and Angular work too; so do [Agent Chat UI](https://github.com/langchain-ai/agent-chat-ui) and LangGraph Studio. ## 8. Remember things between conversations Threads persist a conversation. A **store** persists across conversations — what a user told you last week. Any node can reach it: ```ts import { getStore } from "@langchain/langgraph"; async function remember(state: typeof MessagesAnnotation.State) { const store = getStore(); await store.put(["users", "alice"], "prefers", { units: "celsius" }); const saved = await store.get(["users", "alice"], "prefers"); // …use saved.value in your prompt } ``` The default `dev` uses memory. A server can use Postgres with vector search for semantic recall; a local application can use SQLite with text/filter search. Your node code does not change. See [storage.md](./storage.md) and [memory.md](./memory.md). ## 9. Ship it `dev` is not the production path. Production is `build` + `start`: ```bash npm run dev:services # Postgres + Redis via Docker npm run build # graphs → plain JavaScript in .skein/build npm start ``` Nothing to edit first: the `POSTGRES_URI` and `REDIS_URI` in `.env` already match those services, and `npm start` loads them itself. `build` compiles your TypeScript graphs ahead of time; `start` serves that output with no TypeScript toolchain in the loop. It is exactly what the production container runs. The default `start` selects Postgres and Redis for durable state, queued work, and multiple instances. It is one deployment option. For a desktop app, CLI, daemon, or single Node owner, choose [SQLite](./local-deployment.md): scaffold with `--storage sqlite` to generate SQLite dev/start scripts without Compose. The same graphs, checkpoints, and protocol are used; the application decides whether to retry an abandoned attempt. For a container, `npx skein up` brings up the whole stack, and `npx skein build` produces an image. Then pick a host: [Cloud Run](./deploy-cloud-run.md), [Fly](./deploy-fly.md), [Railway](./deploy-railway.md), [Render](./deploy-render.md), [AWS](./deploy-aws.md), [Kubernetes](./deploy-kubernetes.md), or [a plain VPS](./deploy-vps.md). Full guide: [deploy.md](./deploy.md). ## Where to next **Read [building blocks](./building-blocks.md).** It is the map of everything this guide skipped — checkpoints, threads, multitask, interrupts, memory, what production actually needs — a few lines each, so you can tell which page you want before you need it. If you would rather understand the graph model first, [LangGraph essentials](./langgraph-essentials.md) covers state, reducers, branching, subgraphs and `interrupt()`, and links out to the LangChain docs on each. Then, as you need them: - [Recipes](./recipes/) — auth, human-in-the-loop, background runs, CORS - [The console](./console.md) — test your graph and inspect its output, state, and failures - [Agent Protocol](./agent-protocol.md) — every endpoint your server exposes - [Scaffolding reference](./scaffolding.md) — every flag, and the Nx generators --- # Getting started A guided, end-to-end path to a running Agent Protocol server, then to production. Pick the path that matches what you have right now: - **Nothing yet** → [Path A](#path-a--starting-from-scratch): one command scaffolds a working project. - **A `langgraph.json`** → [Path B](#path-b--i-have-a-langgraphjson-drop-in): a one-line drop-in. - **A compiled graph in code** → [Path C](#path-c--i-have-a-graph-in-code-embed): wrap it in `deps`. All three expose the same Agent Protocol API. For a terse reference instead of a walkthrough, see [using-skein.md](./using-skein.md). Deployment is a separate choice: keep the default memory drivers while developing, use [Postgres/Redis](./deploy.md) for a shared server, or choose [SQLite for a local app or one Node process](./local-deployment.md). The graphs and Agent Protocol remain the same. ## Prerequisites - **New projects (Path A):** Node 24 is recommended. Generated engines support `^22.12.0 || ^24.0.0 || ^26.0.0`. - **Existing graphs (Paths B and C):** Node ≥ 20 and a package manager (`pnpm`/`npm`). - **Node 22, 24 or 26** when selecting SQLite; desktop examples build with Node 24. - For Paths B and C, a **[LangGraph.js graph](https://docs.langchain.com/oss/javascript/langgraph/graph-api)** — a `CompiledStateGraph` from `@langchain/langgraph`. Path A writes one for you. ## Path A — Starting from scratch No graph, no config, no LangGraph experience. One command: ```bash npm create skein-js@latest my-agent cd my-agent npm run dev # → http://localhost:2024, console at /console ``` You get a `langgraph.json`, a keyless echo graph you can edit, a test, and the whole `dev` → `build` → `start` lifecycle wired up. ESLint and Prettier are enabled by default; `--no-linting` opts out. The default memory development server needs no API key or database. [**Your first agent**](./your-first-agent.md) walks through it end to end — what each generated file does, the four LangGraph concepts you need, adding a model and a tool, then deploying. [scaffolding.md](./scaffolding.md) is the flag-by-flag reference, and covers doing it by hand if you would rather not run a scaffolder. ## Path B — I have a `langgraph.json` (drop-in) If you already run `langgraph dev`, this is a one-line change. Keep your `langgraph.json` exactly as it is: ```jsonc // langgraph.json { "graphs": { "agent": "./src/agent.ts:graph" } } ``` Swap the CLI: ```diff - "dev": "langgraph dev", + "dev": "skein dev", ``` ```bash pnpm add -D skein-js pnpm dev # skein dev — in-process, hot-reload, on http://localhost:2024 ``` `skein dev` loads your TypeScript graphs, hot-reloads on save, and restores development state across restarts. Saving a model key in `.env` also reloads the graphs. See the [CLI reference](./langgraph-cli-compat.md) for supported commands, fields, and limitations. Prefer to mount it inside your own Express/Fastify/Nest/Next app instead of the CLI? Point an adapter at the same config — `{ config: "./langgraph.json" }` — using the snippets in [Mount it on your framework](./using-skein.md#mount-it-on-your-framework). ## Path C — I have a graph in code (embed) No `langgraph.json`, no CLI — bring the compiled graph you already hold and wrap it into a `ProtocolDeps` with `embedInMemoryGraphs`, then hand `{ deps }` to any adapter: ```ts // server.ts import { createExpressServer } from "@skein-js/express"; import { embedInMemoryGraphs } from "@skein-js/server-kit"; import { graph } from "./agent.js"; // your CompiledStateGraph const server = await createExpressServer({ deps: embedInMemoryGraphs({ agent: graph }) }); await server.listen(2024); console.log("Agent Protocol on http://localhost:2024"); ``` ```bash pnpm add @skein-js/express @skein-js/server-kit @langchain/langgraph npx tsx server.ts # or your usual TS runner ``` This exposes the same Agent Protocol API. Your host owns development reload and process lifetime; see [embedding](./embedding.md) for graph factories and production drivers. ## Verify your project For CLI paths A and B, open the printed **Console** URL. Select your graph's Assistant in Playground, send a known input, and compare the response with its expected result. A scaffolded **echo** graph returns `echo: hello` when you send `hello`. For structured graphs, choose JSON and follow the input schema. See [Test your graph in the console](./console.md#test-your-graph). Path C does not mount the console automatically. Use the SDK below, or follow the [manual mounting reference](./console-reference.md#mounting-it-yourself). If you scaffolded a project, run `npm test` from its directory: the generated echo tests need neither credentials nor a server. See [Testing your project](./testing-your-project.md) for repeatable graph assertions and an SDK smoke test against your own server. ## Talk to your server With the server running on `:2024`, call it with the standard SDK. Install the client in the calling project if it is not already declared: ```bash pnpm add @langchain/langgraph-sdk ``` The examples below use the scaffold's `echo` Assistant and its `messages` input. For Paths B and C, set `assistantId` to your configured graph key (such as `agent`) and adapt the input to its schema. ```ts import { Client } from "@langchain/langgraph-sdk"; const client = new Client({ apiUrl: "http://localhost:2024" }); const assistantId = "echo"; const thread = await client.threads.create(); const reply = await client.runs.wait(thread.thread_id, assistantId, { input: { messages: [{ role: "user", content: "hello" }] }, }); console.log(reply); ``` Stream tokens as they arrive instead of waiting: ```ts for await (const event of client.runs.stream(thread.thread_id, assistantId, { input: { messages: [{ role: "user", content: "hello" }] }, })) { console.log(event.event, event.data); } ``` Each registered graph has a default Assistant whose `assistant_id` equals its graph key. The full endpoint surface is in [agent-protocol.md](./agent-protocol.md); the stream wire format is in [streaming.md](./streaming.md). ## Add a web UI Follow [the frontend SDK guide](./react-sdk.md#usestream-against-skein-js) to install the React binding and build a UI that displays messages, errors, and cancellation controls. Test it against the same Assistant you used above. A browser on a different origin needs CORS enabled — see the [CORS recipe](./recipes/serving.md#cors-for-a-browser-client). For a same-origin full-stack app (no CORS), the [`nextjs-app`](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-app) example serves the protocol and the UI from one Next.js app. ## Go to production The default dev server uses in-memory drivers. For a durable shared server, choose Postgres for state and checkpoints, and add Redis when workers or streaming clients span instances. For an embedded backend (Path C), install `@skein-js/runtime` before using the durable helper: ```bash pnpm add @skein-js/runtime ``` ```ts // embed path → durable, reading POSTGRES_URI / REDIS_URI from the environment import { embedPostgresGraphs } from "@skein-js/runtime"; import { createExpressServer } from "@skein-js/express"; import { graph } from "./agent.js"; const { deps, dispose } = await embedPostgresGraphs({ agent: graph }); const server = await createExpressServer({ deps }); await server.listen(2024); process.on("SIGTERM", async () => { await server.close(); await dispose(); // release the pools it opened process.exit(0); }); ``` From a `langgraph.json`, either call `buildRuntime({ store: "postgres", queue: "redis" })` and pass its `deps` **and** `channels` to the adapter, or skip code entirely: `skein dev --store postgres --queue redis`, and `skein build` / `skein up` to containerize. Redis is optional for a single instance but required to run more than one. Details: [embedding.md](./embedding.md#going-to-production), [storage.md](./storage.md), [runs-and-redis.md](./runs-and-redis.md), [deploy.md](./deploy.md). ### Or choose SQLite for a local application Scaffold with `npm create skein-js@latest my-app -- --storage sqlite` for SQLite dev/start scripts, or install `@skein-js/local` and the upstream SQLite saver in an existing project: ```bash pnpm add @skein-js/local @langchain/langgraph-checkpoint-sqlite@~1.0.4 pnpm skein dev --store sqlite --queue sqlite ``` The CLI defaults to `.skein/state.sqlite`. For an Electron utility process, a Tauri Node backend, or a CLI using graphs in code, use `embedSqliteGraphs` and let the host own startup and shutdown. See [local deployment](./local-deployment.md) for embedding, examples, and failure accounting. ## Where to next - [Your first agent](./your-first-agent.md) — the from-zero walkthrough, if you took Path A. - [Recipes](./recipes/) — auth, human-in-the-loop, long-term memory, CORS, background runs, deploy. - [Using skein-js](./using-skein.md) — the terse consumer/agent cheat-sheet. - [Examples](https://github.com/skein-js/skein-js/tree/main/examples) — a runnable project per framework and pattern. - [Overview & architecture](./index.md) · [Agent Protocol surface](./agent-protocol.md) --- # Scaffolding a project `create-skein-js` generates a working skein-js project from an empty directory. It is the fastest path from nothing to a running agent server, and it is optional — everything it emits is a file you could write yourself, and the last section shows you exactly that. For a guided walkthrough rather than a reference, start with [your first agent](./your-first-agent.md). ## Quick start ```bash npm create skein-js@latest my-agent ``` ```bash pnpm create skein-js@latest my-agent yarn create skein-js@latest my-agent npx create-skein-js@latest my-agent ``` **Keep the `@latest`** to request the latest published scaffolder instead of relying on a cached version. Node 24 is recommended; generated projects support `^22.12.0 || ^24.0.0 || ^26.0.0`. For an existing workspace, choose a new child directory such as `apps/my-agent`. Use the [Nx generator](#nx-and-other-monorepos) in Nx workspaces. ## What it generates The default storage choice is `memory`. It keeps development simple and supplies Postgres/Redis services for the default production `start`. Choose `--storage sqlite` for a local application; its generated scripts and files differ as described below. ```text my-agent/ ├── langgraph.json Points skein at your graphs; the LangGraph CLI's format ├── package.json The skein CLI lifecycle as scripts ├── tsconfig.json Strict, ESM, bundler resolution ├── .nvmrc Recommended Node version, or a compatible workspace pin ├── eslint.config.mjs Recommended rules, unless inherited or tooling is disabled ├── .prettierrc.json printWidth: 100, unless inherited or tooling is disabled ├── .prettierignore Generated output, runtime state, dependencies and lockfiles ├── vitest.config.ts ├── compose.dev.yaml Postgres + Redis — what `start` needs ├── .env Ready to use, gitignored — never overwritten if one already exists ├── .env.example A committed reference copy of it ├── .gitignore ├── README.md Explains each of these files └── src/ ├── echo-graph.ts Runs with no API key, no network ├── echo-graph.test.ts So `npm test` is green on the first commit └── agent-graph.ts Only with --provider: a ReAct agent with a working tool ``` The scripts are the whole skein CLI lifecycle: | Script | Command | What it does | | -------------- | ------------------------------------------------- | ----------------------------------------------------------- | | `dev` | `skein dev --port 2024` | In-memory drivers, hot reload, state persisted to `.skein/` | | `dev:services` | `docker compose -f compose.dev.yaml up -d --wait` | Postgres + Redis, for the durable `dev` and for `start` | | `dev:postgres` | `skein dev … --store postgres --queue redis` | The same hot reload, against the drivers production uses | | `build` | `skein build --artifact-only` | Graphs → plain JavaScript in `.skein/build` | | `start` | `skein start -c .skein/build/langgraph.json` | Serve that build — the production entrypoint | | `typecheck` | `tsc --noEmit` | | | `test` | `vitest run` | Run the generated echo graph test | | `lint` | `eslint src --max-warnings=0` | ESLint, enabled by default | | `format` | `prettier --write .` | Apply formatting, enabled by default | | `format:check` | `prettier --check .` | Check formatting, enabled by default | Those are the default memory scripts. Pick Postgres and `dev` becomes the Postgres/Redis command, with the in-memory alternative emitted as `dev:memory` instead of `dev:postgres`. `dev:services` uses `--wait` so it returns once both healthchecks pass, not merely once the containers exist — otherwise the very next command races first-boot `initdb`. `start` names the artifact's own `langgraph.json` because `skein start` serves a _build_, not a source project: it wants `schemas.json` beside the config it loads, and that file only exists in `.skein/build`. Run from the project root like this, it also picks up the `POSTGRES_URI` and `REDIS_URI` from the project's `.env` — `skein start` reads a conventional `.env` from its working directory as well as from the config's, because an artifact deliberately carries none of its own (it is the Docker build context). So `dev:services && build && start` works with nothing to edit first. Two deliberate choices worth knowing: - **The default `dev` needs no credential or service.** The `echo` graph is always present and first in `graphs`, so the first request works with an empty `.env` and memory drivers. - **Memory and Postgres projects include `compose.dev.yaml`**, and the `POSTGRES_URI` / `REDIS_URI` in `.env` are live rather than commented out. The default `skein start` selects `--store postgres --queue redis` and requires those services, so a project shipping that `start` script has to ship both the services and the URIs that reach them, or the script is a trap. The values are the ones the generated compose file serves, so there was never anything to decide. ### SQLite projects ```bash npm create skein-js@latest my-app -- --storage sqlite cd my-app npm run dev npm run build npm start ``` SQLite projects install `@skein-js/local`, LangGraph, and its SQLite saver. Both `dev` and `start` select `--store sqlite --queue sqlite`, with the database at `.skein/state.sqlite`; `dev:memory` keeps the memory alternative. There is no Compose file, service-start script, or Postgres/Redis environment configuration. Graphs still build to `.skein/build` and the database stays outside it. Use Node 24 (recommended), Node 22.12+ within 22.x, or Node 26. See [local deployment](./local-deployment.md) for embedding and desktop examples. ## Options ```text create-skein-js [directory] -m, --provider none | google | anthropic | openai (default: prompted, else none) --storage memory | postgres | sqlite (default: prompted, else memory) --pm npm | pnpm | yarn | bun (default: detected) --linting / --no-linting ESLint and Prettier (default: yes) --no-workspace-update Leave workspace registration manual --no-install Skip installing dependencies (else: prompted, default yes) --no-git Skip initializing a git repository (else: prompted) -y, --yes Accept every default; never prompt -f, --force Scaffold into a directory that is not empty -v, --version -h, --help ``` **Passing flags through npm** needs a `--` separator. `pnpm create` and `npx` do not: ```bash npm create skein-js@latest my-agent -- --provider anthropic pnpm create skein-js@latest my-agent --provider anthropic ``` ### `--provider` | | Package added | `.env.example` gains | Emits `agent-graph.ts`? | | ------------------ | ------------------------- | -------------------- | ----------------------- | | `none` _(default)_ | — | — | no | | `google` | `@langchain/google-genai` | `GOOGLE_API_KEY` | yes | | `anthropic` | `@langchain/anthropic` | `ANTHROPIC_API_KEY` | yes | | `openai` | `@langchain/openai` | `OPENAI_API_KEY` | yes | With a provider you also get a ReAct agent wired to a live weather tool that needs no key of its own, so the agent is genuinely runnable the moment you add your model key. Until the key is set, `agent` fails to load naming the variable it wants — a [load-failure block](./errors-and-logging.md#the-load-failure-block), while `echo` keeps serving. `skein dev` watches `.env`, so filling the key in takes effect on save. ### Behaviour you can rely on - **It never hangs unattended.** Prompts appear only when both streams are a TTY, `--yes` was not passed, and `CI` is unset. Otherwise it takes the flag, then the default — safe inside a Dockerfile or a CI job. - **A failed install is not fatal.** Your files are already written; the closing output just adds `install` back to the steps. - **Scaffolding into a fresh clone works.** A directory holding only `.git`, `LICENSE`, editor folders or `.DS_Store` counts as empty, so "create an empty repo, clone it, scaffold into it" needs no `--force`. - **git is skipped inside an existing work tree**, so it never nests a repository in yours — and the closing output says so, rather than leaving you to infer it from a missing `.git`. A failure (git absent, or no `user.email` configured) is reported too, and distinctly: the two used to be the same silent non-event. - **The version is pinned to a matching runtime.** Because every `packages/*` shares one version, `create-skein-js@x.y.z` pins `skein-js@^x.y.z` — the scaffolder and the runtime it scaffolds are always the same release. ## Tooling and Node versions ESLint and Prettier default to enabled in both entrypoints. The CLI asks **Y/n** interactively; `--linting` and `--no-linting` bypass that prompt. `--yes`, CI and non-TTY runs enable tooling. The Nx option `linting` defaults to `true`; use `--linting=false` to opt out. Projects provide `lint`, `format`, and `format:check`. Missing tooling uses recommended JavaScript and TypeScript ESLint rules and Prettier with `printWidth: 100`. Existing root ESLint and Prettier configurations are inherited independently. Inherited tooling uses a generated `scripts/skein-tools.mjs` helper; a standalone project uses the commands shown above. Generated output, dependencies, runtime state and package-manager lockfiles are ignored. Missing tool dependencies are added independently of inherited configuration, and new files follow workspace formatting; see the [workspace tooling and caching](#workspace-tooling-and-caching). The shared policy recommends **Node 24** with engines `^22.12.0 || ^24.0.0 || ^26.0.0`. It derives `.nvmrc` and `langgraph.json.node_version` from the same selection. Compatible numeric workspace pins in `.nvmrc` or `.node-version` are honored. Unsupported numeric pins fail before writing; unresolved aliases such as `lts/*` produce an explanation and fall back to Node 24. Node 26 is an additional supported runtime; production defaults and desktop packaging remain on Node 24. Yarn users on Node 26 must install Corepack separately because it is no longer bundled with Node. ## Nx and other monorepos For Nx **23.2.1 through 23.x**, install the generator at the root: ```bash pnpm add -Dw @skein-js/nx pnpm exec nx g @skein-js/nx:application --directory=apps/my-agent --storage=sqlite pnpm install pnpm exec nx dev apps-my-agent ``` Options are `directory` (required), `provider`, `storage`, and `linting`. The generator supports integrated workspaces (root dependencies) and package workspaces (application dependencies). It extends the TypeScript base, preserves root lifecycle scripts and leaves installation manual. Use `--dry-run` to preview without writing. This replaces issue #19's earlier decision to document manual wiring only; existing projects can retain their targets. Targets use existing `nx:run-commands`. `start` depends on `build`, live targets are uncached, and only `.skein/build` is a build output. SQLite state and WAL/SHM files stay outside cached inputs and outputs. Retain explicit artifact selection and storage flags, for example `skein start -c .skein/build/langgraph.json --store sqlite --queue sqlite`. When a diagnostic suggests a config, keep the original command's other options, including custom `--sqlite-path` settings. For npm, pnpm, Yarn, Bun, Turborepo or Lerna workspaces: ```bash npm create skein-js@latest apps/my-agent ``` The CLI retains that directory, detects the workspace and skips installation, including with `--yes`. Install dependencies from the workspace root after generation. Missing registration is prompted with a yes default; `--yes` accepts and `--no-workspace-update` leaves manual instructions. Existing globs, exclusions, YAML comments, key order, versions and unrelated settings are preserved. A pnpm root without `pnpm-workspace.yaml` gets one containing its existing `package.json` workspace patterns and exclusions, since pnpm ignores `package.json` `workspaces`. Invalid declared workspaces are diagnosed even when another manifest field is invalid. Excluded paths, invalid root metadata, duplicate names and package-manager conflicts fail before writing. Workspace `packageManager` metadata takes precedence over lockfile detection. Compatible root or pnpm catalog ranges are reused, then unanimous compatible sibling ranges. Remaining dependencies use tested defaults and report differences. Integrated Nx workspaces reject root dependency conflicts without changing existing versions. For new child directories in Nx workspaces the CLI directs you to the generator before writing. ### Workspace tooling and caching Scaffold into a new child directory. The CLI rejects an existing workspace root, including with `create-skein-js . --force`, before writing files. Nx workspaces use the application generator. Configuration inheritance and tool dependencies are independent. Existing root ESLint and Prettier configurations stay untouched. A tool declared at the root is reused; otherwise the generated package gets the missing tool dependency (at the root for integrated Nx layouts). Plugins imported by a root configuration remain dependencies of that configuration. Install manually from the workspace root. Both entrypoints format new files with inherited Prettier settings, including `singleQuote` and `semi`. Cached Nx targets include `^default` dependency sources. Build, typecheck and test depend on `^build`, so imported workspace libraries build first and source edits invalidate the application's cache. Keep the workspace's dependency graph accurate and retain the generated SQLite/WAL/SHM exclusions. `start` depends on the application's build; live hosts remain uncached. ## Verify the generated project Start the generated development command, open its console URL, choose **echo**, and send `hello`. Expect `echo: hello` without model credentials. Then run the generated test command (`npm test` for an npm project) to check the echo graph without a server. See [Testing your project](./testing-your-project.md) for an SDK smoke test and persistence checks. Repository validation coverage and publication procedures are contributor topics: [scaffold tests](https://github.com/skein-js/skein-js/blob/main/docs/testing.md#scaffold-consumers) and [publishing](https://github.com/skein-js/skein-js/blob/main/CONTRIBUTING.md#publishing-the-scaffolder). ## If you don't want the scaffolder Nothing here is load-bearing. Two alternatives: **Copy a runnable example.** The [`examples/`](https://github.com/skein-js/skein-js/tree/main/examples) directory has one project per framework and pattern: ```bash npx degit skein-js/skein-js/examples/express-basic my-agent ``` Note that this copies from `main`, which tracks unreleased work: the examples depend on `workspace:*` versions that only resolve inside the monorepo, so you will need to replace those with real version ranges. The scaffolder exists partly to avoid exactly that. **Write the three files yourself.** A skein project is a graph, a `langgraph.json`, and the CLI: ```bash npm install -D skein-js npm install @langchain/core @langchain/langgraph ``` ```ts // src/graph.ts import { AIMessage } from "@langchain/core/messages"; import { MessagesAnnotation, StateGraph } from "@langchain/langgraph"; export const graph = new StateGraph(MessagesAnnotation) .addNode("echo", (state) => ({ messages: [new AIMessage(`echo: ${state.messages.at(-1)?.content}`)], })) .addEdge("__start__", "echo") .addEdge("echo", "__end__") .compile(); ``` ```json // langgraph.json { "node_version": "24", "graphs": { "agent": "./src/graph.ts:graph" }, "env": ".env" } ``` Add `"type": "module"` to your `package.json`, then `npx skein dev`. That is the entire contract — see [langgraph-cli-compat.md](./langgraph-cli-compat.md) for every field it accepts. ## See also - [Your first agent](./your-first-agent.md) — the guided version of all of this - [Getting started](./getting-started.md) — the paths for when you already have a graph - [LangGraph CLI compatibility](./langgraph-cli-compat.md) — every `langgraph.json` field The generated JavaScript client dependency follows the runnable examples: currently `@langchain/langgraph-sdk` `^1.12.7`. React frontends use the dedicated `@langchain/react` package; see [the frontend guide](./react-sdk.md) for thread subscriptions and interrupt responses. --- # Testing your project Use the [console](./console.md) to explore your graph, then save important cases as automated tests. Test graph behavior directly and use a separate smoke test to check that your application can call the running skein-js server. ## Run the generated tests A scaffolded project includes Vitest and `src/echo-graph.test.ts`. From that project's directory: ```bash npm test ``` The keyless echo tests should pass without starting a server or configuring a model. Run them after editing the graph. A failure should show which expected output changed. If your project uses pnpm, Yarn, or Nx, use its existing test command. The commands used to develop the skein-js repository are not prerequisites for testing your own application. ## Assert graph behavior The generated test invokes the exported graph and checks its result: ```ts // src/echo-graph.test.ts import { HumanMessage } from "@langchain/core/messages"; import { describe, expect, it } from "vitest"; import { graph } from "./echo-graph.js"; describe("echo graph", () => { it("echoes the last human message", async () => { const result = await graph.invoke({ messages: [new HumanMessage("hello")] }); expect(result.messages.at(-1)?.content).toBe("echo: hello"); }); it("handles an empty conversation", async () => { const result = await graph.invoke({ messages: [] }); expect(result.messages.at(-1)?.content).toBe("echo: "); }); }); ``` Adapt the input and assertions to your graph's contract: required structured fields, routing choices, tool arguments, expected failures, and state changes. Keep routine tests deterministic by supplying controlled model/tool responses through your graph's existing dependencies. Test live providers separately so credentials, network availability, and model variation do not determine every test run. Choose assertions from the promise your application makes: | Real-world case | Useful assertion | | ------------------------------------- | -------------------------------------------------------------------- | | A customer reports a duplicate charge | The ticket is routed to billing with the correct order reference | | A reviewer rejects a proposed refund | The refund tool is not called | | A user says “use Celsius” | The preference is written to the intended user's memory namespace | | An order service is unavailable | The agent reports the failure without claiming the order was updated | For model-backed output, assert the decision, structured fields, or tool effect you need rather than one exact sentence. Keep exact text assertions for deterministic behavior such as the echo graph. A direct graph test checks your graph logic. It does not check server configuration, authentication, SDK calls, or deployment persistence. ## Check the running server Use this smoke test in a scaffolded project to check the entire request path. It requires the registered **echo** graph. For another project, change the Assistant ID, input, and expected result. Install the standard client if your project does not already declare it: ```bash npm install @langchain/langgraph-sdk ``` Save this as `src/server-smoke.test.ts`. It runs only when `SKEIN_TEST_API_URL` is set, so ordinary `npm test` still works without a server: ```ts import { Client } from "@langchain/langgraph-sdk"; import { expect, it } from "vitest"; const apiUrl = process.env.SKEIN_TEST_API_URL; it.skipIf(!apiUrl)( "serves the echo graph", async () => { const client = new Client({ apiUrl, apiKey: process.env.SKEIN_TEST_API_KEY, }); const thread = await client.threads.create(); try { const result = await client.runs.wait(thread.thread_id, "echo", { input: { messages: [{ role: "user", content: "hello" }] }, }); expect(result).toMatchObject({ messages: [ expect.objectContaining({ content: "hello" }), expect.objectContaining({ content: "echo: hello" }), ], }); const state = await client.threads.getState(thread.thread_id); expect(state.values).toEqual(result); } finally { await client.threads.delete(thread.thread_id); } }, 30_000, ); ``` Start `npm run dev` in one terminal. In another, run the smoke test against the server's API URL: ```bash # macOS / Linux SKEIN_TEST_API_URL=http://localhost:2024 npm test -- src/server-smoke.test.ts ``` ```powershell # PowerShell $env:SKEIN_TEST_API_URL = "http://localhost:2024" npm test -- src/server-smoke.test.ts ``` Expect **one passing smoke test**, rather than a skipped test. It creates a thread, checks both the run result and saved state, and deletes its thread afterward. Use a test server and test credentials; your authentication policy must allow those operations. Set `SKEIN_TEST_API_KEY` if required. If it fails, check the API URL, graph registration, input schema, permissions, and server logs. A successful `/ok` response establishes liveness, but it does not establish that your graph can run. ## Check state across turns and restarts When your application relies on persistence: 1. Run two turns on the same thread and assert that the second sees the first turn's state. 2. Use a new thread and check that conversation state starts fresh. 3. With your intended storage configuration, stop and restart the server, then read the original thread and its checkpoints. Compare with the values recorded before stopping. 4. If you use long-term memory, read the saved item from another thread in the same authorized namespace. The smoke test above deletes its thread; use a separate fixture for a restart test and clean it up once verification finishes. For durable deployment, test [SQLite](./local-deployment.md) or [Postgres](./storage.md) as configured in your application. `skein dev` restores development snapshots; that is different from proving a production deployment's durability. SQLite marks abandoned attempts as failures and leaves recovery decisions to your application. ## Check interrupt and resume If your graph calls `interrupt()`, use a fresh test thread to reach that pause, assert the requested value, and resume with the response your graph expects. Verify both approval and rejection paths on separate threads. See the [interrupt/resume example](./human-in-the-loop.md#resume-it) for the SDK calls. Check that reopening the conversation does not execute the paused action, and that resuming performs the intended action only when authorized. Use controlled tools or test destinations for side effects. ## Before you ship Run graph tests on each change, the server smoke test against your built application, and the [deployment verification](./deploy.md#verify-a-deployment) against your test deployment. Keep live-model quality checks separate from these functional checks; use your evaluation tooling for variable model output. --- # Agent skills for building with skein-js The [Agent Skills](https://agentskills.io/specification) in this repository guide coding agents through building a LangGraph.js project with skein-js. They are optional: skein-js works without an AI coding agent or an installed skill. | Skill | Use it for | | ------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------- | | [`skein-js`](https://github.com/skein-js/skein-js/tree/main/skills/skein-js) | Scaffolding or embedding graphs in a server, desktop app, CLI, or daemon | | [`skein-js-nx`](https://github.com/skein-js/skein-js/tree/main/skills/skein-js-nx) | Nx application generator, workspace tooling, dependency inputs, and runtime-state cache boundaries | | [`skein-js-turborepo`](https://github.com/skein-js/skein-js/tree/main/skills/skein-js-turborepo) | Turbo tasks, storage selection, build outputs, and runtime-state cache boundaries | ## Server or local application The skills cover the same LangGraph execution and Agent Protocol in either deployment. Choose SQLite explicitly for a desktop app, CLI, daemon, or one Node owner; choose Postgres/Redis for a shared server. New projects recommend Node 24 and enable ESLint/Prettier by default; compatible workspace pins and explicit tooling opt-outs are preserved. The local guidance covers `--storage sqlite`, `embedSqliteGraphs`, host-owned startup/shutdown, and persistent state after reopening. Workspace skills keep live SQLite databases and their WAL/SHM files out of build caches while preserving the application's build artifacts. Models, tools, credentials, and decisions to retry or resume remain application choices. Start with the [local deployment guide](./local-deployment.md) and its Electron, Tauri, and CLI examples. Workspace guidance covers independent configuration and tool dependency inheritance, generated formatting, and safe child-directory generation. Nx guidance preserves dependency cache inputs and build ordering. See [workspace tooling and caching](./scaffolding.md#workspace-tooling-and-caching). ## Find and install a skill The skills live in this repository's `skills/` directory. With a recent [GitHub CLI that supports `gh skill`](https://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-cloud-agent/add-skills), inspect and install one into your project: ```bash gh skill preview skein-js/skein-js skein-js@main gh skill install skein-js/skein-js skein-js@main --agent codex ``` Replace `codex` with `cursor`, `github-copilot`, or `claude-code` to install for that agent. Add `--scope user` to make it available across your projects; the default scope is the current project. Install a workspace skill the same way, substituting its name. `@main` selects the current skill even before the next skein-js release; use `@` to select a release instead. Run `gh skill update` later to get changes from this repository. You can also copy a skill directory into your agent's project skill location. Codex, Cursor, and GitHub Copilot read `.agents/skills//SKILL.md`; Claude Code reads `.claude/skills//SKILL.md`. Copy the whole directory, including any supporting files. See each agent's documentation for user-wide installation paths. Once installed, the agent sees the skill's name and description and may select it for a matching task. To select it explicitly, ask it to use the `skein-js` skill while describing the project you want. A skill checked into this repository is available to agents working **on skein-js**; installing it in your own project makes it available to agents building **with skein-js**. Upgrading an npm package does not refresh separately installed or copied skills. Update those from this repository too; `@main` follows current guidance, while `@` selects that release's snapshot. --- # Using skein-js in your app A dense, task-oriented cheat-sheet for **consuming** skein-js — pick a framework, stand up an Agent Protocol server around your LangGraph.js graph, and call it. Written to be skim-friendly for humans _and_ for AI coding agents building on skein. (Working _on_ skein itself? See [AGENTS.md](https://github.com/skein-js/skein-js/blob/main/AGENTS.md).) ## The one thing to know: the `{ config } | { deps }` seam Every framework adapter takes the **same options bag** (`SkeinRuntimeOptions`). You choose one of two inputs, plus optional common fields: ```ts // EITHER: let skein build an in-memory runtime from a langgraph.json (dev / zero-setup) { config: "./langgraph.json", importModule? } // OR: bring your own assembled ProtocolDeps (production drivers, custom auth, in-code graphs) { deps } // plus common: { logger?, cors?, warm? } ``` `config` loads the configured graphs with in-memory drivers. `deps` uses the runtime you assembled: Postgres/Redis for a server, SQLite for a local app, or an in-code graph map. Both expose the same protocol API. The adapter's `config` option does not watch files or save development snapshots; use `skein dev` for that workflow, or let your host manage reload and persistence. ## Install Starting from nothing? `npm create skein-js@latest my-agent` writes the project for you and you can skip this section. Otherwise pick your framework adapter; `@langchain/langgraph` is always a peer dependency (bring your graph). ```bash pnpm add @skein-js/express @langchain/langgraph # Express pnpm add @skein-js/fastify @langchain/langgraph # Fastify pnpm add @skein-js/nestjs @langchain/langgraph # NestJS pnpm add @skein-js/nextjs @langchain/langgraph # Next.js ``` For server drivers add `@skein-js/runtime` (assembles Postgres/Redis). For local SQLite graphs, use `@skein-js/local` with the upstream SQLite saver. Prefer the CLI on-ramp? `pnpm add -D skein-js` and run `skein dev` — a drop-in for `langgraph dev`. ## Three on-ramps **A — You have nothing yet.** Scaffold a working project — a `langgraph.json`, a keyless graph, a test, and the `dev`/`build`/`start` lifecycle: ```bash npm create skein-js@latest my-agent ``` See [scaffolding.md](./scaffolding.md), or [your-first-agent.md](./your-first-agent.md) for the walkthrough. **B — You have a `langgraph.json`** (or use the LangGraph CLI today). Change one script and keep the config unchanged: ```diff - "dev": "langgraph dev", + "dev": "skein dev", ``` Or point an adapter at the config: `{ config: "./langgraph.json" }`. See [langgraph-cli-compat.md](./langgraph-cli-compat.md). **C — You have a compiled graph in code** (no config, no CLI). Wrap it into `deps` and pass `{ deps }`: ```ts import { createExpressServer } from "@skein-js/express"; import { embedInMemoryGraphs } from "@skein-js/server-kit"; import { graph } from "./my-graph.js"; const server = await createExpressServer({ deps: embedInMemoryGraphs({ agent: graph }) }); await server.listen(2024); ``` `embedInMemoryGraphs(graphs, { overrides? })` builds a `ProtocolDeps` (store, queue, bus, checkpointer). See [embedding.md](./embedding.md). ## Mount it on your framework Each adapter ships a **standalone** server (`create*Server`) and an **embed-alongside-your-app** path. All accept the `{ config } | { deps }` seam above. ```ts // Express — standalone, or skeinRouter({...}) to mount on an existing app import { createExpressServer } from "@skein-js/express"; const server = await createExpressServer({ config: "./langgraph.json" }); await server.listen(2024); // Fastify — standalone, or app.register(skeinPlugin, { prefix: "/agent", config }) import { createFastifyServer } from "@skein-js/fastify"; await (await createFastifyServer({ config: "./langgraph.json" })).listen(2024); // NestJS — imports: [SkeinModule.forRoot({ config: "./langgraph.json" })] import { createNestServer } from "@skein-js/nestjs"; await (await createNestServer({ config: "./langgraph.json" })).listen(2024); // Next.js — App Router catch-all: app/api/[...path]/route.ts import { createSkeinRouteHandlers } from "@skein-js/nextjs"; export const runtime = "nodejs"; export const { GET, POST, PUT, PATCH, DELETE, OPTIONS } = createSkeinRouteHandlers({ deps }); ``` Each adapter has a **standalone** entry (a dedicated graph server) and an **embed-alongside-your-app** entry, each with a runnable example: | Framework | Package | Standalone (dedicated server) | Embed in an existing app | Examples | | --------- | ------------------- | --------------------------------------- | --------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Express | `@skein-js/express` | `createExpressServer` | `skeinRouter` (mount the `Router`) | [express-basic](https://github.com/skein-js/skein-js/tree/main/examples/express-basic), [embed-graph](https://github.com/skein-js/skein-js/tree/main/examples/embed-graph) | | Fastify | `@skein-js/fastify` | `createFastifyServer` | `skeinPlugin` (`register` under a `prefix`) | [fastify-basic](https://github.com/skein-js/skein-js/tree/main/examples/fastify-basic), [fastify-app](https://github.com/skein-js/skein-js/tree/main/examples/fastify-app) | | NestJS | `@skein-js/nestjs` | `createNestServer` | `SkeinModule.forRoot` (import it) | [nestjs-basic](https://github.com/skein-js/skein-js/tree/main/examples/nestjs-basic), [nestjs-app](https://github.com/skein-js/skein-js/tree/main/examples/nestjs-app) | | Next.js | `@skein-js/nextjs` | — (the route handlers _are_ the server) | `createSkeinRouteHandlers` (App Router) · `createSkeinPagesHandler` (Pages) | [nextjs-app](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-app), [nextjs-basic](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-basic) | [`embed-graph`](https://github.com/skein-js/skein-js/tree/main/examples/embed-graph) is the framework-agnostic **in-code** pattern (`embedInMemoryGraphs` + `{ deps }`, no `langgraph.json`) shown on the Express adapter; the same `deps` works with any adapter above. [`react-usestream`](https://github.com/skein-js/skein-js/tree/main/examples/react-usestream) is a browser frontend for any of them. See [Expand your setup](#expand-your-setup) to grow from here. ### Where to point your client when embedding The protocol is served wherever you mount it, so set the client's `apiUrl` to the **mount root**, not the server root. How you set the mount differs per adapter: ```ts // Express — mount the router under a path const { router } = await skeinRouter({ deps }); app.use("/agent", router); // → apiUrl: http://localhost:2024/agent // Fastify — the plugin is encapsulated, so `prefix` isolates skein's routes + CORS await app.register(skeinPlugin, { prefix: "/agent", deps }); // → .../agent // NestJS — no skein-side option: it follows your app's global prefix app.setGlobalPrefix("api"); // → apiUrl: http://localhost:2024/api // Next.js — the catch-all's location, via `basePath` (defaults to "/api") // app/api/[...path]/route.ts → apiUrl: http://localhost:3000/api ``` Mount at the root (no prefix) and `apiUrl` is just the server root. NestJS is the odd one out: it reads the mount from the framework rather than from an argument you pass, so there is no skein-side option to keep in sync with `setGlobalPrefix`. ## Local applications with SQLite Use `embedSqliteGraphs` from `@skein-js/local` for an Electron utility process, a packaged Tauri Node backend, a CLI, or a separate daemon. Supply an absolute application-data path, then use the protocol service directly or pass `deps` to an existing adapter. The host owns process lifetime. On the config path, install the local package and saver, then select both SQLite drivers: ```bash skein dev --store sqlite --queue sqlite skein start -c .skein/build/langgraph.json --store sqlite --queue sqlite ``` Both default to `.skein/state.sqlite`; `--sqlite-path` selects a different file. SQLite has one owner per database. Unclaimed jobs persist, but abandoned attempts become failures rather than replaying input. See [local deployment](./local-deployment.md) for the full setup and examples. ## Shared server deployment (Postgres + Redis) {#go-to-production-postgres--redis} Swap the in-memory `deps` for durable ones — everything else stays the same. Two ways: ```ts // In code: durable deps around graphs you hold (reads POSTGRES_URI / REDIS_URI) import { embedPostgresGraphs } from "@skein-js/runtime"; import { createExpressServer } from "@skein-js/express"; const { deps, dispose } = await embedPostgresGraphs({ agent: graph }); const server = await createExpressServer({ deps }); await server.listen(2024); process.on("SIGTERM", () => dispose().then(() => process.exit(0))); ``` ```ts // From a langgraph.json: pick drivers explicitly import { buildRuntime } from "@skein-js/runtime"; const rt = await buildRuntime({ configPath: "./langgraph.json", store: "postgres", queue: "redis", }); const server = await createExpressServer({ deps: rt.deps, channels: rt.channels, cors: rt.cors, }); ``` Redis is optional but **required to run more than one instance** (the in-memory queue is process-local). Or skip the code entirely: `skein dev --store postgres --queue redis`, and `skein build` / `skein up` for a container. See [embedding.md](./embedding.md#going-to-production), [storage.md](./storage.md), [runs-and-redis.md](./runs-and-redis.md). ## Call the server Any Agent Protocol client works — no custom SDK. The two you'll reach for: ```ts // Node / server-to-server — @langchain/langgraph-sdk import { Client } from "@langchain/langgraph-sdk"; const client = new Client({ apiUrl: "http://localhost:2024" }); const thread = await client.threads.create(); const input = { messages: [{ role: "user", content: "hello" }] }; const reply = await client.runs.wait(thread.thread_id, "agent", { input }); for await (const ev of client.runs.stream(thread.thread_id, "agent", { input })) console.log(ev); ``` ```tsx // Browser — @langchain/react useStream (SSE) import { useStream } from "@langchain/react"; const thread = useStream({ apiUrl: "http://localhost:2024", assistantId: "agent" }); thread.submit({ messages: [{ role: "user", content: "hello" }] }); ``` `assistantId` defaults to the `graph_id`. See [react-sdk.md](./react-sdk.md), [streaming.md](./streaming.md). ## Endpoint surface skein implements the standard Agent Protocol REST + SSE contract, so the SDK maps onto it directly. The resources: **assistants** (a served graph + its schemas), **threads** (persistent conversations), **runs** (`/runs/wait`, `/runs/stream`, background `/threads/{id}/runs` with join + cancel), and a long-term **store** (`/store/items`, semantic `/store/items/search`). The full endpoint inventory and the auth route→permission map live in [agent-protocol.md](./agent-protocol.md). ## Package → import map | You want to… | Import | From | | ----------------------------------------------- | ------------------------------------------ | -------------------------- | | Serve on Express / Fastify / Nest / Next | `create*Server` / `skein*` / `SkeinModule` | `@skein-js/` | | Embed a graph in code (in-memory) | `embedInMemoryGraphs` | `@skein-js/server-kit` | | Embed a graph in code (durable Postgres) | `embedPostgresGraphs` | `@skein-js/runtime` | | Assemble prod deps from a `langgraph.json` | `buildRuntime` | `@skein-js/runtime` | | Implement a storage driver / handle edge errors | `SkeinStore`, `SkeinHttpError` | `@skein-js/core` | | Put skein on a framework we don't ship | `skeinRoutes`, `createProtocolRuntime` | `@skein-js/agent-protocol` | ## Expand your setup Grow from the minimal server without rewrites — each step changes one thing: - **Add another graph.** Add an entry to the graph map (`embedInMemoryGraphs({ echo, agent })`) or to `langgraph.json`'s `graphs`. Each becomes an assistant, addressed by its `graph_id`. - **Standalone → embedded.** Move from a dedicated `create*Server` to the embed entry for your framework (`skeinRouter` / `skeinPlugin` / `SkeinModule.forRoot` / `createSkein*Handlers`) — see the adapter table above — to serve the protocol next to your existing routes, under a prefix if you want. - **Go durable / scale out.** Swap the in-memory `deps` for `embedPostgresGraphs(...)` or `buildRuntime({ store: "postgres", queue: "redis" })`. Add Redis to run more than one instance. See [Go to production](#go-to-production-postgres--redis). - **Drain more background runs at once.** Each instance executes 10 queued runs concurrently by default — tune it with `skein dev --concurrency 4`, `SKEIN_RUN_CONCURRENCY=4`, or `worker: { maxConcurrency: 4 }` on any adapter. See [run concurrency](./runs-and-redis.md#run-concurrency). - **Add auth, memory, HITL, webhooks.** These are drop-in — see the [recipes](./recipes/) (custom auth, `getStore()` long-term memory, interrupt/resume, run-completion webhooks). - **A framework we don't ship.** The adapters are thin shims over one transport-neutral handler table (`createProtocolRuntime` + `skeinRoutes`); put skein on any Node HTTP framework by writing ~40 lines of request/response mapping. See [building-an-adapter.md](./building-an-adapter.md). ## Gotchas - **Auth is off by default.** No `auth` block / no `auth` dep → the server is fully open, exactly like `langgraph dev`. Turn it on with a `@langchain/langgraph-sdk/auth` `Auth` instance — see the [provider recipes](./recipes/authentication.md) and [production recipe](./recipes/production.md#custom-auth). - **CORS is off by default.** Browser clients on another origin need `http.cors` in `langgraph.json` (or the `cors` option). Same-origin (e.g. Next.js) needs nothing. - **A long-lived Node process** is required for the background run worker and in-memory drivers — fine on a normal server / `next start`; for serverless, use Postgres + Redis. - **`useStream` needs an absolute URL** — pass `` `${window.location.origin}/api` ``, not a bare `/api`. - **Bundling skein yourself** (rspack/webpack/esbuild, or an unusual Next.js config)? skein is ESM-only but `require()`-resolvable, and `@langchain/langgraph-api` + `@typescript/vfs` must stay external. See [bundling.md](./bundling.md). - **404s on every protocol path?** You're almost certainly pointing at the wrong root — the protocol lives at your **mount path**, not the server root (see [Where to point your client](#where-to-point-your-client-when-embedding)). On NestJS that means `app.setGlobalPrefix("api")` moves it to `/api/threads`. Two red herrings worth ruling out: an `Unsupported route path: "/api/*"` warning in a NestJS boot log is Nest auto-converting the adapter's catch-all and is harmless, and `/info` isn't part of the surface — a 404 there is correct. ## Go deeper - [Getting started](./getting-started.md) — the guided, end-to-end walkthrough. - [Recipes](./recipes/) — auth, human-in-the-loop, long-term memory, CORS, background runs, deploy. - [Overview & architecture](./index.md) · [Agent Protocol surface](./agent-protocol.md) · [Embedding](./embedding.md) · [Building a custom adapter](./building-an-adapter.md) --- # Framework adapters An adapter is a thin transport shim: it takes the framework-agnostic handler table from `@skein-js/agent-protocol` and puts it on Express, Fastify, NestJS, Next.js, or the Web Fetch API. The engine, the endpoints and the wire format are identical on all five — what differs is how you mount it, and a handful of host-framework details this page exists to spell out. Every adapter takes the **same options bag**: `{ config }` to build a runtime from a `langgraph.json`, or `{ deps }` to bring [your own assembled `ProtocolDeps`](./embedding.md). That seam is the whole API. Picking a different adapter changes neither. ## Pick one Each adapter has a **standalone** entry — a dedicated agent server — and an **embed** entry that mounts the protocol inside an app you already run: | Adapter | Standalone | Mount into your app | Example | | ------------------- | ------------------------ | ------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `@skein-js/express` | `createExpressServer` | `skeinRouter` → `app.use()` | [express-basic](https://github.com/skein-js/skein-js/tree/main/examples/express-basic) | | `@skein-js/fastify` | `createFastifyServer` | `skeinPlugin` under a `prefix` | [fastify-basic](https://github.com/skein-js/skein-js/tree/main/examples/fastify-basic) · [fastify-app](https://github.com/skein-js/skein-js/tree/main/examples/fastify-app) | | `@skein-js/nestjs` | `createNestServer` | `SkeinModule.forRoot()` | [nestjs-basic](https://github.com/skein-js/skein-js/tree/main/examples/nestjs-basic) · [nestjs-app](https://github.com/skein-js/skein-js/tree/main/examples/nestjs-app) | | `@skein-js/nextjs` | — (the routes _are_ it) | `createSkeinRouteHandlers` / Pages handler | [nextjs-app](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-app) · [nextjs-basic](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-basic) | | `@skein-js/fetch` | `createSkeinFetchServer` | — (no mount export) | — (select with `skein build --runtime`) | Express is the default: it is what `skein dev` and the production image run, and what the other docs use in examples. Reach for another when you already have an app on it. ## What is identical everywhere The full endpoint surface (`/threads`, `/assistants`, `/runs`, `/store`, `/crons`), SSE streaming with the same frames, human-in-the-loop, auth, and every client — `useStream`, Agent Chat UI, the LangGraph SDK — work against any of them with only a URL change. A `ProtocolDeps` you assembled for one adapter mounts on any other unchanged. ## What differs This is the part worth reading before you pick: | | Express | Fastify | NestJS | Next.js | Fetch | | ------------------ | --------------------------- | ------------------------------ | ------------------------------------- | --------------------------- | ---------------------------------- | | **Mount path** | `app.use("/agent", …)` | `register({ prefix })` | `app.setGlobalPrefix()` only | `basePath` (default `/api`) | `basePath` (default `""`) | | **CORS** | bundled `cors` package | optional peer `@fastify/cors` | skein's middleware, skein routes only | built in | built in | | **JSON body cap** | 100 kb, via `json.limit` | Fastify's `bodyLimit` (1 MB) | the host's parser, else unbounded | unbounded | 100 kb, via `maxBodyBytes` | | **Logger default** | none — stays silent | `fastify.log` (pino) | Nest's `Logger` | none — stays silent | none — stays silent | | **`/ok` route** | standalone only | standalone only | standalone only | **never** | always | | **Shutdown** | `close()` drains the worker | `close()`, or plugin `onClose` | needs `enableShutdownHooks()` | **none** — nothing drains | `close()` after the listener stops | Two of these bite most often. **The body cap is not uniform** — only Express and Fetch impose one of their own, so on NestJS with `bodyParser: false`, and on Next.js, a large body is read into memory before auth runs. And **only Express and Fetch drain cleanly by default**: a Next.js deploy has no shutdown path at all, because the runtime lives on `globalThis` for the life of the process. ## Express The default, and the reference implementation every other adapter is checked against. ```ts import { createExpressServer, skeinRouter } from "@skein-js/express"; // Standalone — a dedicated agent server with a /ok probe: const server = await createExpressServer({ config: "./langgraph.json" }); await server.listen(2024); // Or mounted on an app you already run. `skeinRouter` is async — it seeds assistants // and starts the run worker before returning, so the router is ready to serve: const { router } = await skeinRouter({ deps }); app.use("/agent", router); ``` Express strips the mount path for you, so nothing needs to be kept in sync. Two options no other adapter has: `json.limit` (default 100 kb — a limit the parser can't understand is rejected at mount time rather than silently becoming unlimited) and `requestLog`. **It logs nothing by default.** Express owns no logger, and a library shouldn't decide to start writing to its host's stdout — pass `createConsoleLogger()` if you want output. ## Fastify ```ts import { createFastifyServer, skeinPlugin } from "@skein-js/fastify"; // Standalone: const server = await createFastifyServer({ config: "./langgraph.json" }); await server.listen(2024); // Embedded — encapsulated, so skein's hooks don't leak into your app: await app.register(skeinPlugin, { prefix: "/agent", config: "./langgraph.json" }); ``` Three things to know. **CORS needs an optional peer**: `@fastify/cors` is imported lazily and only when CORS is on, so enabling it without installing the package throws at registration rather than failing a request later. The plugin **replaces the JSON content-type parser** so an empty body sent with `Content-Type: application/json` is read as `{}` — Fastify's default parser 400s on it where Express does not, and the adapters have to agree. And SSE **hijacks the reply** and writes to the raw socket, which is why a client disconnect is detected there rather than on the request. There is no `json.limit`; Fastify's own `bodyLimit` applies. > [!WARNING] > **`createFastifyServer` ignores `http.disable_*`.** The standalone server does not forward the > route table that a `langgraph.json`'s disable flags produce, so it mounts the full protocol > surface even when the config asks for less. `skeinPlugin` honours them correctly, as do the other > four adapters. This is a bug, not a design choice — until it is fixed, use the plugin if you rely > on those flags. ## NestJS ```ts import { SkeinModule } from "@skein-js/nestjs"; @Module({ imports: [SkeinModule.forRoot({ config: "./langgraph.json" })] }) export class AppModule {} ``` NestJS is the odd one out on mounting: it reads its prefix from the framework via `app.setGlobalPrefix()` rather than from an argument you pass, so there is no skein-side option that can drift out of sync — but also no way to mount it somewhere else. Three consequences that look like bugs and aren't: - **`GET /api` 404s**, and so does `/info`. Nest never routes the bare prefix root to middleware, and no skein route lives at `/`. - **`SkeinModule` registers no `/ok`.** Only `createNestServer` does. Point health checks at a route you own, or use the standalone server. - **A log line about an unsupported route path** during boot is harmless. **Call `app.enableShutdownHooks()`** when embedding. skein stops the run worker on `beforeApplicationShutdown` so in-flight SSE streams settle before the server closes, and without the hooks that never fires. The standalone server does it for you. CORS is applied by skein's own middleware, scoped to skein's routes, so it behaves identically standalone and embedded — `app.enableCors()` governs your routes, not these. ## Next.js There is no standalone entry: the route handlers **are** the server, same-origin with your UI. ```ts // app/api/[...path]/route.ts — App Router import { createSkeinRouteHandlers } from "@skein-js/nextjs"; export const runtime = "nodejs"; // the run worker needs a long-lived process, not the edge export const { GET, POST, PUT, PATCH, DELETE, OPTIONS } = createSkeinRouteHandlers({ deps }); ``` ```ts // pages/api/[...path].ts — Pages Router import { createSkeinPagesHandler } from "@skein-js/nextjs"; // externalResolver tells Next this route settles the response itself, which silences the // "API resolved without sending a response" warning that SSE streams otherwise trigger. export const config = { api: { bodyParser: true, externalResolver: true } }; export default createSkeinPagesHandler({ deps }); ``` `basePath` defaults to `/api` and must match where you put the catch-all. The runtime is memoized on `globalThis`, keyed by config path or `deps` identity, so two route files sharing a config share one runtime — which is why a handler's own `logger` option outranks the shared runtime's, rather than whichever handler served the first request deciding for both. **This needs a warm process.** The in-memory drivers and the background run worker don't survive a function that scales to zero — a serverless deploy needs the Postgres store and Redis queue. See [deploy-serverless.md](./deploy-serverless.md). ## Fetch (Bun / Deno) The web-standard transport, and the production path on Bun and Deno. `skein build --runtime bun|deno` selects it for you. ```ts import { createSkeinFetchServer, startBunServer } from "@skein-js/fetch"; const skein = await createSkeinFetchServer({ config: "./langgraph.json" }); const listener = startBunServer(skein, { port: 2024 }); // Stop accepting connections first, then drain: await listener.stop(); await skein.close(); ``` It is the only adapter that caps request bodies itself — `maxBodyBytes`, default 100 kb — and it has to: `Bun.serve` defaults to 128 MB and `Deno.serve` has no limit at all, and the body is read before auth runs, so an unbounded read is an unauthenticated way to exhaust memory. `/ok` is always served, `basePath` defaults to `""`, and there is **no mount export and no invoke surface** — the handler is a whole-server `fetch` function. Composing it into a host router is yours to do. ## What you must get right - **`skeinRouter` is async.** `skeinRouter({ deps }).router` is `undefined`; await it first. It seeds assistants and starts the run worker before returning. - **Auth handlers that match on a path are not portable between adapters.** Express and NestJS report the full mount-inclusive URL, while Next.js and Fetch report it with the mount prefix stripped — so a handler matching `/threads` sees that on one and `/api/threads` on another. Match on the protocol-relative suffix, or keep the handler adapter-specific. - **Auth is off by default on the `{ deps }` path**, whichever adapter you mount it on. See the warning in [embedding.md](./embedding.md). - **Only the standalone servers add `/ok`** — except Next.js, which never does, and Fetch, which always does. Check the table above before pointing a platform health check at it. - **No adapter serves the console.** `http.console` is honoured by `skein start`'s Node runtime, which mounts it for you; on Bun/Deno the flag warns and does nothing. Mount it into your own app yourself, whichever adapter you are on — see [console.md](./console.md#mounting-it-yourself). ## See also - [Using skein-js](./using-skein.md) — the terse cheat-sheet version of this page - [Embedding a graph](./embedding.md) — the `{ deps }` seam every adapter accepts - [Errors & logging](./errors-and-logging.md#what-each-adapter-does-by-default) — logger defaults in full - [A graph as a plain endpoint](./serving-a-single-graph.md) — the invoke surface, per adapter - [Building an adapter](./building-an-adapter.md) — putting the handler table on a framework we don't ship --- # LangGraph CLI compatibility The `skein` CLI is a **drop-in replacement for the LangGraph CLI** — the migration path onto skein-js for anyone already using it. That means two things: read the same `langgraph.json`, and mirror the same command surface. ## Command mapping | LangGraph CLI | skein-js | Behavior | | ---------------------- | ------------------ | -------------------------------------------------------------- | | `langgraph dev` | `skein dev` | In-process dev server, hot reload, **no Docker**. Local state. | | `langgraph up` | `skein up` | Docker Compose stack (app + **Postgres + Redis**). | | `langgraph build` | `skein build` | Build a deployable Docker image from the config. | | `langgraph dockerfile` | `skein dockerfile` | Emit a standalone Dockerfile from the config. | | `langgraph deploy` | — | Out of scope (hosted-platform push). | Shared flags where sensible: `--port`, `--host`, `--no-reload`, `--config`, `-n, --n-jobs-per-worker`. Production commands (`up`, `build`, `dockerfile`, and the generated `start`) also accept `--runtime node|bun|deno` and `--runtime-version`. Precedence is CLI flag, then `skein.runtime` in config, then Node 24 LTS. Legacy `node_version` remains the Node version fallback. **skein-only:** `skein start` serves a pre-built `.skein/build` artifact — plain compiled JS, no vite, no reload. It is the production image's entrypoint, and it is also how you run skein **without a container**: `skein build --artifact-only` writes the artifact on any machine with Node, and `skein start` serves it from a systemd unit or any process manager. See [Without Docker](./deploy.md#without-docker). The LangGraph CLI has no equivalent — it transforms TypeScript at runtime inside the image. **skein-only dev flags** (beyond the LangGraph CLI) — run `skein dev` against production-shaped storage without Docker: | Flag | Values | Default | Notes | | ---------------------- | ------------------------------ | --------------------- | ----------------------------------------------------------------------------------- | | `--store ` | `memory`, `postgres`, `sqlite` | `memory` | Postgres selects `PostgresSaver`; SQLite selects the upstream SQLite saver. | | `--queue ` | `memory`, `redis`, `sqlite` | `memory` | Redis provides the shared queue/bus; SQLite persists jobs in the resource database. | | `--sqlite-path ` | File path | `.skein/state.sqlite` | Resolved against the working directory; also accepted by `start`. | Graph hot-reload works with any driver; the `.skein/` snapshot is skipped for durable drivers (they persist inherently). SQLite is a skein deployment option for a desktop app, CLI, daemon, or one Node owner. Install `@skein-js/local` and `@langchain/langgraph-checkpoint-sqlite@~1.0.4`, then select both drivers: ```bash skein dev --store sqlite --queue sqlite skein start -c .skein/build/langgraph.json --store sqlite --queue sqlite ``` Mixed SQLite/non-SQLite pairs are rejected. `dev` still defaults to memory/memory and `start` to Postgres/Redis. SQLite requires Node 22/24/26, does not import memory snapshots, and keeps its file outside the build artifact. Models and tools remain in the graph. See [local deployment](./local-deployment.md) for ownership, crash behavior, and native packaging. **Background-run concurrency.** `-n, --n-jobs-per-worker ` is honored on `skein dev` **and** `skein start`, with the same default of **10** the LangGraph CLI uses — so an existing `langgraph dev -n 4` command line ports over unchanged. `--concurrency ` is the same knob under a self-evident name, and wins if both are passed. Without a flag, skein reads `SKEIN_RUN_CONCURRENCY`, then the LangGraph-compatible `N_JOBS_PER_WORKER` — which is how the setting reaches a container. See [runs-and-redis.md](./runs-and-redis.md#run-concurrency). One deliberate cosmetic difference: skein's banner prints `Starting 1 worker, up to 10 concurrent runs` where `langgraph dev` prints `Starting 10 workers`. LangGraph really does spawn 10 loops; skein runs one worker whose consumer executes 10 runs at a time. Same throughput, honest wording. `skein up`/`build`/`dockerfile` accept `--tag` (build) and `--output` (dockerfile) in addition to `--config`. **`skein build --artifact-only`** (skein-only) writes `.skein/build` — the bundled graphs, the pinned `package.json`, the baked `schemas.json`, and the generated Dockerfile — and stops without invoking Docker. Use it when something else builds the image: Kaniko or Cloud Build in CI, a platform that builds from a committed Dockerfile, or any machine that has Node but no Docker daemon. **Private/authenticated npm registries.** When your production dependencies include private scoped packages (e.g. `@myorg/*` behind a token), pass an `.npmrc` so the image's dependency install can authenticate. `skein build --npmrc ` and `skein up --npmrc ` mount it as a BuildKit **secret** — it authenticates the install without ever landing in an image layer or build history. The generated Dockerfile always declares this secret mount (`id=npmrc`); it is optional, so public-registry builds that pass no `.npmrc` are unaffected. Building the standalone `skein dockerfile` output by hand? Supply the same secret directly: ```bash docker build --secret id=npmrc,src=$HOME/.npmrc -t my-app . ``` `skein build`/`up` also emit a `.dockerignore` that excludes `.env*`, `node_modules`, `.git`, and `.skein` — so secrets are **never baked into image layers**. Runtime configuration reaches the container through the environment instead: `skein up` sets `POSTGRES_URI`/`REDIS_URI` in the compose `environment:` block, and you add any provider keys (e.g. `OPENAI_API_KEY` for semantic search) there too. Generated Docker assets carry a `# Generated by skein` marker; skein regenerates its own files but leaves a hand-edited `Dockerfile`/`compose.yaml` untouched. References: - LangGraph CLI docs — - `@langchain/langgraph-cli` source — ## Migrating your project {#under-the-hood-what-skein-js-changes-transparently} Keep your graph exports and `langgraph.json`. Replace the development command with `skein dev`, then verify a known input in the [console](./console.md). Run your graph tests and an [SDK smoke test](./testing-your-project.md) before switching a deployment. TypeScript path aliases work in development and production builds, including workspace imports. Build production artifacts with `skein build`; `skein start` serves the compiled output without a runtime TypeScript loader. Keep your production dependencies installed in the deployed environment. Check the [compatibility reference](./langgraph-compatibility.md) for supported client combinations and intentional differences. CLI compatibility does not mean every feature of a managed deployment service is included. Contributor architecture details are in the [reuse guide](https://github.com/skein-js/skein-js/blob/main/docs/reuse.md). ## `langgraph.json` — fields we honor skein-js parses an existing `langgraph.json` **unchanged**. A `skein.json` may extend/override it but is never required. ```jsonc { // REQUIRED: map of graph id -> "path:export" "graphs": { "agent": "./src/agent.ts:graph", // exported compiled graph instance "chat": "./src/chat.ts:makeGraph", // or a factory function }, // JS/Node runtime pin (20 | 22 | 24) "node_version": "24", // Native production runtime (skein extension). Omit for Node. "skein": { "runtime": { "name": "bun", "version": "1.3.14" }, // Retention for `Idempotency-Key` records (skein extension; see below). Tuning only — // omitting the block does NOT disable the header. "idempotency": { "retention_hours": 24, "in_flight_minutes": 15 }, // Delivery policy for run-completion callbacks (skein extension; see below). Tuning only — // omitting the block does NOT make callbacks fire-once. "webhooks": { "retries": { "max_attempts": 12 }, "allowed_hosts": ["hooks.example.com"] }, }, // .env path OR inline map "env": ".env", // long-term memory store; semantic search config drives our pgvector index "store": { "index": { "embed": "openai:text-embedding-3-small", "dims": 1536, "fields": ["$"] }, // your own LangGraph BaseStore instead of skein's driver (skein extension) "adapter": "./src/my-store.ts:store", }, // checkpointer backend; "default" == Postgres (via PostgresSaver) "checkpointer": { "type": "default" }, // server customization — `cors` plus LangGraph's route toggles "http": { "cors": { "allow_origins": ["*"] }, "disable_assistants": false, "disable_threads": false, "disable_runs": false, "disable_crons": false, // skein-only: also stops the scheduler, not just the routes "disable_store": false, "disable_docs": false, // hide /docs and /openapi.json while keeping /info and /metrics "disable_meta": false, // turns off /info, /openapi.json, /docs and /metrics; never /ok "console": false, // skein-only: serve the console at /console (or a path). Off unless set; // `skein dev` serves it anyway. See docs/console.md. "app": "./src/http-app.ts:app", // Fetch function or exported Hono/Fetch application "disable_mcp": false, "disable_a2a": false, }, // custom authentication + authorization (see below) "auth": { "path": "./src/auth.ts:auth", // a "@langchain/langgraph-sdk/auth" `Auth` instance "disable_studio_auth": true, // default; CLI always requires credentials for Studio "dev_ui_access": false, // skein-only; dev key for console + Studio when true }, // where runs report themselves — traces + lifecycle events (a skein extension; see below) "telemetry": { "langsmith": true, "posthog": { "host": "https://eu.i.posthog.com" }, "otel": true, "paths": ["./src/my-telemetry.ts:sink"], // your own TelemetrySink }, // packages your code loads BY NAME at runtime, which no bundler can discover (skein extension) "dependencies": ["@langchain/openai"], // extra Dockerfile lines appended after the base image "dockerfile_lines": [], } ``` ### How each field maps into skein-js | `langgraph.json` field | skein-js wiring | | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `graphs` | [`@skein-js/config`](./storage.md) resolves each `path:export`, loading a compiled graph or `makeGraph` factory. Drives `/agents` introspection + run execution. | | `node_version` | Used by `skein build` / `skein dockerfile` base image selection. Defaults to Node 24 LTS when omitted; an explicit value is honoured verbatim, including an older one. | | `skein.runtime` | Native production server and pinned official image: `node`, `bun`, or `deno`. Bun/Deno use `@skein-js/fetch` with `Bun.serve`/`Deno.serve`, never Express compatibility. | | `skein.idempotency` | Retention for `Idempotency-Key` records. Tuning only — the header is honoured either way. See [below](#idempotency-skeinidempotency). | | `skein.webhooks` | How run-completion callbacks are retried and bounded. Tuning only — a run carrying a `webhook` owes a callback either way. See [below](#webhooks-skeinwebhooks). | | `env` | Loaded into `process.env` at boot (dev) / baked into the image (build). | | `store` | `store.index.{embed,dims,fields,hnsw}` configures pgvector semantic search on the Postgres driver (`hnsw: true` opts into the approximate index); `store.ttl.{default_ttl,refresh_on_read,sweep_interval_minutes}` expires items. `store.adapter` is a **skein extension** — `path:export` to your own LangGraph `BaseStore` (e.g. `PostgresStore`, `MongoDBStore`), which then serves the whole `/store` surface. See [storage.md](./storage.md). | | `checkpointer` | `"default"` → `PostgresSaver`; dev falls back to an in-memory `MemorySaver`. | | `http` | `http.cors` maps to the adapter's CORS options; the `disable_*` flags remove that resource's routes before mounting, so it 404s from the host app as under `langgraph dev`. `http.console` is skein-only — it serves the [console](./console.md) at `/console` (off unless set; `skein dev` serves it regardless). `http.app` loads an exported Fetch/Hono application before protocol dispatch; a 404 falls through. Authentication runs before custom application dispatch. | | `auth` | `auth.path` loads an `Auth` from `@langchain/langgraph-sdk/auth`; every request is authenticated + authorized. The CLI requires credentials for Studio regardless of `disable_studio_auth`; `auth.dev_ui_access: true` or loopback `skein dev --dev-ui-access` provides a temporary key for Studio and the console. The setting is stripped from production builds. | | `telemetry` | **skein extension.** Builds the telemetry sinks runs report to — see [observability.md](./observability.md). Unknown to `langgraph dev`, which ignores it. | | `dependencies` | **skein extension on the JS side** (LangGraph's schema has it for Python only). Extra packages `skein build` pins into the artifact — see below. | | `dockerfile_lines` | Appended by `skein dockerfile` / `skein build`. | **`dependencies` means something different here.** In LangGraph it is a Python-only field, required and used to drive `pip install`; the JS schema has no such field, because `langgraph build` copies your project into the image and runs your package manager over it. skein bundles instead, so it derives the image's dependency list from what the bundle still imports — and `dependencies` is the escape hatch for the packages that list cannot contain: the ones you load **by name at runtime** (`import("@langchain/" + provider)`), which never appear in any module graph. Local-path entries (`"."`, `"./pkg"`) are accepted and ignored — that source is already bundled. See [bundling.md](./bundling.md#what-skein-build-inlines-vs-externalizes). ## Graph loading (`path:export` notation) `@skein-js/config` resolves entries exactly like the LangGraph CLI: - `"./src/agent.ts:graph"` — imports the module and reads the `graph` export, which must be a `CompiledStateGraph`. - `"./src/agent.ts:makeGraph"` — reads a factory export and calls it (optionally with config) to obtain a `CompiledStateGraph`. This is the same contract LangGraph.js users already write against, so **no code changes are required** to move a project onto skein-js. ## Idempotency (`skein.idempotency`) A skein extension with no LangGraph counterpart, so it lives under the reserved `skein` namespace rather than at the top level — a top-level key would collide if LangGraph ever claimed the same name. ```jsonc { "skein": { "idempotency": { "retention_hours": 24, // how long a recorded response stays replayable "in_flight_minutes": 15, // how long an unfinished create blocks a retry "sweep_interval_minutes": 60, // how often expired records are reclaimed }, }, } ``` **This block is tuning, not an on/off switch.** Omitting it does not disable `Idempotency-Key`: a caller who sends one has been promised their retry will not start a second run, and honouring that is not a deployment opinion. The defaults are the values shown above. What each one is for: - `retention_hours` — the dedup window. Long enough to cover any provider's retry schedule, short enough that the table doesn't grow without bound. Raise it if you integrate a provider that retries over days. - `in_flight_minutes` — how long a create that hasn't finished holds its key before a retry may take it over. Keep it above your slowest `POST /runs/wait`, or a retry could be admitted alongside the original; keep it well under `retention_hours`, or an instance killed mid-request leaves its keys 409-ing until the retention expires. - `sweep_interval_minutes` — cadence of the background reclaim. Purely housekeeping: an expired record is taken over by the next claim whether or not the sweeper has run, so this affects disk, never correctness. Honoured identically on every driver combination, including `skein dev` with no Docker. See [agent-protocol.md](./agent-protocol.md#idempotent-run-creation-idempotency-key) for the request semantics and the replay table. ## Webhooks (`skein.webhooks`) LangGraph's `webhook` field is a bare URL with no delivery policy at all, so this block is a skein extension under the reserved namespace, for the same reason `skein.idempotency` is. The **payload shape is unchanged** — this configures delivery, not the body. ```jsonc { "skein": { "webhooks": { "retries": { "max_attempts": 12, // attempts before a callback is dead, counting the inline first one "initial_delay_ms": 1000, // the first retry's delay; the rest double from it }, // The signing key. Prefer SKEIN_WEBHOOK_SECRET, which wins over this — see below. "secret": "whsec_…", "max_payload_bytes": 262144, // cap on the stored body; over it, `values` is truncated "retain_hours": 24, // how long a settled delivery is kept before it is reclaimed "allowed_hosts": ["hooks.example.com"], // absent = no restriction (today's behaviour) "require_https": true, // refuse plaintext callbacks; absent permits them }, }, } ``` **This block is tuning, not an on/off switch.** Omitting it does not make callbacks fire-once: a run that carries a `webhook` owes a callback, and how hard the server tries is a deployment decision rather than permission to try at all. `retries.max_attempts: 1` is how you ask for one shot. What each one is for: - `max_attempts` — read it as a **time horizon, not a count**. The delays double, so 12 attempts is 1+2+4+…+1024 seconds ≈ **34 minutes**, which rides out a rolling deploy with room to spare. Six is not "half as patient": it is ~31 seconds, which does not survive one redeploy. There is deliberately no ceiling knob — the doubling tops out around 17 minutes at any sane attempt count, and on Redis the schedule is BullMQ's own exponential backoff rather than ours. - `initial_delay_ms` — the first retry's delay and the base the rest double from. Raise it for a receiver you know is slow to come back; lower it only if you also lower `max_attempts`. - `max_payload_bytes` — the body is **stored** so a retry has something to send, so an unbounded payload is unbounded rows. Over the cap, `values` is replaced by a truncation marker inside the signed body; everything a receiver needs to fetch the state itself survives. - `retain_hours` — how long a delivered or dead row is kept. Disk only; nothing is incorrect if the sweep never runs. - `secret` — the signing key, or a list during a rotation (the first signs). **Prefer the `SKEIN_WEBHOOK_SECRET` environment variable**, which takes precedence: a key here is a key in version control, and skein warns about it once at startup. Note that skein does **not** expand `${VAR}` anywhere in `langgraph.json`, so `"secret": "${SKEIN_WEBHOOK_SECRET}"` would sign every callback with that literal 23-character string. Absent on both → callbacks are unsigned, which is today's behaviour. - `require_https` — refuse a callback whose URL is not `https:`. Off by default, because an internal receiver on a trusted network is legitimate and turning it on for everyone would break those deployments on upgrade. Turn it on whenever callbacks leave your network: the body carries the run's final state, and retries mean it crosses the wire up to `max_attempts` times rather than once. - `allowed_hosts` — an exact-hostname allowlist. Set it if you accept run creates from untrusted callers: `webhook` is a caller-supplied URL, so it is a server-side request to a target they chose, and retrying it turns a one-shot SSRF probe into a repeated one. **Off by default**, because turning it on for everyone would make an upgrade start dropping deployments' own callbacks. A refused host is recorded `dead` with the reason rather than silently skipped. **Where the retries actually run depends on your queue driver.** With Redis, the whole schedule is BullMQ's — delayed jobs, exponential backoff with jitter, and re-delivery of a job whose worker was killed — so a retry still waiting survives a restart. Without it, skein polls the outbox instead: the stored due time and delivery lease survive restart whenever the store is durable. SQLite and Postgres therefore retain waiting retries without a separate delivery queue. With a durable store, the delivery and its due time are recorded in the same transaction as the run's terminal status. Waiting retries therefore survive restart without a separate queue. See [webhooks.md](./webhooks.md). ## Authentication + authorization (`auth`) skein-js implements LangGraph's **custom-auth** model. Point `auth.path` at a module exporting a [`@langchain/langgraph-sdk/auth`](https://github.com/langchain-ai/langgraphjs/tree/main/libs/sdk) `Auth` instance — the same class LangGraph Platform uses, so an existing auth file is drop-in: ```ts import { Auth, HTTPException } from "@langchain/langgraph-sdk/auth"; export const auth = new Auth() .authenticate(async (request) => { const token = request.headers.get("authorization")?.replace(/^Bearer /, ""); const user = token ? await verify(token) : undefined; if (!user) throw new HTTPException(401, { message: "Unauthorized" }); return { identity: user.id, permissions: user.scopes }; // becomes ctx.user }) .on("threads", ({ user }) => ({ owner: user.identity })); // scope threads (and their runs) by owner ``` - `.authenticate(...)` runs on every request; throwing an `HTTPException` (or returning nothing) is a `401`. Its return becomes the `user` passed to the `.on(...)` handlers. - `.on(":", ...)` (or a broader `"threads"` / `"*"`) authorizes each operation: return `false` for a `403`, or a **filter** object to scope the resource. A returned filter both hides other owners' rows on reads and stamps ownership onto new rows. Runs authorize through their owning thread (there is no separate `runs` resource), matching LangGraph. - **No `auth` block** → the server is fully open (unauthenticated), exactly as before. - Studio's `x-auth-scheme: langsmith` header is caller-controlled, so the CLI requires credentials for Studio even when `disable_studio_auth` is `false`. `skein dev --dev-ui-access` or `auth.dev_ui_access: true` prints a temporary API key accepted by Studio and the console on a loopback server. `skein start` never enables it, and production builds strip the setting. Direct config loaders also default `disable_studio_auth` to `true`; an explicitly injected engine controls its own policy. See [agent-protocol.md](./agent-protocol.md#authentication--authorization) for the full request lifecycle and the route → resource/action map. ## `dev` vs `up` - **`skein dev`** — a single Node app process with hot reload. It defaults to in-memory (optionally file-backed) state, so Docker is not required; pass `--store postgres --queue redis` to develop against the production drivers, including Postgres and Redis running in Docker. Fast feedback, the exact `langgraph dev` niche. - **`skein up`** — Docker Compose bringing up the app plus Postgres (checkpoints + protocol resources + pgvector) and Redis (queue + cross-instance streaming). Mirrors production — see [runs-and-redis.md](./runs-and-redis.md). ## Migrating an existing `langgraph dev` state When you run `langgraph dev`, LangGraph persists all local state — assistants, threads, runs, store items, and graph **checkpoints** (state + full history) — under a `.langgraph_api/` directory. skein reads that format and carries everything over, so switching loses nothing. **Automatic (the common case).** The first time you run `skein dev` in a project that has a `.langgraph_api/` directory but no skein state yet (`.skein/dev-state.json`), skein imports it on boot and logs what it brought over. From then on it persists to `.skein/` as usual, so the import runs exactly once. Nothing to do — your old threads and their history are just there. **Explicit — `skein import-langgraph`.** For re-running, previewing counts, or importing into a durable database: | Flag | Values | Default | Notes | | ------------------ | -------------------- | -------------------------- | ------------------------------------------------------------------------------------- | | `-c, --config` | path | `langgraph.json` | Resolves the project directory. | | `--store ` | `memory`, `postgres` | `memory` | `memory` writes `.skein/dev-state.json`; `postgres` loads a live DB (`POSTGRES_URI`). | | `--from ` | path | `/.langgraph_api` | Source directory to read. | | `--force` | — | off | Overwrite an existing `.skein/dev-state.json` (memory target). | ```bash skein import-langgraph # → .skein/dev-state.json, then `skein dev` skein import-langgraph --store postgres # → your POSTGRES_URI (checkpoints + resource rows) ``` **What carries over, and what doesn't.** Assistants (with their **version history**), threads (with their latest state), runs, store items, and the **full checkpoint history** are all migrated. Because skein uses LangGraph's own `MemorySaver`/`PostgresSaver`, the checkpoint format is identical — history is preserved exactly. Minor, deliberate drops: the store's derived **embedding index** (`vectors`) is not copied (skein re-indexes on write / a configured `store.index` re-embeds on Postgres import), and LangGraph **retry counters** aren't part of skein's model. Run **webhooks** _are_ carried through, so an imported run still fires its completion webhook. > The `.langgraph_api/` layout is a `@langchain/langgraph-api` internal (stable across 1.2.x–1.4.x), > not a public API. The importer is best-effort and guarded — a format it can't read never blocks > `skein dev` from starting. > > **Already on Postgres?** If your LangGraph deployment already uses a Postgres checkpointer, you > don't need to migrate checkpoints at all: point skein at the **same `POSTGRES_URI`** and it shares > the exact same checkpoint tables. `skein import-langgraph --store postgres` is for moving an > in-memory `langgraph dev` (including one run in production) onto a durable skein deployment. ## Graph UI and custom HTTP applications Standard `ui` and `ui_config` use the public `@langchain/langgraph-ui` build/watch API. For example: ```json { "graphs": { "agent": "./src/agent.ts:graph" }, "ui": { "agent": "./src/ui.tsx" }, "ui_config": { "shared": ["react", "react-dom"] }, "http": { "app": "./src/http-app.ts:app" } } ``` A UI source exports a default component registry. Development watches the sources; `skein build` compiles assets, schemas and the custom application into the artifact. Production serves `POST /ui/{agent}` bootstrap HTML and `GET /ui/{agent}/{asset}` from those compiled assets without loading the UI compiler or project sources. Application exports may be a Fetch function or an object with `fetch(request)`, including Hono. They return a standard `Response`; a 404 continues to UI and Agent Protocol routes, with the original request body available even if the app read its copy. All five adapters share this behavior, including mounted path prefixes. Deployment authentication runs before custom routes and UI dispatch, preserving the mounted protocol routes’ existing exemptions: the `/info` handshake is public, and channel callbacks use provider-signature verification. When rendering external UI scripts or styles in a browser, use credentials the browser can supply on asset requests (such as same-origin cookies), or explicitly allow those asset paths in the deployment's auth policy. `auth.openapi` may describe that existing policy using standard `securitySchemes` and `security`; it does not change authentication or authorization. --- # Embedding a graph you already have > **User guide.** skein-js has **two on-ramps**. If you already run the LangGraph CLI, the > [drop-in](./langgraph-cli-compat.md) (`langgraph dev` → `skein dev`, unchanged `langgraph.json`) is > for you. This doc is the **other** on-ramp: you have a LangGraph.js graph in your own app and never > adopted the LangGraph Platform's project shape — no `langgraph.json`, no CLI. Bring the compiled > graph in code and get the same Agent Protocol server in a few lines. ## Two on-ramps | | Config-based loading (`{ config }`) | In-code embedding (`{ deps }`) | | -------------------- | ------------------------------------------------------ | --------------------------------------------------------------- | | You start from | a `langgraph.json` in your application | a **compiled graph object** in your own code | | Wiring | `createExpressServer({ config: "./langgraph.json" })` | `createExpressServer({ deps: embedInMemoryGraphs({ graph }) })` | | Graph loading | `path:export` loaded by your host or supplied importer | you already hold the graph — nothing is loaded from disk | | Best for | migrating off / comparing against the LangGraph CLI | greenfield apps, or anyone who never used the Platform | | Static graph schemas | ✅ extracted from source | 🟡 stubbed (see [trade-off](#the-one-trade-off-schemas)) | Both expose the same thread/run APIs. Your chosen storage and graph capabilities determine persistence and client support. The CLI supplies its own development watcher and snapshots; an adapter's `{ config }` option does not start that workflow. Use a TypeScript-capable host or supply `importModule` when your config points at TypeScript graphs. ### Choose an assembly for your host | Helper | Storage | Choose it for | | ------------------------------------------------- | ------------------------------------------------- | ----------------------------------------------------------- | | `embedInMemoryGraphs` from `@skein-js/server-kit` | Memory | Development, tests, or state owned elsewhere | | `embedSqliteGraphs` from `@skein-js/local` | SQLite resources, queue, and upstream checkpoints | A local app, CLI, daemon, or one Node process | | `embedPostgresGraphs` from `@skein-js/runtime` | Postgres, optionally Redis | A durable server, with Redis for workers spanning instances | The memory example below is the smallest server. For the local option, see [SQLite embedding and host lifecycle](./local-deployment.md). Its helper starts no worker or listener; your host owns startup and stops its runtime before disposing storage. ## The whole thing For receipt-based access without HTTP, use the runtime's supported [native client](./native-access.md). It adds authorized atomic admission, strict resume targets and retained results for embedded callers. The optional BullMQ access adapter uses the same boundary. ```ts import { createExpressServer } from "@skein-js/express"; import { embedInMemoryGraphs } from "@skein-js/server-kit"; import { graph } from "./my-graph.js"; // ← your existing `new StateGraph(...).compile()` const server = await createExpressServer({ deps: embedInMemoryGraphs({ agent: graph }) }); await server.listen(2024); ``` That's a full server. Point any Agent Protocol client at `http://localhost:2024`: ```ts import { Client } from "@langchain/langgraph-sdk"; const client = new Client({ apiUrl: "http://localhost:2024" }); const thread = await client.threads.create(); await client.runs.wait(thread.thread_id, "agent", { input: { messages: [{ role: "user", content: "hello" }] }, }); ``` `embedInMemoryGraphs` ([`@skein-js/server-kit`](https://github.com/skein-js/skein-js/tree/main/packages/server-kit)) turns a **graph map** into a `ProtocolDeps` backed by in-process drivers — the store, run queue, event bus, and checkpointer. No config file, and nothing to import from a storage package. `{ deps }` is the seam **every** adapter accepts, so the same `deps` mounts on Express, Fastify, NestJS, or Next.js unchanged. Check the returned state against your graph's expected output. To repeat this automatically, adapt [the SDK smoke test](./testing-your-project.md#check-the-running-server) to the `agent` Assistant. Framework adapters do not serve the console automatically; follow [manual mounting](./console-reference.md#mounting-it-yourself) if you want interactive inspection. Runnable version: [`examples/embed-graph`](https://github.com/skein-js/skein-js/tree/main/examples/embed-graph). > **⚠️ Auth is off by default.** `embedInMemoryGraphs` sets no `auth`, so the server it produces > **authenticates nothing** — every request is allowed (the same default as a `langgraph.json` with no > `auth` block). That's fine behind your own middleware or on a private network, but mounting `{ deps }` > on a **public** app exposes `/threads`, `/runs`, and `/store` to anyone — including running your graph > (spending model tokens) and reading/writing the long-term store. Add an `auth` engine before you go > public — see [Bring your own drivers, auth, logger](#bring-your-own-drivers-auth-logger). ## The graph map Keys become graph ids (one auto-registered assistant each). Values are either a **compiled graph** or a **factory** — a function that builds one, called with the run's `configurable`. Factories are how you defer expensive or key-requiring construction until a graph is actually run: ```ts embedInMemoryGraphs({ echo, // a compiled graph, imported eagerly // built lazily on first use — keeps a keyless boot when the model needs an API key: agent: async () => (await import("./agent-graph.js")).graph, // or per-run config: (config) => buildGraph(config.configurable?.model), }); ``` A concretely-typed `.compile()` result (e.g. from `MessagesAnnotation`) is accepted **without a cast** — the [`EmbeddableGraph`](#api-reference) type leaves the graph's generics open on purpose. ## Standalone or embedded `{ deps }` works with every adapter, in both its standalone and embedded form: ```ts const deps = embedInMemoryGraphs({ agent: graph }); // Express — standalone server, or mounted on your existing app. // `skeinRouter` is async: it seeds assistants and starts the run worker before returning. await (await createExpressServer({ deps })).listen(2024); app.use((await skeinRouter({ deps })).router); // Fastify — plugin under a prefix: await app.register(skeinPlugin, { prefix: "/agent", deps }); // NestJS — dynamic module: @Module({ imports: [SkeinModule.forRoot({ deps })] }) // Next.js — App Router catch-all (same-origin, no second server): export const { GET, POST, PUT, PATCH, DELETE, OPTIONS } = createSkeinRouteHandlers({ deps }); ``` The Next.js App Router case is the lightest full-stack story — an 11-line `route.ts` serving the protocol same-origin behind a `useStream` UI. See [`examples/nextjs-app`](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-app). Each adapter mounts a little differently — prefixes, CORS, body limits, `/ok` and shutdown all vary. [adapters.md](./adapters.md) covers each one in full. ## Bring your own drivers, auth, logger `embedInMemoryGraphs(graphs, overrides)` takes a second argument that replaces any field of `ProtocolDeps` except `graphs` (the first argument is the single source of graphs) — a driver, an `auth` engine, a `logger`. Supplying `auth` is how you close the open-by-default surface from the warning above: ```ts import { loadAuthEngine } from "@skein-js/config"; embedInMemoryGraphs({ agent: graph }, { auth: await loadAuthEngine(/* … */), logger: myLogger }); ``` A `logger` set here is used unless the adapter is given an explicit `logger` option, which wins. Under NestJS and Fastify, leaving both unset falls back to the host framework's own logger rather than to silence — see [errors-and-logging.md](./errors-and-logging.md#what-each-adapter-does-by-default). ## Going to production The in-memory drivers are ideal for a single long-lived process (dev, tests, a small app). For durable, horizontally-scalable state, use **`embedPostgresGraphs`** — the persistent sibling of `embedInMemoryGraphs`. Same graph-in-code call, but it assembles a Postgres store + `PostgresSaver` checkpointer and (when a Redis URL is present) a Redis run queue + event bus, reading `POSTGRES_URI` / `REDIS_URI` from the environment: ```ts import { createExpressServer } from "@skein-js/express"; import { embedPostgresGraphs } from "@skein-js/runtime"; import { graph } from "./my-graph.js"; const { deps, dispose } = await embedPostgresGraphs({ agent: graph }); // reads POSTGRES_URI / REDIS_URI const server = await createExpressServer({ deps }); await server.listen(2024); // It owns pools/connections, so release them on shutdown (embedInMemoryGraphs has nothing to release): process.on("SIGTERM", () => dispose().finally(() => process.exit(0))); ``` > **⚠️ Auth is still off by default — and this is the production path.** Like `embedInMemoryGraphs`, > `embedPostgresGraphs` sets no `auth`, so the server **authenticates nothing**. That's easy to miss > here precisely because you reach for this helper to _deploy_: shipping it public with no `auth` > exposes `/threads`, `/runs`, and `/store` — and running your graph (spending model tokens) — to > anyone. Pass an `auth` engine via `overrides` before you go public (see below). It lives in [`@skein-js/runtime`](https://github.com/skein-js/skein-js/tree/main/packages/runtime), not `@skein-js/server-kit` — a persistent helper pulls in the Postgres/Redis drivers that `server-kit` deliberately avoids. Pass explicit `postgresUri` / `redisUri` (and `index` for pgvector, `ttl`, `poolMax`, `sslNoVerify`) instead of env vars if you prefer, and `overrides` for `auth` / `logger` / etc. — see the [API reference](#api-reference): ```ts import { loadAuthEngine } from "@skein-js/config"; const { deps, dispose } = await embedPostgresGraphs( { agent: graph }, { overrides: { auth: await loadAuthEngine(/* … */) } }, ); ``` > **Redis is optional, but then you're single-instance.** With no `redisUri` / `REDIS_URI`, the run > queue + event bus fall back to in-memory: state still survives a restart (it's in Postgres), but the > run queue is process-local and streaming isn't fanned across instances, so you **can't run more than > one instance**. Set a Redis URL to scale horizontally. **Sizing the in-memory bus.** On the Redis-less path the event bus holds run frames in the process, bounded by `SKEIN_MEMORY_BUS_MAX_FRAMES_PER_RUN` (10000) and `SKEIN_MEMORY_BUS_MAX_RETAINED_RUNS` (50) rather than by a Redis TTL. The worst case is roughly `MAX_FRAMES_PER_RUN × (concurrent runs + MAX_RETAINED_RUNS)` — half a million frames at the defaults, which is hundreds of MB. Both are read from the environment, so they reach an embedded host without a code change. Size them against what your graph actually emits, and note that a far-behind subscriber loses the oldest frames here where Redis would still have them: [performance.md](./performance.md) has the sizing table and the triage symptoms. **Signing run-completion callbacks.** `SKEIN_WEBHOOK_SECRET` is read on the embed paths too, so setting it is all an embedded host has to do to get signed callbacks — there is no `langgraph.json` here to carry a `skein.webhooks` block. Pass `overrides.webhooks` to configure the rest of the delivery policy (retries, `allowed_hosts`, `max_payload_bytes`); it is spread last, so an explicit value wins over the environment. See [webhooks.md](./webhooks.md). > Before this was wired up, an embedded host that exported `SKEIN_WEBHOOK_SECRET` sent **unsigned** > callbacks with no warning — the env var was only read on the `langgraph.json` path. If you embed and > rely on signatures, check you are on a version that includes this. Prefer to assemble the drivers yourself (e.g. a Postgres store with an in-memory queue, or your own pool)? Pass them through `embedInMemoryGraphs`' `overrides`: ```ts import { PostgresSaver } from "@langchain/langgraph-checkpoint-postgres"; import { RedisRunEventBus, RedisRunQueue } from "@skein-js/redis"; import { createPostgresPool, PostgresSkeinStore } from "@skein-js/storage-postgres"; const checkpointer = new PostgresSaver(createPostgresPool(process.env.POSTGRES_URI!)); await checkpointer.setup(); const deps = embedInMemoryGraphs( { agent: graph }, { store: await PostgresSkeinStore.connect(process.env.POSTGRES_URI!), checkpointer, queue: new RedisRunQueue(process.env.REDIS_URI!), bus: new RedisRunEventBus(process.env.REDIS_URI!), }, ); ``` If you _do_ have a `langgraph.json`, [`@skein-js/runtime`](https://github.com/skein-js/skein-js/tree/main/packages/runtime)'s `buildRuntime({ configPath, store: "postgres", queue: "redis" })` assembles all of these for you — `embedPostgresGraphs` is the same assembly for graphs you hold in code. Serverless/edge deploys need durable drivers: the in-memory drivers (and the background run worker) assume one warm process, so they don't survive a function that scales to zero. See [storage.md](./storage.md) and [runs-and-redis.md](./runs-and-redis.md). Bundling the result yourself? The in-code path is the friendly one — it never reaches the `langgraph.json` graph loader, so everything on it bundles cleanly, `@skein-js/storage-postgres` included. See [bundling.md](./bundling.md). ## The one trade-off: schemas A **compiled** graph no longer carries its TypeScript source, so the in-code path can't extract real input/output/state JSON schemas — `embedInMemoryGraphs` returns a minimal `{ graph_id }` stub for the assistants introspection endpoints. This is enough for **everything `useStream` and Agent Chat UI render**; the only thing that degrades is **LangGraph Studio's** schema-driven forms and its graph/step views. If you need full static schemas, use the [`{ config }` path](./langgraph-cli-compat.md) — the `langgraph.json` loader runs `getStaticGraphSchema` over the graph source at build time. ## API reference From [`@skein-js/server-kit`](https://github.com/skein-js/skein-js/tree/main/packages/server-kit): ```ts // Build a ProtocolDeps around in-process drivers. Pass a graph map OR a ready GraphResolver. function embedInMemoryGraphs( graphs: GraphResolver | Record, overrides?: Omit, "graphs">, // every driver/auth/logger except `graphs` ): ProtocolDeps; // Turn just the graph map into a GraphResolver (the ids/load/schemas seam the engine consumes). function graphMapToResolver(graphs: Record): GraphResolver; // A graph you can embed: any compiled LangGraph.js graph, or a factory that builds one per run. type EmbeddableGraph = CompiledGraph | ((config: { configurable?: Record }) => …); ``` > `embedInMemoryGraphs` was previously named `createInMemoryDeps`. The old name is still exported as a > deprecated alias, so existing imports keep working — prefer `embedInMemoryGraphs` in new code. `graphMapToResolver` is useful on its own when you want the resolver but your **own** `ProtocolDeps`. `normalizeEmbeddableGraphs(graphs)` accepts either a graph map or a ready `GraphResolver` and returns a `GraphResolver` — the same normalization both embed helpers apply. ### Assembling `ProtocolDeps` by hand The engine drives an **`AgentGraph`** (`stream` + `getState` required, the rest optional), so `@skein-js/agent-protocol` installs with no graph runtime. Running **LangGraph.js** graphs therefore needs the binding wired in — four fields the embed helpers set for you, and which a hand-assembled `ProtocolDeps` must set itself: ```ts import { cloneLangGraphCheckpoint, langGraphResolver, SkeinBaseStore } from "@skein-js/langgraph"; import { MemorySaver } from "@langchain/langgraph"; const deps: ProtocolDeps = { store, queue, bus, checkpointer, // Translates the engine's command envelope into a LangGraph `Command`. // Without it, human-in-the-loop resume silently no-ops. graphs: langGraphResolver(graphMapToResolver({ agent })), // Bridges long-term memory in, so nodes reach it via `getStore()`. storeBridge: (repo) => new SkeinBaseStore(repo), // The throwaway saver `POST /invoke/:graph_id` runs against, so nothing persists. ephemeralCheckpointer: () => new MemorySaver(), // Clones a checkpoint when it is re-put under a new thread id (copy / prune / rollback). cloneCheckpoint: cloneLangGraphCheckpoint, }; ``` `langGraphResolver` is safe to apply to any resolver: it binds LangGraph compiled graphs and returns anything else untouched, so a resolver fronting your own `AgentGraph` still receives the envelope. From [`@skein-js/runtime`](https://github.com/skein-js/skein-js/tree/main/packages/runtime) (the durable path — see [Going to production](#going-to-production)): ```ts // Build a durable ProtocolDeps (Postgres store + PostgresSaver, Redis queue/bus when configured) and a // dispose() to release the pools/connections it owns. Async, because it connects + migrates on the way up. function embedPostgresGraphs( graphs: GraphResolver | Record, options?: { postgresUri?: string; // default process.env.POSTGRES_URI (required — one of the two) redisUri?: string; // default process.env.REDIS_URI (absent → in-memory queue/bus, single instance) index?: StoreIndexConfig; // pgvector semantic search (a resolved embedder) ttl?: StoreTtl; // store-item expiry + background sweep threadTtl?: ThreadTtl; // thread expiry — the in-code `checkpointer.ttl` poolMax?: number; // default env PG_POOL_MAX sslNoVerify?: boolean; // default env DATABASE_SSL_NO_VERIFY connectionTimeoutMs?: number; // default env PG_CONNECTION_TIMEOUT_MS, else 30s (0 = wait forever) idleTimeoutMs?: number; // default env PG_IDLE_TIMEOUT_MS statementTimeoutMs?: number; // default env PG_STATEMENT_TIMEOUT_MS, else 30s (0 = no limit) maxPageSize?: number; // default env SKEIN_MAX_PAGE_SIZE, else 1000 (list/search bound) overrides?: Omit, "graphs" | "store" | "queue" | "bus" | "checkpointer">; }, ): Promise<{ deps: ProtocolDeps; dispose(): Promise }>; ``` Unlike `embedInMemoryGraphs` (which returns a plain `ProtocolDeps`), `embedPostgresGraphs` is **async** and returns `{ deps, dispose }` — it owns Postgres pools and Redis connections, so you must `dispose()` them on shutdown. ## See also - [langgraph-cli-compat.md](./langgraph-cli-compat.md) — the other on-ramp (drop-in CLI + `langgraph.json`) - [agent-protocol.md](./agent-protocol.md) — the endpoints you get either way - [building-an-adapter.md](./building-an-adapter.md) — putting the engine on any HTTP framework - [storage.md](./storage.md) · [runs-and-redis.md](./runs-and-redis.md) — swapping in production drivers - [`examples/embed-graph`](https://github.com/skein-js/skein-js/tree/main/examples/embed-graph) · [`examples/nextjs-app`](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-app) For durable local embedding, use [`embedSqliteGraphs`](./local-deployment.md) from `@skein-js/local`. It uses the same protocol and graph bindings with one owned SQLite file. --- # Durable local deployment `@skein-js/local` assembles the existing Agent Protocol engine with SQLite persistence and LangGraph's SQLite checkpointer. Use the same graphs in a desktop application, terminal tool, separate daemon, or server. The application chooses models and tools, including any connection to a local model. Choose this deployment when one Node process can own the database file. For a shared service with multiple instances, use [Postgres and Redis](./deploy.md). The default development server continues to use memory drivers; SQLite is selected explicitly and does not change existing defaults. ## Embed agents ```sh npm install @skein-js/local @langchain/langgraph @langchain/langgraph-checkpoint-sqlite@~1.0.4 ``` ```ts import { embedSqliteGraphs } from "@skein-js/local"; import { createProtocolRuntime } from "@skein-js/agent-protocol"; const local = await embedSqliteGraphs( { assistant: graph }, { sqlitePath: "/absolute/application-data/state.sqlite", // maxPageSize, ttl, threadTtl and non-owned ProtocolDeps overrides are optional. }, ); const runtime = createProtocolRuntime(local.deps); await runtime.service.assistants.registerGraphAssistants(); runtime.worker.start(); // For authorized receipt-based work, use runtime.createNativeClient({sourceId, allowedAssistants}). // Use runtime.service for trusted administration, or give local.deps to a framework adapter. // The host stops its worker/adapter before disposing owned resources. await runtime.worker.stop(); await local.dispose(); ``` The helper starts no assistants, workers, listener, application process or global signal handler. See [native access](./native-access.md) for durable receipts, retained results and strict resume without HTTP. Native process-loss outcomes use the existing outbox; SQLite still never automatically replays an abandoned execution. It owns storage and a small item-expiry timer. The existing protocol runtime owns the scheduler, thread-retention sweep, idempotency sweep and delivery worker. An adapter manages that runtime's lifecycle through its existing APIs. `dispose()` is idempotent. For direct storage use, `openSqliteStorage` from `@skein-js/storage-sqlite` returns the existing `store` and `queue` contracts, the borrowed native `database`, and `dispose()`. The helper's `sqlitePath` and driver's `filename` must be absolute. Both create the requested parent directory. Do not close or change ownership settings on the borrowed connection. `overrides` retains existing auth, logger, clock, telemetry, webhook and external `storeItems` injection. The owned graph/storage/queue/bus/checkpointer bindings cannot be replaced there. The two libraries do not install server-kit, schema/config loading, Postgres, Redis, BullMQ or HTTP adapters. Add an existing adapter when a transport is needed. ## CLI and scaffolding ```sh npm create skein-js@latest my-agent -- --storage sqlite cd my-agent npm run dev npm run build npm start ``` Or select the SQLite pair on an existing project: ```sh skein dev --store sqlite --queue sqlite skein start -c .skein/build/langgraph.json --store sqlite --queue sqlite ``` Install `@skein-js/local` and the saver in the project first. Both commands accept `--sqlite-path`; the CLI resolves it against the working directory and defaults to `.skein/state.sqlite` there. The file remains outside the build artifact. SQLite does not import memory development snapshots. Mixed SQLite/non-SQLite store and queue pairs are refused. Existing command defaults remain unchanged. The LangGraph import command retains its memory and Postgres targets. The config path keeps auth, channels, telemetry, retention and external-store configuration. SQLite provides text/filter search and rejects `store.index`; configure an external semantic store through its existing adapter seam. SQLite scaffolds omit Compose and external-service environment settings and include `dev:memory` as an alternative. The scaffolder adds no desktop-framework or model axis. ## What survives a process ending | Driver | Unclaimed queued work | Abandoned executing attempt | Stream frames | | ------ | ----------------------------------- | ---------------------------------------------------------- | ---------------------------- | | Memory | Lost with the in-memory queue | No durable failure accounting | Process-local | | SQLite | Persists with its original due time | Existing `error` status with `ProcessLostError`; no replay | Process-local | | Redis | Persists in Redis | Existing BullMQ stalled-job/redelivery behavior | Existing Redis bus retention | SQLite holds exclusive file ownership from an initialization write until disposal. A second owner is refused; an OS process exit releases the lock. Startup examines abandoned runs and queue claims in internal batches without the public page cap. Pending inline requests also become failures. Terminal outcomes, including real approval interruptions, retain their results. Saved values, checkpoints and interrupt payloads remain available through existing state/history APIs. Run creation records server-owned background intent and its absolute due time. SQLite commits the run, kwargs and queue job together, including guarded admission and cron claims. Enqueue deduplicates without shortening delays. Cron jobs remain immediately eligible; cron `after_seconds` remains ignored. Failure accounting does not retry input, resume a checkpoint, cancel a thread's pending backlog, or reconstruct a successful outcome. Applications use existing run/state APIs to make those choices. Reconciliation does not synthesize completion callbacks, telemetry or application cleanup. Previously committed delivery records remain eligible for the existing polling retry worker after restart. ## Desktop and daemon hosts The maintained consumers are distinct applications: | Consumer | Application | Model tools and local actions | | ------------------------------------------------------------------------------------ | ------------------------------------ | ------------------------------------------------------------------------------------- | | [Electron](https://github.com/skein-js/skein-js/tree/main/examples/desktop-electron) | **Daybook**, a meeting workspace | Read project context, search earlier meetings, review and save follow-ups | | [Tauri](https://github.com/skein-js/skein-js/tree/main/examples/desktop-tauri) | **Reading Room**, a document library | Read a source document, search the library, review a brief with exact source excerpts | | [CLI](https://github.com/skein-js/skein-js/tree/main/examples/local-cli) | **Dayplan**, a daily planner | Read a selected task file and prior plans, review focus blocks, save a Markdown plan | From the repository on Node 24, install dependencies and choose a provider in desktop Settings or the CLI environment: ```sh pnpm install pnpm nx run example-desktop-electron:start # Or: pnpm nx run example-desktop-tauri:start pnpm nx run example-local-cli:start ``` Desktop start commands package and open the applications; their first run requires native build prerequisites and can take several minutes. The CLI starts directly. The examples offer **Vertex AI**, **Gemini API**, **OpenAI**, **Anthropic**, and **Ollama**. Desktop Settings configures the provider, model and connection. The CLI uses environment variables: | Provider | Configuration | | ---------- | ------------------------------------------------------------------------------------------------ | | Vertex AI | `MODEL_PROVIDER=vertex`, Google ADC, optional `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` | | Gemini API | `MODEL_PROVIDER=gemini`, `GOOGLE_API_KEY` (or `GEMINI_API_KEY`) | | OpenAI | `MODEL_PROVIDER=openai`, `OPENAI_API_KEY` | | Anthropic | `MODEL_PROVIDER=anthropic`, `ANTHROPIC_API_KEY` | | Ollama | `MODEL_PROVIDER=ollama`, optional `OLLAMA_BASE_URL` and `OLLAMA_MODEL` | For Vertex, run `gcloud auth application-default login`; the app discovers a project from environment or gcloud settings, or accepts one in Settings. No project name/ID is hardcoded. `MODEL_NAME` selects a model; defaults are `gemini-3-flash-preview` for Google and `gpt-5-mini` for OpenAI. Provider-specific `GOOGLE_MODEL`, `OPENAI_MODEL`, `ANTHROPIC_MODEL` and `OLLAMA_MODEL` also work. The Vertex region defaults to `global`. Cloud providers require valid access and available billing/quota. Explicit environment preferences override saved UI preferences. UI-entered keys live only in backend session memory; only non-secret preferences are persisted. Disconnecting stays in effect across restarts until the user reconnects, even when environment credentials remain available. Selected source material and tool results are sent to the selected provider. Anthropic defaults to `claude-sonnet-5-5`; Ollama defaults to `qwen3:8b` at `http://127.0.0.1:11434`. Ollama needs an installed tool-capable model and no API key. The app never downloads models automatically. All model names and the Ollama endpoint are editable. This is application code: Skein adds no provider configuration or credential store. Model integration remains application-owned. Desktop UIs use `@langchain/langgraph-sdk` over the existing HTTP/SSE adapter. Background `runs.create` and `runs.joinStream` let preparation continue while the user navigates. Actual tool activity and generated answers stream through `custom` and `messages` modes. Drafts autosave, conversations persist in separate threads, and pending reviews reopen from SQLite checkpoints. Approval sends `command.resume`; the host-selected local file action runs only after review. **History and versions** shows actual requests, run status and checkpoints. **Create alternative** uses thread copy and state-update APIs without replacing the original. Saved output is searchable through Skein's shared store. These application features require no additional library API. Normal launches use the real model. Keyless deterministic fixtures are explicitly selected only by tests and packaging gates. Hosts choose application-data paths, launch and shutdown behavior. A separate daemon uses the same assembly; its host owns supervision, endpoint discovery and credential exchange. No CLI daemon lifecycle or desktop-specific protocol is introduced. The examples bind loopback on an OS-assigned port and use a per-launch bearer through existing auth, with `studioAuthDisabled: true`. Credentials are passed through environment/private bridges, rather than command-line arguments or saved files. The target matrix is Node 22/24/26, Electron 44 and Tauri 2 on Linux x64, Windows x64, and macOS ARM64/x64. When packaging your host, verify native addon loading, a writable application-data path, streamed output, approval resume after reopening, and rejection of invalid credentials on each target platform. Test the packaged application, since a source build does not establish native packaging compatibility. Tauri packages an official portable Node 24 distribution, verified against its SHA-256 manifest, plus the backend directory and native assets. Electron rebuilds the SQLite addon against its own ABI. See [Tauri resources](https://tauri.app/develop/resources/) for the bundle mapping. This release does not support SQLite in browsers, Bun/Deno or distributed workers, or promise a single executable containing every native dependency. Measure package size and startup on the platform you ship. Historical package measurements and platform validation are available as [contributor evidence](https://github.com/skein-js/skein-js/blob/main/docs/local-deployment-validation.md). --- # Native access and BullMQ Use `runtime.createNativeClient` when a trusted host admits work without HTTP and needs a receipt that survives retries. It uses the same assistants, threads, authorization, run engine and storage as HTTP. The optional `@skein-js/bullmq` package connects external command and outcome queues to that client. It does not replace Skein's internal execution or delivery queues. The [runnable approval example](https://github.com/skein-js/skein-js/tree/main/examples/native-transports) includes an embedded SQLite client and an HTTP/BullMQ backend with a durable outcome inbox. Read [security and operations](./native-access-security.md) before exposing a producer to real data. ## Embedded applications ```ts import { embedSqliteGraphs } from "@skein-js/local"; import { createProtocolRuntime } from "@skein-js/agent-protocol"; const local = await embedSqliteGraphs( { agent: graph }, { sqlitePath: "/absolute/application-data/agents.sqlite", }, ); const runtime = createProtocolRuntime(local.deps); await runtime.service.assistants.registerGraphAssistants(); const client = runtime.createNativeClient({ sourceId: "desktop-approval-worker", // stable across restarts allowedAssistants: ["agent"], // assistant IDs or graph IDs; required, empty permits none }); runtime.worker.start(); const receipt = await client.start( { assistant_id: "agent", input: {} }, { idempotencyKey: "approval-42", correlationId: "request-42", }, ); // Admission has committed; execution may still be pending. const run = await client.getRun(receipt.runId); const recovered = await client.getReceipt("approval-42"); // Once terminal: client.getResult(receipt.runId) returns the whole retained RunResult. await runtime.worker.stop(); await local.dispose(); ``` The constructor is guaranteed on `createProtocolRuntime`'s concrete return and optional on the `ProtocolRuntime` interface for compatibility with third-party implementations. There is no separate native constructor. The existing `runtime.service` remains useful for trusted administrative work; it does not add native authorization, receipts or admission guarantees to independently composed calls. Sources and caller identity are host configuration. When authentication is enabled, every native call requires the host's verified `authContext`. The client calls the existing authorizer with canonical resource IDs and applies its ownership filters. It does not authenticate arbitrary broker claims or turn graph metadata into authority. HTTP and native sources can use the same authorized thread when the host maps their credentials to the same principal. Keys are scoped by stable source and principal. Omit a key to generate one and receive it in `{ idempotencyKey, threadId, runId }`. Identical retries return the original receipt; changed input returns `idempotency_key_reused`. Receipt replay and reads recheck current authorization. Access revocation takes effect immediately. Retention, expiry and resource deletion bound replay guarantees. Keep a returned generated key before retrying; a new key means a new mutation. Trusted adapters that derive a run from a command may set `idempotencyInput` to the **original** command, including all caller-controlled meaning. This keeps retry identity stable if host routing or start/resume selection changes. Do not set it to a constant or let a caller choose a reduced fingerprint. The first admission retains its route, execution identity and deadline. ## Interrupts, cancellation and deadlines After a run reaches `interrupted`, read its state and submit an answer with that exact checkpoint: ```ts const state = await client.getState(receipt.threadId); const answer = await client.resume( { thread_id: receipt.threadId, assistant_id: "agent", expected: { runId: receipt.runId, checkpoint: state.checkpoint, // interruptIds: ["expected-interrupt-id"], // optional additional precondition }, resume: "approved", // also accepts LangGraph's resume map }, { idempotencyKey: "answer-42", authContext: verifiedContext }, ); const cancelled = await client.cancel(answer.runId, { idempotencyKey: "cancel-42", authContext: verifiedContext, wait: true, // action: "rollback", // existing cancellation semantics }); ``` Only one admitted resume can consume an expected interrupted run/checkpoint. Admission and invocation validate it under the thread execution lease. A stale target returns `stale_resume`; fetch current state before submitting a new answer. Checkpoint updates use the same lease. Resuming an already accepted key recovers its receipt even after execution advanced the checkpoint. Separate checkpoint, status and concurrency overrides are rejected at runtime. Strict resume validates every supplied `checkpoint_map` namespace against its current checkpoint before admission and invocation; a current root ID cannot authorize an older nested checkpoint. Native state reads hold the execution lease and can wait behind active work. Retry `native_state_changed` if a target changes during authorization or lease acquisition. Cancellation returns `{ receipt, status, stopped }`. A terminal status records the authoritative decision; `stopped` separately confirms execution has stopped. Without waiting, a remote execution may still be unwinding. Repeated cancellation with `wait: true` still waits for a cancelled execution. Graphs and tools must honor cancellation signals. Lease loss aborts protected execution after it is observed; this does not fence arbitrary external effects from tools that ignore their signal. `deadlineAt` is a trusted absolute ISO timestamp in call options. Expired commands are rejected before admission. An accepted command keeps its receipt and uses the earlier of its remaining absolute budget and `runTimeoutMs`; retries cannot extend it. `after_seconds` retains its first absolute eligibility time. Result reads return `run_result_pending` while active and `run_result_unavailable` if no output was retained, including cancellation before execution and unserializable output. Serialization failure does not rewrite a successful execution as a failed run or invent replacement output. ## Durable outcomes Bind an optional trusted `outcomeTarget` on the source and compose `ProtocolDeps.webhookDispatcher` to handle it. This opts in to durable terminal accounting for success, error, timeout, interruption, pending cancellation/displacement and SQLite process loss. Without it native results remain available, but no callback is owed. Legacy HTTP/channel callback defaults are unchanged. The existing outbox records the outcome atomically with terminal status and optional retained result. The dispatcher receives the existing `deliveryId` and `attempt`; channel and routed destination callbacks now also receive those fields as optional context. Receiver deduplication must persist `deliveryId` before acknowledging. Delivery is at least once, including a crash after publication but before outbox acknowledgement. A removed BullMQ job cannot serve as a durable deduplication record. Outcomes use existing payload/retention limits. Large outcomes carry run references and a truncation marker. **Stored results remain whole**, independent of delivery truncation and checkpoint pruning. Results follow existing run/thread deletion. Snapshot import and thread copy retain results but exclude live native receipt, dispatch and delivery obligations. Memory guarantees last only for that process. SQLite keeps atomic run/job admission and marks abandoned attempts `ProcessLostError` on reopen; it does not automatically replay their input. Postgres recovery drains bounded leased dispatch intents for explicitly admitted background work. Acknowledging enqueue keeps ownership until run deletion: an active run becomes eligible again when its lease expires, so a lost queue job can be reconstructed with the same run ID. Terminal runs are skipped under the execution lease before checkpoint effects. Ordinary inline runs and unrelated producers' rows are not adopted. Explicit cancellation, displacement, deletion and shutdown grace expiry broadcast abort requests even when a local registration is only waiting for a remote executor. Failed cancellation publication remains recoverable. Eligible execution owners also observe durable terminal status to stop after a missed pub/sub notification. Native rollback preserves the admitted run and receipt; explicit resource deletion still removes them. ## BullMQ access transport ```sh npm install @skein-js/bullmq bullmq@^5 ioredis@^5 ``` ```ts import { Queue } from "bullmq"; import { createBullMqTransport } from "@skein-js/bullmq"; import { createProtocolRuntime } from "@skein-js/agent-protocol"; const outcomes = new Queue("agent-outcomes", { connection }); const access = createBullMqTransport({ commandQueue: "agent-commands", outcomeQueue: outcomes, // borrowed connection, // options or borrowed supported ioredis connection authenticate: async (job) => ({ authContext: await verifyQueueCaller(job) }), // route: command => mapToAllowedGraph(command), // projectResult: (result, runId) => applicationProjection(result, runId), }); const runtime = createProtocolRuntime({ ...deps, webhookDispatcher: access.wrapDispatcher(existingDispatcher), }); await runtime.service.assistants.registerGraphAssistants(); const native = runtime.createNativeClient({ sourceId: "backend-queue", allowedAssistants: ["agent"], outcomeTarget: access.outcomeTarget, }); runtime.worker.start(); await access.start(native); // creating the transport consumes nothing // Shutdown consumption before the runtime and storage. await access.stop(); await runtime.worker.stop(); await outcomes.close(); // host owns this queue/connection ``` Workers own their blocking connection and are closed by `stop`; borrowed queues/connections remain open. Keep the outcome queue name and Redis prefix stable across restart. Host-controlled authentication, routing, deadlines, destinations and output projection are functions/configuration, not authority accepted from message fields. HTTP remains available independently. `projectResult` covers outcomes and `result.get`; `state.get` and `run.get` remain full authorized reads. Deny raw operations in the host's existing callbacks if a producer should see only projections. Redis credentials also grant access to stored job payloads/return values; configure ACLs, TLS and separate retention for those copies. See [the security guide](./native-access-security.md). The adapter owns the following versioned wire format, exposed as `BullMqCommand`: ```ts await commands.add( "skein.command.v1", { version: 1, commandId: "work-42", operation: "start", input: { assistant_id: "agent", input: {} }, }, { attempts: 5, backoff: { type: "exponential", delay: 1000 } }, ); ``` | Operation | Fields beyond `version`, `commandId`, optional `correlationId` | Job return value | | ------------- | -------------------------------------------------------------- | ---------------------------- | | `start` | `input: NativeStartInput` | Admission receipt | | `resume` | `input: NativeResumeInput` | Admission receipt | | `cancel` | `runId`, optional `action`, `wait` | Cancellation response | | `run.get` | `runId`, optional `offset`, `digest` | Bounded chunk of current run | | `state.get` | `threadId`, optional `offset`, `digest` | Bounded chunk | | `result.get` | `runId`, optional `offset`, `digest` | Bounded chunk | | `receipt.get` | `idempotencyKey` | Receipt | `commandId` becomes the native idempotency key. Mutations acknowledge durable acceptance. The first admission keeps its diagnostic correlation ID; changing only `correlationId` on a retry does not change command meaning. Execution completion arrives as a `BullMqOutcome` (`version: 1`, `operation: "run.settled"`) on the outcome queue. `eventId` is the outbox delivery ID, with `commandId`, resource IDs, status and a `resultRef`. Small outcomes may include projected output; oversized ones omit it and set `truncated: true`. One run has one authoritative terminal outcome; outcomes for different runs may arrive out of order. Persist receiver deduplication by `eventId` and compare `runId` with the receipt for the turn your application is displaying before changing that turn's state. Reconcile through `run.get`/`state.get` when needed; checkpoint coordinates and chunk digests identify the snapshot being read. Arrival order does not establish thread order. BullMQ handles retries according to the producer's bounded `attempts`/`backoff`. Validation, authorization, stale answers and unsupported capabilities fail with `UnrecoverableError`; its JSON message includes the stable code and status. Admission contention, pending results and infrastructure failures use normal retry behavior. Default BullMQ attempts remain one unless the producer sets them. For command acknowledgement waiters, start `QueueEvents` with a retained cursor such as `lastEventId: "0-0"`, as the runnable example does. Connection readiness alone does not prove that event consumption has begun; a fast completion can otherwise fall between the waiter's status check and its first stream read. If acknowledgement times out, recover admission by its stable command key. Each run/state/result request reauthorizes and returns a base64 byte chunk with SHA-256 `digest`, `offset`, `nextOffset` and `totalBytes`. Start at offset zero, then supply the returned digest and next offset. Concatenate decoded bytes and verify the digest before parsing JSON. `result_digest_mismatch` means the value/projection changed; restart retrieval. No HTTP is needed to reconcile missed or truncated outcomes. Native `getResult` always returns the whole result. `maxMessageBytes` defaults to the existing 256 KiB payload cap and can be reduced to 1 KiB. The supported dependency majors are BullMQ 5/6 and ioredis 5/6. The [validation record](./native-access-validation.md#bullmq-and-ioredis-compatibility) describes the installed consumer matrix. In mixed BullMQ 5/ioredis 6 installations, pass connection options to BullMQ queues and workers so BullMQ retains its own client; the Skein transport also accepts borrowed ioredis 6 `Redis` and `Cluster` clients. No casts are needed in application code. The adapter uses the borrowed outcome queue's publishing operations, so an application's BullMQ major upgrade can retain a different supported version inside the adapter. For explicitly typed options, use `BullMqTransportOptions` to retain the application's full Queue type; existing `BullMqTransportOptions` annotations keep their original default. ## Custom drivers and verification Native writes explicitly require optional `SkeinStore.admission` and `RunRepo.settle`/`getResult`; missing guarantees return `native_admission_unsupported` instead of emulating transactions with CRUD. `admitRun` commits the scoped key/fingerprint, guarded run/displacements, receipt, resume reservation and dispatch intent together. `admitCancel` does the corresponding cancellation transaction. `claimDispatches` leases bounded batches; `markDispatched` only acknowledges the matching token. Run intents remain recoverable while pending/running after acknowledgement; cancel intents stop after acknowledgement. Retain admission ownership until run deletion, and atomically refuse a transition from terminal to running for owned runs, returning the effective row unchanged. `settle` atomically records the first terminal status, first matching result and optional identified delivery, returning the effective rows. Existing `recordResult`, `getResult`, `finalizeWithDelivery` and third-party-driver paths remain supported. Postgres leases may provide an optional loss signal. New `settle` deliveries start pending at attempt zero; the delivery worker starts attempt one. Legacy `finalizeWithDelivery` keeps its inline attempt and existing defaults. `RunAdmission.signal` is an optional ephemeral `AbortSignal`. Drivers observe it before mutations and at their pre-COMMIT boundary and roll back when it is aborted before COMMIT dispatch. It is never stored in kwargs, keys or snapshots. Abort can race a COMMIT already sent: a committed receipt remains authoritative and is recovered by key. This is cooperative admission cancellation, not database fencing. To verify an integration, run the same command twice with the same scoped key and confirm the receipt is reused. Exercise resume, cancellation, and outcome delivery with test tools, then restart your host and read the recorded result. The [native-transports example](https://github.com/skein-js/skein-js/tree/main/examples/native-transports) shows an HTTP/BullMQ approval backend and durable receiver deduplication. Repository validation is recorded in the [contributor evidence](https://github.com/skein-js/skein-js/blob/main/docs/native-access-validation.md). Existing MCP and A2A endpoints keep their protocol-specific identity, state and transport behavior. This boundary adds no protocol endpoints, SDK dependency, workflow engine, artifact service or exactly-once external-effect guarantee. --- # Native access security and operations Native and BullMQ access compose existing authorization, admission, execution and outbox guarantees. The host supplies verified identity and policy. Start with the [runnable approval example](https://github.com/skein-js/skein-js/tree/main/examples/native-transports) and the [API guide](./native-access.md). ## Authority at the boundary Construct native clients in trusted code. Keep `sourceId`, `allowedAssistants` and `outcomeTarget` stable and outside caller-controlled input. With authentication enabled, supply verified `authContext` on every operation. A principal, scopes or destination in metadata or a queue payload does not authenticate anyone. Receipt replay and reads recheck current authorization and ownership. For BullMQ, `authenticate(job)` must verify the producer using the host's credential arrangement or bind a queue restricted to one trusted backend to a server-owned principal. Trusting `job.data.userId` would let a producer impersonate another user. The example uses one trusted backend and loopback HTTP with a demonstration token. Replace that identity policy for deployment. Authentication-disabled deployments preserve existing unauthenticated behavior. An allowlist accepts assistant or graph IDs and limits operations for that source. It does **not** partition data inside a shared thread: other authorized assistants/transports may have written its checkpoints. Use separate threads and ownership rules for data isolation. Revocation prevents future authorized reads; it cannot retract values already returned or copied to Redis. Strict resume accepts the expected interrupted run/checkpoint, optional interrupt IDs and answer. Runtime validation rejects separate checkpoint, status or concurrency overrides, including JSON fields TypeScript excludes. The execution lease protects validation and invocation; atomic reservation prevents replicas consuming the same interrupt. Each supplied `checkpoint_map` entry must also name the current checkpoint in that namespace for the authorized thread. A valid root ID cannot redirect a child to an older checkpoint. This check runs at admission and before invocation, using the existing checkpointer contract without interpreting LangGraph's internal metadata. Valid nested coordinates retain LangGraph's own execution semantics. Native state reads authorize a resolved target, then hold the execution lease while constructing its graph and reading checkpoints. They can wait behind execution. `native_state_changed` asks the caller to retry if the target changes during authorization or lease acquisition. Pending admissions cannot switch the graph mid-read. Factories retain the selected run's sanitized configuration and use the **requesting** principal. ## Projection and broker storage `projectResult` applies to outcomes and `result.get`. It does not redact `state.get` snapshots or `run.get` metadata. Reject raw query operations in `authenticate` or `route` when a producer must see only projected output; both existing callbacks can throw `SkeinHttpError`. Apply corresponding policy to HTTP and other sources available to that principal. Native `getResult` returns the whole retained result and belongs behind the host's trust boundary. BullMQ stores inputs and return values in Redis. Queue names/prefixes organize keys; they are not tenant access controls. Redis readers can inspect values without Skein authorization. Restrict credentials, network access and ACLs, and configure TLS. Avoid secrets in commands/results; use application-managed encryption when needed. Follow [BullMQ's production guidance](https://docs.bullmq.io/guide/going-to-production) and [Redis security guidance](https://redis.io/docs/latest/operate/oss_and_stack/management/security/). Set retention for completed/failed commands, query return values, outcomes and receiver inboxes. Skein run/thread deletion and outbox expiry do **not** erase external copies. Automatic job removal does not replace receiver deduplication: persist `eventId` with application database effects before acknowledging. Keep destination queue identity stable across restart so existing outbox obligations continue reaching it. Chunk digests detect changes between reads; they do not authenticate the broker or encrypt output. Each chunk request reauthorizes. Bound total reassembly size for your application, verify offsets, size and digest, and restart retrieval on a changed digest. Truncated outcomes leave whole retained results available through reconciliation. ## Cancellation, recovery and resource use Acknowledgement, terminal status and confirmed execution stop are separate facts. `wait: true` reports `stopped: true` after acquiring valid execution rights. A failed stop check returns retryable `execution_stop_unconfirmed` (503) while the accepted receipt remains recoverable. Waiting can queue behind other work on that thread. If an abort channel is configured without a cross-instance execution gate, waiting cancellation is rejected before admission with `execution_stop_unsupported` (501); a local mutex cannot prove remote execution stopped. Supported Postgres assembly supplies both capabilities. Tools must honor abort signals; status changes cannot undo arbitrary external effects. Redis pub/sub has [at-most-once delivery](https://redis.io/docs/latest/develop/pubsub/). Recovery retries failed cancellation publication before acknowledging its intent. Successful publication cannot prove reception. Native executions, and executions with both an abort channel and atomic admission capability, also observe durable status once per second with one read in flight per execution. For N active eligible executions, budget approximately N reads/second. Read failures are logged and retried. Observed terminal status aborts local control without rebroadcasting. Ordinary single-process runs without these capabilities retain their existing path. Native rollback retains the admitted run, receipt and stored result for reconciliation. Explicit deletion still removes them. Cleanup does not revert a later run that advanced the thread. Legacy runs retain established rollback deletion behavior. Expired trusted `deadlineAt` is rejected after authorization and immediately before admission. Once accepted, the original deadline/route remain authoritative. Execution uses the earlier of remaining budget and configured timeout. This is cooperative admission cancellation, not a database clock fence or a promise to stop external effects at an exact instant. Memory is process-local. SQLite requires one file owner, records process loss as failure and never automatically replays abandoned input. Postgres recovery leases bounded batches of owned dispatches and reconstructs lost queue jobs with original run IDs. It does not adopt unrelated runs. Observe dispatch warnings, outbox depth, failed jobs, capacity and receiver lag; reconcile by receipt/run ID. ## Dependency review Audit the dependencies in your deployed lockfile, including transitive validators and broker clients. Apply compatible fixes and verify the application after upgrades. Historical repository dependency findings are kept in the [validation record](https://github.com/skein-js/skein-js/blob/main/docs/native-access-validation.md). Enforce request byte limits, **pre-validation** collection limits and rate/concurrency budgets at host boundaries, including user-facing bridges into native calls. Check whether your installed validator rejects large collections before expensive validation; a length check performed afterward does not bound that work. BullMQ's message cap helps bound input but is not a general resource-exhaustion guarantee. --- # Serving a graph as a plain endpoint — no threads, no runs > **User guide.** skein's main surface is the full [Agent Protocol](./agent-protocol.md) — threads, > assistants, runs, streaming, history, human-in-the-loop. That's what a **chat** app needs. Plenty of > LangGraph work isn't chat: a classifier, an extractor, an enrichment step another service calls. For > those, this page describes a smaller surface — **one graph, one endpoint, called like a function**. ## Which surface do I want? | | Full Agent Protocol | Single-graph invoke | | ------------------ | -------------------------------------------------------------- | -------------------------------------------- | | Mount | `skeinRouter` / `skeinPlugin` / `SkeinModule` / route handlers | `skeinInvoke*` / `SkeinInvokeModule` | | Endpoint | `/threads`, `/assistants`, `/runs`, `/store`, … | `POST /invoke/:graph_id` | | Request | `{ input, config, stream_mode, … }` on a thread | the graph input, raw | | Response | a run object (or an SSE run stream) | the final graph state | | Conversation state | persisted per thread (resume, history, time travel) | **none** — each call is independent | | Best for | chat, HITL, anything resumable | classification, extraction, batch, workflows | | Clients | `useStream`, Agent Chat UI, LangGraph SDK | `fetch`, `curl`, any HTTP client | Both run the _same_ graph through the same LangGraph machinery. The difference is how much protocol sits in front of it. Mounting both is supported and safe: each call attaches its checkpointer and store to a per-call clone of the compiled graph, so an invoke never disturbs a concurrent protocol run's durable state. ## The whole thing ```ts import { skeinInvokeRouter } from "@skein-js/express"; import { embedInMemoryGraphs } from "@skein-js/server-kit"; import express from "express"; import { graph as triage } from "./triage-graph.js"; const app = express(); const { router } = await skeinInvokeRouter({ deps: embedInMemoryGraphs({ triage }) }); app.use(router); app.listen(2024); ``` ```bash curl -sX POST localhost:2024/invoke/triage \ -H 'content-type: application/json' \ -d '{"text":"Refund charge failed — urgent!"}' # {"text":"Refund charge failed — urgent!","category":"billing","priority":"P1"} ``` Runnable version: [`examples/invoke-endpoint`](https://github.com/skein-js/skein-js/tree/main/examples/invoke-endpoint). ## The contract **The request body is the graph input. The response is the graph's final state.** No envelope in either direction — the graph behaves like a function over HTTP. - **One endpoint per graph.** Every id in the graph map (or in `langgraph.json`) is mounted at `POST /`. The prefix defaults to `/invoke`. - **An unregistered id is a 404**, with `code: "graph_not_found"`. - **An empty body runs the graph with its state defaults** rather than surfacing LangGraph's opaque `EmptyInputError` — useful for graphs that take no input at all. - **Responses are serialized with the same wire encoder** the protocol uses, so LangChain messages in a final state flatten to the `{ type, content }` shape clients expect. - **A graph that throws is an error response**, mapped through the adapter's normal error path. ## Streaming The default is a single JSON response. Send `Accept: text/event-stream` and the _same_ endpoint streams the graph's steps as SSE instead, ending with a terminal `end` (or `error`) event: ```bash curl -NsX POST localhost:2024/invoke/triage \ -H 'content-type: application/json' -H 'accept: text/event-stream' \ -d '{"text":"Outage — everything is down"}' ``` ```text id: 1 event: values data: {"text":"Outage — everything is down","category":"","priority":""} id: 2 event: values data: {"text":"Outage — everything is down","category":"general","priority":""} event: end data: {"status":"success"} ``` The mode defaults to `values` (each frame is the full state after a step, ending at the value the JSON response would have returned). Override it per mount with `streamMode`, or per request with `?stream_mode=updates` (comma-separate for several). Modes are validated at the boundary, so an unknown one is a 400 rather than an opaque failure deeper in. If the graph throws mid-stream the failure arrives as an `error` frame — headers are already sent, so it can't become an HTTP status. `stream_mode=events` is **not** available here: it is not a Pregel stream mode (the run engine serves it from `graph.streamEvents`, while this surface drives `graph.stream`), so it is rejected with a 400. Use the Agent Protocol run endpoints for token-level events. ## On every adapter The surface is the same everywhere; only the mount idiom differs. ```ts const deps = embedInMemoryGraphs({ triage, extract }); // Express — a Router you mount const { router } = await skeinInvokeRouter({ deps }); app.use(router); // Fastify — a plugin, encapsulated under its prefix await app.register(skeinInvokePlugin, { prefix: "/agent", deps }); // → POST /agent/invoke/:graph_id // NestJS — a dynamic module, alongside your controllers @Module({ imports: [SkeinInvokeModule.forRoot({ deps })] }) export class AppModule {} // Next.js App Router — app/api/invoke/[graph_id]/route.ts export const runtime = "nodejs"; export const { POST } = createSkeinInvokeRouteHandlers({ deps, basePath: "/api/invoke" }); ``` Each accepts the same `{ config } | { deps }` seam as the full protocol, so `skeinInvokeRouter({ config: "./langgraph.json" })` works too — see [embedding.md](./embedding.md). The Express/Fastify/NestJS mounts claim only their own path and pass everything else through, so the host app's routes are untouched. ## Auth `deps.auth`, when configured, is enforced here exactly as on a run — invoking a graph _runs_ it (spending model tokens), so this is not a way around the gate. The caller is authenticated (401 on failure) and authorized against `threads:create_run` (403 on deny), and the authenticated principal is stamped into the graph's `configurable` as `langgraph_auth_user`, just like a protocol run. Auth runs _before_ the graph-exists check, so an unknown id also returns 401 rather than 404 — an anonymous caller can't enumerate which graphs you serve by telling the two apart. > **⚠️ With no `auth` configured, this endpoint is open** — the same default as the full protocol > (and as `langgraph dev`). That's fine behind your own middleware or on a private network; anywhere > public, wire an `auth` engine before you ship. See [embedding.md](./embedding.md#bring-your-own-drivers-auth-logger). ## What it deliberately doesn't do Each call is **independent**: no thread is created, nothing is persisted between calls, and the run never appears in `/runs`. Concretely, this surface has no: - **conversation state** — no thread id, no history, no time travel; - **interrupts / human-in-the-loop** — there is no thread to resume into; - **background runs, run rows, or webhooks** — the call is inline; there is no run to cancel by id; - **assistants** — you address the `graph_id` directly, with no assistant config layer. The **long-term store is still injected**, so nodes reach cross-thread memory via `getStore()` as usual. If you need any of the above, use the full protocol — that's what it's for. The call's lifetime really is the request: the graph runs under an `AbortSignal` that fires when the client disconnects (all four adapters wire this) and when `deps.runTimeoutMs` elapses — the same budget the run engine applies — so a disconnected or hung call doesn't keep burning model tokens. ## API reference From [`@skein-js/agent-protocol`](https://github.com/skein-js/skein-js/tree/main/packages/agent-protocol) (the shared handler every adapter wraps): ```ts // A ProtocolHandler for `POST /:graph_id`. Resolves the graph, injects the store, invokes. function createGraphInvokeHandler( deps: ProtocolDeps, options?: { streamMode?: StreamMode | StreamMode[] }, // SSE modes; default "values" ): ProtocolHandler; // The one-route table, shaped like `skeinRoutes` so catch-all adapters can match it identically. function graphInvokeRoutes(prefix?: string): RouteBinding[]; // default prefix "/invoke" // Build a matcher over any route table (used by the NestJS/Next.js catch-all mounts). function createRouteMatcher(bindings: readonly RouteBinding[]): RouteMatcher; ``` Per adapter — each takes `SkeinRuntimeOptions` (`{ config } | { deps }`, plus `logger`/`cors`) and the `streamMode` option above: | Adapter | Entry point | Path option | | ------- | ----------------------------------------- | -------------------- | | Express | `skeinInvokeRouter(options)` | `prefix` (`/invoke`) | | Fastify | `skeinInvokePlugin` | `invokePrefix` | | NestJS | `SkeinInvokeModule.forRoot(options)` | `prefix` | | Next.js | `createSkeinInvokeRouteHandlers(options)` | `basePath` | From [`@skein-js/server-kit`](https://github.com/skein-js/skein-js/tree/main/packages/server-kit): ```ts // Resolve `{ config } | { deps }` to just the deps — no assistants seeded, no run worker started. function resolveRuntimeDeps(options: SkeinRuntimeOptions): Promise<{ deps: ProtocolDeps; cors? }>; ``` ## See also - [embedding.md](./embedding.md) — bringing a graph in code (`{ deps }`), the on-ramp this builds on - [agent-protocol.md](./agent-protocol.md) — the full surface, and when you want it instead - [streaming.md](./streaming.md) — how skein maps run frames onto SSE - [`examples/invoke-endpoint`](https://github.com/skein-js/skein-js/tree/main/examples/invoke-endpoint) · [`examples/embed-graph`](https://github.com/skein-js/skein-js/tree/main/examples/embed-graph) --- # Agent Protocol surface skein-js implements LangChain's [**Agent Protocol**](https://github.com/langchain-ai/agent-protocol), an OpenAPI-specified, framework-agnostic HTTP + streaming contract for serving LLM agents. **What this gives you:** a standard REST + SSE API your client already speaks — assistants, threads, runs (wait / stream / background), streaming, interrupts, and a long-term store. Because it's the same contract LangGraph Platform serves, your existing [`@langchain/langgraph-sdk`](./react-sdk.md) and [`useStream`](./react-sdk.md) code works against a skein-js server by changing only the URL. You almost never call these endpoints by hand — the SDK does — but this page is the map of what's available and what ships in the MVP. For the streaming wire format, see [streaming.md](./streaming.md); for building a frontend on top, see [react-sdk.md](./react-sdk.md). ## Core resources | Resource | Description | | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Assistants / agents** | A served graph plus its introspectable input/output/state/config schemas. | | **Threads** | Multi-turn conversation containers with persistent state and history; track status (`idle`, `busy`, `interrupted`, `error`). | | **Runs** | Atomic executions of a graph — stateless (ephemeral), streaming, or background. | | **Store** | Long-term memory organized by namespace + key, with CRUD and (semantic) search. Also injected into graph runs as a LangGraph `BaseStore` — see [storage.md](./storage.md#long-term-memory-in-the-graph-getstore). | | **Crons** | Schedules that fire a run on a cadence. A LangGraph Platform extension, not part of the open spec — see [crons.md](./crons.md). | | **Messages** | First-class primitives aligned with OpenAI/Anthropic formats. | ## Endpoint inventory Every endpoint below is implemented (✅). The route table in [`packages/agent-protocol/src/http/routes.ts`](https://github.com/skein-js/skein-js/blob/main/packages/agent-protocol/src/http/routes.ts) is the source of truth. Individual runs use the SDK's thread-scoped paths (`/threads/{thread_id}/runs/{run_id}`); Skein's cross-thread search is listed below. ### Assistants Full CRUD + version history (LangGraph parity). Assistants are auto-registered one-per-graph at startup (`assistant_id` defaults to `graph_id`), and can also be created/updated/deleted over the API. Every `PATCH` mints a new **immutable version**; the live row tracks the currently-active version and mirrors its fields, and `POST .../latest` rolls back to any past version. (Routes use the `/assistants/...` spelling the `@langchain/langgraph-sdk` client sends — not `/agents/...`.) | Method | Path | Notes | | -------- | --------------------------------------------- | --------------------------------------------- | | `POST` | `/assistants` | Create; `if_exists: "raise" \| "do_nothing"` | | `GET` | `/assistants/{assistant_id}` | | | `PATCH` | `/assistants/{assistant_id}` | Update — mints a new version | | `DELETE` | `/assistants/{assistant_id}` | `?delete_threads=true` cascades owned threads | | `POST` | `/assistants/search` | Filter by graph_id/name/metadata; sort + page | | `POST` | `/assistants/count` | Count matching the search filters | | `GET` | `/assistants/{assistant_id}/schemas` | Input/output/state/config schemas | | `GET` | `/assistants/{assistant_id}/graph` | Drawable graph JSON (`?xray`) | | `GET` | `/assistants/{assistant_id}/subgraphs[/{ns}]` | Subgraph schemas by namespace (`?recurse`) | | `POST` | `/assistants/{assistant_id}/versions` | Version history, newest-first (filter + page) | | `POST` | `/assistants/{assistant_id}/latest` | Roll back to an existing version | ### Threads | Method | Path | Notes | | -------- | -------------------------------------------- | --------------------------------------------- | | `POST` | `/threads` | Create; `if_exists`, `supersteps` seed state | | `GET` | `/threads/{thread_id}` | | | `POST` | `/threads/search` | | | `POST` | `/threads/count` | How many match the filters (no pagination) | | `POST` | `/threads/prune` | Bulk `delete`, or `keep_latest` history trim | | `GET` | `/threads/{thread_id}/state` | Current state snapshot (`useStream` hydrates) | | `POST` | `/threads/{thread_id}/state` | Time travel: fork state at a checkpoint | | `GET` | `/threads/{thread_id}/state/{checkpoint_id}` | Time travel: state at a checkpoint | | `POST` | `/threads/{thread_id}/state/checkpoint` | Same read, checkpoint given as an object | | `POST` | `/threads/{thread_id}/history` | Checkpoint history, newest-first (paged) | | `GET` | `/threads/{thread_id}/history` | Same, with `?limit` (the SDK sends `POST`) | | `PATCH` | `/threads/{thread_id}` | | | `POST` | `/threads/{thread_id}/copy` | Duplicates the thread + its history | | `DELETE` | `/threads/{thread_id}` | | **`if_exists` on thread creation.** `POST /threads` with a `thread_id` you choose defaults to `if_exists: "raise"` — a 409 when that id is taken. `"do_nothing"` returns the **existing** thread untouched, which makes `client.threads.create({ threadId: stableKey, ifExists: "do_nothing" })` a get-or-create: the idiom for addressing a conversation by an external identity (a phone number, an email thread, a ticket id) without tracking skein's own ids. Uniqueness is enforced in the storage driver, not by a read-then-write in the service, so two instances racing the same id cannot both win. **Seeding a thread with `supersteps`.** `POST /threads` accepts a `supersteps` array — updates written straight into the thread's checkpoint history, so you can **import** an existing conversation rather than replaying it through the graph. Each superstep is one tick and becomes one checkpoint; each update carries `values` (or a `command`) and a required `as_node` saying which node to attribute it to. Writing state needs a graph, and a thread this new has no run to infer one from — so pass `graphId`, which the SDK folds into `metadata.graph_id`. Without it this is a 400. That graph id is also what lets the seeded state be **read back**: `GET /threads/{id}/state` and the history routes fall back to `metadata.graph_id` when a thread has never run. Bounded at 100 supersteps of 100 updates: each update is a graph-state write to the checkpointer. Note supersteps are applied to whatever the create returned — including a thread that `if_exists: "do_nothing"` merely _found_, in which case they append to it. That matches `@langchain/langgraph-api`, which runs its bulk write on the result of the create either way. **Filtering threads by graph.** `POST /threads/search` matches on a metadata subset. When a run is created, skein stamps the run's `graph_id` and `assistant_id` into the thread's metadata (matching LangGraph), so listing the threads for a graph is just: ```jsonc // POST /threads/search { "metadata": { "graph_id": "my_graph" } } ``` The stamp reflects the thread's most recent run; a thread that has never run carries no `graph_id`. **Paging checkpoint history.** `POST /threads/{id}/history` takes its options in the **body** (as the LangGraph SDK sends them): `{ limit?, before?, metadata? }`. It returns at most **100** checkpoints when `limit` is omitted, and rejects a `limit` above 1000 — each element is a checkpoint's whole graph state, so a long thread's full history is one of the largest single responses skein can produce. Page back through it with `before` (a checkpoint config carrying `checkpoint_id`, or a bare `checkpoint_id`) — the bound is exclusive, so pass the last checkpoint you received — and narrow it with `metadata`, which becomes the checkpointer's filter. Checkpoint IDs, namespaces and checkpoint maps reach the checkpointer; the thread scope remains server-owned, so a `thread_id` in `before.configurable` is dropped. A `?limit=` query param is still accepted for hand-rolled callers, but it is _clamped_ to 1000 rather than rejected, preserving the established lenient query behavior. The body wins if both are present. `useStream` sends `limit: 10`, so 10 checkpoints come back rather than every one, matching LangGraph Platform. The rendered transcript is unaffected (the newest checkpoint's `values` carry the whole message list); what shrinks is how far back the branch/edit tree reaches. Raise it by passing your own `limit` if you need deeper history. The 100-checkpoint default is independent of `SKEIN_MAX_PAGE_SIZE` — history is read from the checkpointer, not from the store, so the store's page bound does not apply to it. **Time travel (fork from a checkpoint).** `POST /threads/{id}/history` is read-only, but you can also _branch_ from any past checkpoint: - `POST /threads/{id}/state` with `{ values, as_node?, checkpoint_id? }` calls `graph.updateState` to write a **new checkpoint** that forks history at `checkpoint_id` (or the tip). It returns the new checkpoint pointer, `{ "checkpoint": { "thread_id", "checkpoint_ns", "checkpoint_id" } }`, and mirrors the forked values onto the thread row. Rejected with `409` while a run is in flight on the thread. - `GET /threads/{id}/state/{checkpoint_id}` reads the state snapshot at a specific checkpoint, and `POST /threads/{id}/state/checkpoint` reads the same thing with the pointer in the body. Both exist because the SDK picks between them by argument _type_: `threads.getState(id, "ckpt-1")` takes the `GET`, while `threads.getState(id, { checkpoint_id, checkpoint_ns })` takes the `POST`. The body pointer preserves `checkpoint_id`, `checkpoint_ns` and `checkpoint_map`. A namespace-only pointer reads that namespace’s current state. `subgraphs: true` includes recursive task state; the GET routes accept `?subgraphs=true`. - Run creation accepts a top-level **`checkpoint_id`** to start a run from a chosen checkpoint instead of the thread tip. A full `checkpoint` pointer or validated `config.configurable` checkpoint coordinates also work. An explicit top-level ID wins. The server always supplies the owning thread ID. Coordinates survive queued execution, so the worker branches from the same checkpoint the caller selected. ### Runs — stateless / ephemeral Each of these creates its own thread. `on_completion` decides what happens to it (below). Cross-thread reads use `POST /runs/search`: status, assistant/thread identifiers, creation/update windows, and bounded offset pagination. See [run search](./runs.md#search-runs-across-threads) for the wire fields, ownership rules, and HTTP example. | Method | Path | Notes | | ------ | -------------- | ------------------------------------------------------------- | | `POST` | `/runs/search` | Cross-thread filters and pagination; authorized before paging | | `POST` | `/runs/wait` | Run to completion, answer with the final values | | `POST` | `/runs/stream` | Run and stream frames as SSE | | `POST` | `/runs` | Queue a background run, answer with the `Run` | | `POST` | `/runs/batch` | An **array** of run-creates; max 100 per request | | `POST` | `/runs/cancel` | `cancelMany` — see below | ### Runs — background (thread-scoped) | Method | Path | Notes | | -------- | -------------------------------------------------- | --------------------------------- | | `POST` | `/threads/{thread_id}/runs` | Start a background run | | `GET` | `/threads/{thread_id}/runs` | List a thread's runs (paginated) | | `GET` | `/threads/{thread_id}/runs/{run_id}` | Fetch one run | | `GET` | `/threads/{thread_id}/runs/{run_id}/stream` (join) | Join a run's stream | | `GET` | `/threads/{thread_id}/runs/{run_id}/join` | Block until it settles, then JSON | | `POST` | `/threads/{thread_id}/runs/{run_id}/cancel` | Cancel a run (`?action`, `?wait`) | | `DELETE` | `/threads/{thread_id}/runs/{run_id}` | Delete a run | | `GET` | `/runs/{run_id}/stream` (join) | Join by run id (thread-agnostic) | **Run-completion deliveries.** A run created with a `webhook` records its callback in an outbox, so the attempts are inspectable and a failed one can be re-sent: | Method | Path | Notes | | ------ | -------------------------------------------------------------------- | -------------------------------------- | | `GET` | `/threads/{thread_id}/runs/{run_id}/deliveries` | `?status=` · `?limit=` · `?offset=` | | `POST` | `/threads/{thread_id}/runs/{run_id}/deliveries/{delivery_id}/replay` | Makes a delivery due again immediately | The list strips the stored `payload` — up to 256 KiB of the run's final state per row — and reports a boolean `replayable` in its place; read the state from the run. Replay `409`s on a delivery that already succeeded (its payload was cleared on success, so there is nothing left to send) and `404`s on a delivery id that does not belong to the run in the path. Both sit in the **`runs` route group** deliberately: a run is already an ownership-scoped resource, so `http.disable_runs` and an existing `@auth.on.threads` handler cover them with no new switch and no new `RouteGroup` — which, once shipped, could never be withdrawn. Delivery semantics, signing and retention are in [webhooks.md](./webhooks.md). **Configured-channel inventory.** When a deployment configures `skein.channels`, it also mounts `GET /channels` for operator tooling. The response is intentionally small and sanitized: ```json { "channels": [ { "route_name": "support-whatsapp", "assistant": "support", "allowed_assistants": ["refunds"], "channel_name": "twilio", "delivery_supported": true } ] } ``` The inbound path is derived as `/channels/{route_name}`. The endpoint never returns module paths, `public_url`, raw configuration, credentials or reply destinations. It is absent when no channel is configured and, under custom auth, requires `assistants:read`. This is a skein operator extension, not part of the Agent Protocol or the upstream SDK resource clients; the console reuses the SDK's base transport for it. **Joining a run: two shapes.** `.../runs/{run_id}/stream` tails a run as SSE (`client.runs.joinStream()`, resumable with `Last-Event-ID`). `.../runs/{run_id}/join` is the blocking form (`client.runs.join()`): it waits for the run to settle and answers the thread's final `values` as plain JSON, or `{ "__error__": ... }` for a failed one — the same envelope `POST /runs/wait` uses. Joining a run that has _already_ settled returns immediately, including long after its frames have aged out of the event bus, because the wait is decided by the run row rather than by the bus. `?cancel_on_disconnect` is honoured on the **streaming** form and accepted-but-ignored on the blocking one, matching `@langchain/langgraph-api` — see `on_disconnect` under the run endpoints below. **Cancelling in bulk.** `POST /runs/cancel` takes `{ thread_id?, run_ids?, status? }`, narrowest selector first: explicit `run_ids`, else one thread's inflight runs, else **every** inflight run on the server. `status` is `pending` / `running` / `all` (the default). An unknown — or non-owned — run id is skipped rather than failing the sweep, so the response reports what actually happened: `{ cancelled_count, cancelled_run_ids, truncated }`. `truncated: true` means the whole-server sweep filled the store's page bound and should be repeated; the SDK types this call as returning `void` and ignores the body, so it is skein's to shape. `truncated` is deliberately a boolean about the caller's own page rather than a count of what is left. The per-thread concurrency guard the sweep reads through is **not** ownership-scoped — it has to see every inflight run on a thread whoever started it — so a total would tell an authenticated caller how much work every other principal has in flight. **`?action` and `?wait` on a cancel.** Both are sent by every `client.runs.cancel(...)`. `action=interrupt` (the default) settles the run `cancelled` and keeps whatever it wrote; `action=rollback` additionally discards its checkpoint writes and deletes the run row, so the turn reads as never having happened. `wait=1` returns only once the run has actually stopped executing rather than as soon as it is marked. **`on_completion` on a stateless run.** `"delete"` removes the server-created thread once the run settles; `"keep"` leaves it. **skein defaults to `keep`, LangGraph to `delete`** — a deliberate divergence, so a stateless run stays inspectable and so adding the field did not silently change what `/runs/wait` and `/runs/stream` already did. Pass `"delete"` for LangGraph's behaviour. An `interrupted` run keeps its thread either way: it has yielded to a human, and its checkpoint is the whole value of the turn. **`if_not_exists` on run creation.** Naming a thread that does not exist is a **404** by default (`if_not_exists: "reject"`, matching LangGraph). Pass `"create"` to have the run bring the thread into existence instead — the other half of addressing a conversation by an external key, so an inbound event can start a run without a round trip to create the thread first. All three thread-scoped run routes agree, and the default is the safe one — a mistyped thread id fails loudly rather than running against a fresh empty thread with none of the history the caller expected. Pass `if_not_exists: "create"` to restore the old behaviour. **`if_thread_status` on run creation.** A skein extension, and the counterpart to `multitask_strategy`: that one arbitrates `pending`/`running`, this one guards every other thread status. It matters most for `interrupted`, which is a **terminal** run status — a thread waiting on a human holds no inflight run, so no multitask strategy protects it and a plain start discards the pending interrupt. Send `if_thread_status: ["idle", "error"]` and an interrupted thread answers **409 `thread_status_mismatch`** with the observed status in `details`. Settled inside the driver's atomic create, so it holds across replicas. A storage driver that does not implement the precondition answers `501 if_thread_status_unsupported` rather than degrading to a non-atomic check. A thread created this way still belongs to the **caller**, not to the run: `on_completion: "delete"` never removes it, because the caller named it. Only a thread the server minted (no `thread_id` at all) is the run's to delete. The field is inert on `POST /runs` and `/runs/batch`, which strip `thread_id` outright — the server owns a stateless run's thread, so nothing can be missing. **`after_seconds` on run creation.** Holds the run for that many seconds before it starts — the SDK's "schedule a future run". Capped at **86400** (a day); anything longer is a [cron](./crons.md). A **background** run (`POST /runs`, `/runs/batch`, `POST /threads/{id}/runs`) is held by the queue itself. Redis keeps delayed jobs in BullMQ's delayed set; SQLite persists unclaimed jobs with their original due times. Both retain waiting jobs across a process restart. On the in-memory queue the delay is a timer, lost on restart along with the other queued jobs. Abandoned active attempts follow each [queue driver's crash behavior](./runs-and-redis.md#queue-drivers). An **inline** run (`/runs/wait`, `/runs/stream`) has no queue to hold it, so the server waits with your connection open. A stream still responds immediately — the run row, its `Content-Location`, and the SSE heartbeats all come first, and only execution is deferred — but `/runs/wait` sends nothing until the run finishes, so a long `after_seconds` there will hit a proxy's idle timeout. Prefer a background run for anything more than a few seconds. Two consequences worth knowing. The run row exists for the whole delay with status `pending`, so it counts as **inflight**: a second run on that thread under the default `multitask_strategy: "reject"` is refused until the delayed one starts. That is the honest reading — work _is_ scheduled on the thread — but it surprises people. And a delayed run is cancellable like any other; `POST .../cancel` settles it before it ever runs. **`on_disconnect` on run creation.** `"cancel"` settles the run when the caller's connection drops; `"continue"` lets it finish. Only the inline routes (`/runs/wait`, `/runs/stream`) hold a connection to drop, which is why the SDK does not send it on a background create. **skein defaults to `"continue"`.** A proxy idle timeout or a load-balancer reset is indistinguishable from a real hang-up at this layer, so defaulting to cancel would let routine infrastructure kill a healthy run. Note `useStream` sends `"cancel"` on every submit unless the stream is resumable — so with a browser client, closing the tab stops the run. That is LangGraph's behaviour. The related query flag `?cancel_on_disconnect` is honoured on `GET .../runs/{run_id}/stream` (`client.runs.joinStream()`), matching `@langchain/langgraph-api`. It stays **accepted and ignored** on the blocking JSON `.../join`, because the reference server does not read it there either — a blocking join should not behave differently against the two servers the same client talks to. Adapters supply the disconnect signal from their own transport (`res` close, or the Web `Request`'s `signal`). An adapter that cannot observe disconnects simply omits it, and `"cancel"` degrades to `"continue"` rather than failing. **`Content-Location` on run creation.** Every run-create response (and `POST /threads/{id}/stream`) carries `Content-Location: /threads/{thread_id}/runs/{run_id}`. The `@langchain/langgraph-sdk` client parses it to fire `onRunCreated`, which is what `useStream` stores to rejoin a stream after a remount — so without it that callback never fires and `reconnectOnMount` cannot work. It is also the only way a caller learns the thread id of a stateless `/runs/wait`, whose body is the graph's state. ### Crons (LangGraph Platform extension) | Method | Path | Notes | | -------- | --------------------------------- | ------------------------------------------------ | | `POST` | `/runs/crons` | Stateless cron — a fresh thread per fire | | `POST` | `/threads/{thread_id}/runs/crons` | Thread cron — reuses the named thread | | `POST` | `/runs/crons/search` | Filter + sort + page; `x-pagination-total` | | `POST` | `/runs/crons/count` | Returns a **bare integer** | | `GET` | `/runs/crons/{cron_id}` | | | `PATCH` | `/runs/crons/{cron_id}` | Tri-state `end_time`/`timezone`; metadata merges | | `DELETE` | `/runs/crons/{cron_id}` | **200 with a JSON body**, not 204 | These are **not** in the open Agent Protocol spec — its `openapi.json` has no cron paths, and the OSS `@langchain/langgraph-api` throws `500 Not implemented` on all of them. skein serves them against the LangSmith Deployment OpenAPI spec plus the SDK's types. Two response shapes are deliberately unusual because the official client requires them: `count` answers a bare integer, and `DELETE` answers 200 with a body (the SDK skips `response.json()` only for 202 and 204). Full semantics — schedule format, catch-up, driver support, the scheduler — are in [crons.md](./crons.md). ### Meta | Method | Path | Notes | | ------ | ------- | -------------------------------------------------------------------- | | `GET` | `/info` | Version + `flags` capability handshake (Studio reads it) | | `GET` | `/ok` | Liveness probe — served by each adapter, **outside** the route table | `/ok` sits outside the protocol table on purpose. LangGraph groups it with `/info` under one `disable_meta` flag; in skein it is the container health check (see the generated Dockerfile), so no config flag may be able to make a healthy instance read as dead. **`/info` is served unauthenticated**, even with an `auth` block configured — matching `@langchain/langgraph-api`, whose auth middleware skips it explicitly. It is a handshake: Studio and monitoring clients probe it before they have credentials, so requiring auth would break connecting to a server `langgraph dev` would have answered. It exposes only versions and which resources are served. ### Store (long-term memory) | Method | Path | Notes | | -------- | --------------------- | ---------------------------------------------- | | `PUT` | `/store/items` | Upsert an item (optional `ttl`) | | `GET` | `/store/items` | Fetch by `?namespace=a.b&key=…` | | `DELETE` | `/store/items` | `{ namespace, key }` in the body, or the query | | `POST` | `/store/items/search` | pgvector semantic search, `filter` → `{items}` | | `POST` | `/store/namespaces` | Prefix/suffix/depth → `{namespaces}` | **`DELETE` takes a body.** The SDK's two single-item methods use different transports on the same path: `store.getItem` sends `?namespace=a.b&key=…`, while `store.deleteItem` sends a JSON body with `namespace` already an array. skein accepts **either** on both, body first. Reading only the query made every SDK `deleteItem` a silent no-op — empty namespace, empty key, nothing deleted, `204`. **Response shapes.** Search returns `{ "items": [...] }` and namespaces `{ "namespaces": [...] }` — the envelopes the SDK's `store.searchItems` / `store.listNamespaces` read. Store items carry `created_at`/`updated_at`, like every other resource here. **`filter` on search** narrows by the **top-level** keys of an item's `value` — `"a.b"` is the literal key `"a.b"`, not a path into a nested object. Keys are ANDed, and so are multiple operators on one key. The operator set is LangGraph's: `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$in`, `$nin`; a bare scalar means equality. Filtering happens before paging, so a page is a page of matches. An unknown operator or a non-scalar filter value is a **400**, not a silent narrowing — see [storage.md](./storage.md) for the full semantics, including the one place skein's ordering operators deliberately differ from LangGraph's. **Namespace matching** takes `prefix` and `suffix` (ANDed) and `max_depth`. `"*"` matches exactly one segment, positionally — so `["users","*"]` selects the `users` subtree, matching `["users","1"]` and `["users","1","memories"]` alike. Before 0.14 a wildcard was dropped and read as _no prefix_, so that same request returned **every namespace in the store**. `max_depth` truncates each match and de-duplicates, before paging. **Pagination on these two.** `GET /threads/{thread_id}/runs` takes `?limit`/`?offset` and `POST /store/namespaces` takes `limit`/`offset` in the body; both default to a **100**-row page, the same default the SDK sends for `store.listNamespaces`. A query `limit` above 1000 is clamped rather than rejected, matching every other query-string limit here. Truncation is not signalled on the response (only assistant search carries `x-pagination-total`), so page until you receive fewer rows than you asked for. `status` on `runs.list` **is** honoured, filtered in the driver so it pages the filtered set; `select` on `runs.list` remains accepted without projection. Store reads and searches honor `refresh_ttl` as a per-request override of `store.ttl.refresh_on_read`. ### Thread streaming (SSE) | Method | Path | MVP | | ------ | ------------------------------------ | --- | | `POST` | `/threads/{thread_id}/stream` | ✅ | | `GET` | `/threads/{thread_id}/stream` | ✅ | | `GET` | `/threads/{thread_id}/stream/events` | ✅ | | `POST` | `/threads/{thread_id}/stream/events` | ✅ | | `POST` | `/threads/{thread_id}/commands` | ✅ | `POST /threads/{thread_id}/stream/events` subscribes to the thread event protocol. Its body contains `channels`, optional namespace prefixes, `depth`, and an optional numeric `since` cursor. It never creates a thread or run. Without a cursor it sends a current state baseline followed by live events; with a cursor it replays retained events after that sequence and then follows live events. An expired or future cursor reports `thread_cursor_expired`; restore state and subscribe afresh. `POST /commands` recognizes `{ id, method, params }` envelopes and answers correlated JSON success/error responses. Implemented methods are `run.start`, `input.respond`, `agent.getTree`, `state.get`, `state.listCheckpoints`, and `state.fork`. Run commands enqueue through the existing worker; a subscription or reconnect never invokes the graph. Existing resume/goto/update bodies continue to return legacy SSE. State reads require read permission, run starts require `create_run`, and forks also require state-write and destination-creation permission. `state.fork` verifies the source checkpoint in the configured saver before copying; custom-agent snapshots do not need a `createdAt` timestamp to identify a valid stored checkpoint. `GET /stream/events` remains an alias for `GET /stream`. Without parameters these join one run and finish with it, preserving Skein's established default. Explicit `stream_mode` (repeated or comma-separated; what the SDK's `threads.joinStream` sends) opts into a thread stream that stays open across turns. `run_modes` includes every run frame and completion metadata. `lifecycle` emits only `metadata` events: `{ run_id, attempt: 1 }` when execution starts and `{ status: "run_done", run_id }` when it ends. `state_update` emits `{ thread_id, state }`, where `state` is the full checkpoint snapshot captured when the run ends or state is updated directly. Replay preserves each historical snapshot. Individual frame modes and the Skein-specific `thread_state` thread-row events remain supported. SSE event IDs are thread-wide cursors, rather than per-run frame IDs. Without `Last-Event-ID` it starts with live events; with one it replays after it. Thread subscriptions require `ProtocolDeps.threadBus`; the built-in runtime assemblers provide it. Memory replay is process-local. Redis replay is shared between API replicas and workers. Neither publisher stores subscriber credentials or subscription filters: authorization and filtering happen for each reader during replay and live delivery. See the [tested compatibility matrix](./langgraph-compatibility.md). > The protocol also describes a WebSocket upgrade for bidirectional streaming. That is > **post-MVP** — SSE covers the full client UX (see [streaming.md](./streaming.md)). ## Request/response conventions - JSON for all non-streaming payloads. - Request bodies carry `input`, optional `metadata`, optional `config`. - Responses carry status (`pending` / `success` / `error`), timestamps, and resource IDs. - A failed run also carries `error` — a skein extension over the SDK's `Run`, which records only _that_ a run failed. See [errors-and-logging.md](./errors-and-logging.md). - Schemas use JSON Schema for interoperability. ## Idempotent run creation (`Idempotency-Key`) A skein extension — LangGraph Platform has no equivalent. Send an `Idempotency-Key` header on a run create and a retry of that request returns the **original response** instead of starting a second run. This exists because every webhook provider retries: Twilio on timeout or 5xx, Stripe, GitHub, Slack, SendGrid. Without it, a retried delivery means a second reply to the end user or two agents acting on one event — and the failure is silent. The only other defence is a dedup table re-implemented inside every caller's application. **Supported on** `POST /threads/{thread_id}/runs`, `POST /threads/{thread_id}/runs/wait`, `POST /runs`, `POST /runs/wait`, and `POST /runs/batch`. Omitting the header is exactly today's behaviour, so this is purely additive. **Not supported on the streaming creates** (`POST /runs/stream`, `POST /threads/{thread_id}/runs/stream`, `POST /threads/{thread_id}/stream`, `POST /threads/{thread_id}/commands`), which answer with a live SSE stream: there is no body to record, and consuming the stream to make one would break it for the caller who asked. Sending the header there is a **422** rather than being ignored — a silently-dropped key would leave you believing your retries are deduplicated while every one starts another run. | Case | Response | | ------------------------------------------------ | -------------------------------------------------------------- | | Key unseen | Executes normally; the response is recorded | | Key seen, same request, original done | The recorded response verbatim, plus `Idempotent-Replay: true` | | Key seen, same request, original still in flight | `409` `idempotency_key_in_flight` — retry shortly | | Key seen, **different** request body | `422` `idempotency_key_reused` — a caller bug; do not retry | | Key over 255 chars, or not printable ASCII | `422` `invalid_idempotency_key` | The `code` carries the meaning on the 422s: these routes already answer 422 for a _transient_ reason (`thread_busy`, under the `reject` multitask strategy), and the two demand opposite client behaviour. Keys follow Stripe's 255-character limit. `Content-Location` is replayed with the rest of the response, so the SDK's `onRunCreated` still fires and `useStream`'s `reconnectOnMount` still works on a retry. A few properties worth knowing: - **The claim is atomic across instances.** The record is inserted first and the store's uniqueness constraint arbitrates, so two provider retries landing on two pods milliseconds apart still produce exactly one run. Use a stable key for retries of the same request. - **Keys are scoped per principal** when auth is configured, so one caller cannot replay another's response by guessing their key. - **A replay is re-authorized.** It answers from the record without reaching the handler table, so the route's `@auth.on.*` check runs again first — and when it returns ownership filters, the recorded thread is re-read through the scoped store. Revoking a caller's access stops their replays at once rather than at the end of the retention window. - **Failures are never recorded.** A create that throws or answers non-2xx releases its key, so the next retry really runs — pinning a transient 503 for the retention would make a momentary outage permanent for that key. - **At-least-once, not exactly-once.** The guarantee is that a retry of the _same_ request does not create a second run. Choose keys that are stable across retries (the upstream provider's message id is usually right). - **Deleting a thread or a run erases its recorded responses.** A recorded response has to outlive its run for a replay to mean anything, and for `POST /runs/wait` that response _is_ the graph's final state — so `DELETE /threads/{thread_id}` takes the matching records with it, as do `DELETE /threads/{thread_id}/runs/{run_id}`, an assistant delete, and thread-TTL expiry. Erasure means erasure; a retry afterwards executes again rather than replaying, and on a deleted thread that is a 404. The one deletion that does **not** erase is a stateless run tidying up its own server-created thread (`on_completion: "delete"`). That thread is an implementation detail of `POST /runs`, disposed of on every such run — scrubbing there would destroy the record moments after writing it and break retries for exactly the case the header exists for. Those records expire on `retention_hours` like any other. Lower it if you want that window shorter. `POST /runs/batch` records carry no thread: their runs may span several, and the body is a list of run rows rather than conversation content. Retention is tunable under `skein.idempotency` in `langgraph.json` — see [langgraph-cli-compat.md](./langgraph-cli-compat.md#idempotency-skeinidempotency). ## Authentication + authorization Auth follows LangGraph's [custom-auth model](https://docs.langchain.com/langsmith/custom-auth) and is **transport-neutral** — it wraps the handler table, so every adapter inherits it identically. Active only when an `Auth` engine is configured (a `langgraph.json` `auth` block or an injected `auth` dep); otherwise the server is unauthenticated. Per request: 1. **Authenticate** — run the user's `authenticate` handler → an `AuthContext`, or `401`. The CLI requires credentials for Studio, regardless of its self-declared `x-auth-scheme: langsmith` header. In loopback `skein dev`, `--dev-ui-access` or `auth.dev_ui_access: true` creates a temporary API key for both Studio and the console. A directly injected `AuthEngine` may choose its own Studio policy. 2. **Authorize** — run the matching `@auth.on.*` handler (priority `resource:action` → `resource` → `*:action` → `*`) → `403` on `false`, else an ownership **filter**. 3. **Dispatch** — with the authenticated `user`. Ownership scoping applies to the `threads` family (threads + their runs): a non-owned row reads as absent (`404`, never `403`), and the filter's values are stamped onto rows it creates. `crons` scope the same way and **fall back to the `threads` handler when no `@auth.on.crons` callback is registered**, since callbacks match by exact event key and a deployment that scoped only threads would otherwise serve crons unscoped. `assistants` is **gate-only** — a handler can deny, but no filter applies, because auto-registered graph assistants have no owner and must stay runnable. Store scoping works through the namespace instead: ### Scoping the store A store item carries no metadata, so an ownership filter has nothing to match on. LangGraph's idiom is to **rewrite** `value.namespace`, and skein honours it on all five store routes: ```ts auth.on("store", ({ user, value }) => { value.namespace = [user.identity, ...(value.namespace ?? []).slice(1)]; }); ``` In-place mutation counts (`value.namespace[0] = …`), and the body namespace wins over query params on `GET`/`DELETE`. `value.key` is honoured the same way on `put`/`get`/`delete`. A rewrite that is not a `string[]` is a **500** (`store_namespace_rewrite_invalid`), not a silent fall-through — a handler that meant to scope and got the shape wrong is exactly the failure this prevents. Assigning `value.key` on `search` or `list_namespaces` is likewise a 500 (`store_key_rewrite_invalid`): there is nowhere for it to land. If you supply your own `AuthEngine`, its `authorize` must return the **same** `value` object for a rewrite to be observable. > **With no store handler registered, nothing narrows a store read.** The namespace is a request > parameter, so any authenticated caller can send `{"namespace_prefix": ["memories"]}` — or omit the > prefix entirely — and read **every** tenant's items. `POST /store/namespaces` likewise returns every > tenant's namespace names. Three traps if you validate rather than rewrite: ```ts // A namespace label is one segment: it must contain neither `.` (the query separator) nor `*` (the // positional wildcard). encodeURIComponent escapes neither, so do it explicitly. const tenantLabel = (identity: string) => encodeURIComponent(identity).replace(/\./g, "%2E").replace(/\*/g, "%2A"); auth.on("store", ({ user, value }) => { const namespace = value.namespace; // server-derived; absent means "every namespace" on search if (!namespace?.length || namespace[0] !== tenantLabel(user.identity)) { throw new HTTPException(403, { message: "Out of scope." }); } }); ``` - **Require a namespace; don't just reject suspicious ones.** A deny handler can refuse a request but never filter the result. An absent prefix reads every tenant, and a `"*"` wildcard matches a strict **subset** of what the shorter literal prefix already returns — so blocking wildcards buys nothing. - **Encode the identity.** `GET /store/items` takes a dot-joined query string, so `alice@corp.com` writes to `["alice@corp.com", …]` but reads back as `["alice@corp", "com", …]`. Emails and OIDC subjects hit this constantly. - **A handler cannot cover `getStore()` inside a graph** — that is not an HTTP request and has no principal. Build the namespace from `config.configurable.langgraph_auth_user_id`, never from model output. `value.namespace` is **server-derived** — set after merging the body, from the field the endpoint will really use. Store bodies are `passthrough`, so otherwise a caller could send a decoy `namespace` that satisfied the handler while the endpoint searched from an absent `namespace_prefix`. ### Where scoping runs **In the database.** An ownership-filtered thread search becomes a metadata containment clause (`metadata @> …` in Postgres, hitting `threads_metadata_idx`), so `limit`/`offset` page owned rows directly rather than reading everything and filtering in JS. The in-process `matchesFilters` check still runs and is what actually enforces ownership. The translation errs **broad**, omitting any clause it cannot express exactly, because a too-strict clause would silently hide rows a caller owns. A **custom** `AuthEngine` whose `matchesFilters` is stricter than its own filters is the exception — pages then come back short; keep the two consistent. **Principal in the run config.** Nodes read `config.configurable.langgraph_auth_user`, `langgraph_auth_user_id` and `langgraph_auth_permissions`. Server-owned and reserved, so a client cannot spoof them, and persisted on the run so a background run resumed elsewhere injects the same principal. With no `auth` configured, no keys are added. Route → resource/action (runs authorize through their owning thread — there is no `runs` resource): | Endpoint(s) | resource\:action | | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------- | | `GET /assistants/{id}`, `/assistants/{id}/schemas` | `assistants:read` | | `POST /assistants/search` | `assistants:search` | | `POST /threads` | `threads:create` | | `GET /threads/{id}`, `/state`, `/state/{checkpoint_id}`; `POST /state/checkpoint`, `/history`; `GET /history`, `.../runs`, `.../runs/{run_id}`, run join (stream or blocking) | `threads:read` | | `POST /threads/search`, `POST /threads/count` | `threads:search` | | `PATCH /threads/{id}`; `POST /threads/{id}/state` (state fork); run cancel; `POST /runs/cancel` | `threads:update` | | `DELETE /threads/{id}`; run delete; `POST /threads/prune` | `threads:delete` | | run create (wait/stream/background/stateless/batch), thread stream / commands | `threads:create_run` | | `GET /info` | _(unauthenticated — see the Meta section)_ | | `PUT/GET/DELETE /store/items`, `/store/items/search`, `/store/namespaces` | `store:{put,get,delete,search,list_namespaces}` | `POST /runs/cancel` sweeps broadly but authorizes narrowly — every run goes back through the ownership-filtered `get`, so a non-owned run reads as absent and is skipped. The per-thread concurrency guard is deliberately **not** ownership-filtered: it must see every in-flight run on a thread whoever started it, or two could execute at once and interleave checkpoint writes. Nothing leaks, since the thread itself is gated. ## References - Agent Protocol repo + OpenAPI — - aegra's Agent Protocol implementation (Python prior art) — ### Execution, search and retention options Run creation accepts optional `timeout_ms`: integer milliseconds from **1** through **2,147,483,647**. It is persisted with the run, including queued, native start/resume, and BullMQ command inputs. The duration begins when execution starts, so queue delay does not consume it. The effective budget is the minimum of this duration, the deployment's run limit, and any remaining native deadline. Timeouts follow the same cancellation and terminal `timeout` settlement path as deployment limits. `GET /info` optionally reports `flags.per_run_timeout: true` when supported. No new configuration key or endpoint is needed; omission preserves existing behavior. Thread responses optionally include `ttl`, `expires_at`, and `ttl_strategy`; store item/search responses optionally include `ttl` and `expires_at`. Durations are minutes. Missing means unavailable; `null` means known no expiry. Refresh-enabled store reads return the updated expiry. These are optional projections, preserving compatibility with custom drivers. Run requests preserve `durability: "sync" | "async" | "exit"` through the queue. It takes precedence over deprecated `checkpoint_during`; explicit `true` maps to `async`, explicit `false` to `exit`, and omission keeps LangGraph’s default. Legacy SSE also recognizes `tasks` and `checkpoints` modes. Thread search supports `exclude_metadata` (one nonempty object or a nonempty list, excluding any matching subset), `sort_by: "state_updated_at"`, `select`, and up to ten named `extract` paths. Extraction accepts roots `values`, `metadata`, `config`, and `interrupts`, dot properties and array indices such as `values.messages[-1].content`; absent values become `null`. Extracted aliases are returned in an `extracted` object, alongside selected fields, and cannot overwrite thread identity. Filters run before pagination; projection and extraction run after authorization. Assistant and cron search support `select`. HTTP assistant name matching is case-insensitive substring matching; direct repository calls retain exact matching unless `nameMatch: "substring"` is requested. Store writes accept `index: false | string[]` and `ttl: null` for an explicit no-expiry item. `GET /store/items?refresh_ttl=false` and store search `refresh_ttl` control sliding expiry per read. Custom stores that cannot honor requested indexing or TTL controls fail explicitly. Thread TTL objects accept `strategy: "keep_latest"` with their TTL minutes. The sweeper retains the thread and current checkpoint in every retained namespace, including referenced child checkpoints and pending writes. State updates refresh that window; active runs are skipped. `delete` remains the default strategy and explicit `ttl: null` pins a thread. See [storage](./storage.md). ### MCP and A2A `/mcp` uses the official MCP SDK’s stateless Streamable HTTP transport. Tools list authorized assistants and their input schemas. Calling a tool creates a run through the same authorized run service; cancellation uses the existing run cancellation machinery. `POST /a2a/{assistant_id}` uses the official A2A SDK’s JSON-RPC handlers and legacy codecs. Discovery is `GET /.well-known/agent-card.json?assistant_id=...`; `DEFAULT_A2A_ASSISTANT_ID` supplies an omitted assistant and `LANGGRAPH_API_URL` supplies the public origin. Supported method families are `SendMessage`, `SendStreamingMessage`, `GetTask`, `CancelTask`, `ListTasks`, `GetExtendedAgentCard` and the four legacy send/stream/get/cancel names. An A2A context maps to a thread. Each new task starts a queued run; a continuation keeps the first run’s task ID. Final output is captured before terminal status on existing run rows, independently of checkpoint pruning and event replay. Optional `RunRepo.recordResult/getResult` support is required before A2A admits work; ordinary Agent Protocol runs remain usable with older drivers. Resume checks the saved checkpoint under the existing execution lease and rejects stale tasks. Cancellation status takes precedence over a computed output. History defaults to the context. The advertised history-scope extension accepts `task` in `configuration.historyScope` for sends, in `params.historyScope` for reads/listing, or through `LangGraph-A2A-History-Scope`. Length is bounded to ten, after scope. Streaming omits response history. Assistant `metadata.a2a.input_modes`/`output_modes` customize nonempty advertised MIME lists. File parts preserve bytes or URLs, MIME and filenames through standard content blocks. Skein advertises HTTP/SSE, history scope and streaming. It does not advertise A2UI, task push notifications or subscription-to-task support. These unsupported methods fail explicitly. `http.disable_mcp` and `http.disable_a2a` remove their route groups. ### Discovery and storage diagnostics `GET /openapi.json` and `/docs` are generated on each request from the mounted routes and the same internal operation contracts used by request validators and typed response constructors. They describe request fields, response schemas, status codes, content types, and deployment-supplied authentication. There is no separate OpenAPI JSON file to update. Graph-defined payloads remain open and are described by the assistant schema endpoints. Set `http.disable_docs: true` to hide `/docs` and `/openapi.json` independently. `http.disable_meta: true` retains its broader meaning: hide docs, `/info`, and `/metrics`. Only literal `true` disables a group; `/ok` remains available. Settings take effect on server startup. `/metrics` exposes host-owned OpenTelemetry collection (501 when unavailable). Authorized `GET /threads/{thread_id}/storage` returns concrete saver payload counts and bytes, or 501 for a custom saver without an inspector. These routes use the common adapter/runtime paths; see [diagnostics](./observability.md#http-diagnostics) for measurement boundaries. --- # Building blocks Use this map to choose the skein-js capabilities your project needs. Start with a working graph, then add conversation state, human approval, background work, or durable deployment as required. Each section explains the user decision and links to a practical guide. ```mermaid flowchart LR Y["Yours
model · prompt · tools"] L["LangGraph
state · nodes · edges · interrupt"] S["skein
runs · threads · memory
schedules · callbacks · streaming"] Y --> L --> S class Y accent ``` Read it through once; after that it is a map — find the pressure you are under and follow the link. ## The model and the prompt — yours skein never sees your model choice. You construct it in your graph, from any LangChain provider, and your key lives in `.env`. One habit worth forming early: keep raw data in state and format the prompt inside the node. State is checkpointed every step, so a rendered prompt in a channel is a copy you pay for on every turn. → [Your first agent](./your-first-agent.md#5-give-it-a-real-model) ## Tools — yours A function plus the metadata the model reads to decide when to call it. That metadata is the model's _only_ context for the decision, so write it for someone who knows nothing else. **The trap:** a failing tool does not fail the run. The error goes back to the model, which will often try again. Bound that in the graph, not in hope. → [LangGraph essentials](./langgraph-essentials.md#tools) ## Control flow — LangGraph's Nodes and edges: what runs, and what runs next. Cycles are expected — an agent loop _is_ a node that routes back to the model. skein serves whatever you compile and wraps none of it, which is why the same graph runs on LangGraph Platform. → [LangGraph essentials](./langgraph-essentials.md) ## A run is one turn One input, one execution of your graph. What differs is how you wait: hold the connection, stream, or take an id and let it work. Two surprises. A run is not tied to the connection that started it — drop the stream and it keeps going. And a second message mid-run does not queue: the default strategy is `reject`, which fails the _second_ run with a 422 and leaves the first alone. → [Runs](./runs.md) ## Checkpoints, and everything they buy Use checkpoints when a conversation must continue from saved state, pause for approval, or branch from an earlier point. Inspect them in the [console](./console.md#time-travel) to test those paths. LangGraph saves state at every node boundary; skein persists it. One mechanism, four consequences: ```mermaid flowchart LR CP[checkpoint at each
node boundary] CP --> A[multi-turn
conversations] CP --> B[pause for
a human] CP --> C[time travel
fork a past turn] CP --> D[crash
recovery] class CP accent ``` > [!WARNING] > **No checkpointer, none of the above** — and nothing warns you. Runs start empty and resume quietly > does nothing. `skein dev`, `embedInMemoryGraphs` and `embedPostgresGraphs` all configure one; the > mistake is constructing your own and handing it to `.compile()`. A related trap arrives with it. A run carries four bags — `input`, `context`, `config.configurable`, `metadata` — and only `input` becomes state the agent remembers. → [State, context & persistence](./state-and-context.md) ## Threads: one conversation A thread is the state plus every checkpoint behind it. Address one by a key you already have — a ticket number, a phone number — and your system needs no mapping table. → [Threads](./threads.md) ## Memory: what outlives the conversation A thread remembers a conversation, not a person. Two stores, two lifetimes: | | Scope | Written by | | ---------------- | ------------------------------------------ | ---------------------- | | **Checkpointer** | One thread | skein, every step | | **Store** | Namespaces you choose, across every thread | Your nodes, explicitly | **The trap:** a user's name in the thread. They start a new conversation and the agent has forgotten. Facts about a person go in the store; facts about this conversation stay in state. → [Memory](./memory.md) · [Storage](./storage.md) ## Pausing for a human The moment your agent can spend money or send email, someone will want to be asked first. `interrupt()` _ends the run_ on a checkpoint — no connection held, no timer, nothing billed while it waits. Approval comes an hour or a week later and the graph resumes where it stopped. A pause that cannot survive a redeploy is not worth much, which is why this is the feature that most often justifies durable storage on its own. → [Human-in-the-loop](./human-in-the-loop.md) ## Watching it think Runs stream over server-sent events, and LangGraph's stream modes map onto the wire unchanged — which is why `useStream` works against a skein server with a URL change and nothing else. A dropped stream reconnects and joins the same run. → [Streaming](./streaming.md) · [Frontend SDKs](./react-sdk.md) ## Work that outlives the request Not everything is a chat turn. - [Background jobs](./background-jobs.md) — hand it over, get an id - [Crons](./crons.md) — schedule runs across your server instances - [Webhooks](./webhooks.md) — your service told when a run settles The webhook contract is worth reading twice: delivery is durable, committed in the same transaction as the run's terminal status, but **at-least-once**. Dedupe on `X-Skein-Delivery-Id` and answer `2xx` only once your own work is committed. ## Config without a redeploy An **assistant** is a named, versioned configuration of one graph. Ship a prompt change as a version, roll it back in one call, graph untouched. → [Assistants](./assistants.md) ## Knowing what it did Connect LangSmith, PostHog, or OpenTelemetry to inspect your agent's production runs in a backend you already use. For a local investigation, first inspect the run's output and state in the [console](./console.md#runs). What skein does not do is judge the output — that is LangSmith or your own harness. Its job is making every run recorded and replayable. → [Observability](./observability.md) · [Errors & logging](./errors-and-logging.md) ## Where this lives in production The default development server uses memory drivers. Durable deployment is a choice: SQLite for one local Node owner, or Postgres/Redis for a shared server. They provide different capabilities: | | Buys you | | ------------ | -------------------------------------------------------------------------------------------------- | | **Postgres** | State, checkpoints, threads, runs, memories and pending webhooks across a restart | | **Redis** | A shared durable run queue and streaming across instances | | **SQLite** | Local resources, checkpoints, unclaimed queued jobs, and delivery outbox records in one owned file | Your graph code is identical either way — drivers are injected, not imported. SQLite records abandoned attempts as failures; applications choose whether to retry or resume. Redis retains its existing stalled-job redelivery behavior. Delivery retries poll the store's outbox and do not require Redis. → [Storage](./storage.md) · [Local deployment](./local-deployment.md) · [Runs & Redis](./runs-and-redis.md) · [Deploy](./deploy.md) ## What was yours all along The model, the prompt and the tools — the reason your agent is _your_ agent. The graph is LangGraph's. Everything else is plumbing skein exists so you do not write. Want it as a lookup table instead of a story? [Features](./features.md). Want the graph model itself? [LangGraph essentials](./langgraph-essentials.md). --- # LangGraph essentials skein-js serves LangGraph.js graphs **unchanged**. It wraps none of the API below, adds no dialect of its own, and the graph you write here runs on LangGraph Platform without an edit. This page is the shallow dive: the parts of LangGraph you need to read and modify your own agent, each with a link to the LangChain docs that own it in full. It is not a LangGraph tutorial. When a section is the one you actually need, [Thinking in LangGraph](https://docs.langchain.com/oss/javascript/langgraph/thinking-in-langgraph) and the reference behind each **Go deeper** link are where to spend the time. ## The smallest graph State, a node, edges and `.compile()` — the first four concepts below, all in the graph [`npm create skein-js`](./scaffolding.md) scaffolds for you: ```ts import { AIMessage, type BaseMessage } from "@langchain/core/messages"; import { MessagesAnnotation, StateGraph } from "@langchain/langgraph"; function echo(state: typeof MessagesAnnotation.State): { messages: BaseMessage[] } { const last = state.messages.at(-1); const text = typeof last?.content === "string" ? last.content : ""; return { messages: [new AIMessage(`echo: ${text}`)] }; } export const graph = new StateGraph(MessagesAnnotation) .addNode("echo", echo) .addEdge("__start__", "echo") .addEdge("echo", "__end__") .compile(); ``` ## State and channels State is the data flowing through your agent, declared as **channels**. `MessagesAnnotation` is the ready-made one for chat — a list of messages that appends. When you need your own shape, declare it: ```ts import { Annotation, MessagesAnnotation } from "@langchain/langgraph"; const State = Annotation.Root({ ...MessagesAnnotation.spec, draft: Annotation(), }); ``` Keep **raw data in state and format prompts inside nodes**. Everything in state is checkpointed on every step, so a formatted prompt stored in a channel is a copy you pay for forever. **Go deeper →** [Graph API](https://docs.langchain.com/oss/javascript/langgraph/graph-api) ## Reducers A channel's reducer decides what happens when a node returns a value for it: replace, or combine. The `messages` reducer appends, which is why returning one message adds it rather than truncating the conversation to a single entry. ```ts // appends to messages — it does not overwrite the list return { messages: [new AIMessage("done")] }; ``` This bites in one specific place: **editing state by hand**, from the API or the [console](./console.md). You write through the same reducers, so patching `{ messages: [...] }` to fix a transcript appends to it. Use `asNode` to attribute the write to a node whose reducer does what you meant. See [state & context](./state-and-context.md#state-what-the-agent-knows-right-now). **Go deeper →** [Graph API](https://docs.langchain.com/oss/javascript/langgraph/graph-api) ## Nodes A node is a plain function. It receives the current state and returns **only the part that changed** — never the whole state. Because it is a plain function, you can import it in a test, call it with a literal, and step through it in a debugger without a server anywhere. Nodes may be async, and they may reach for injected resources — see [persistence](#persistence-the-checkpointer-and-the-store) below. **Go deeper →** [Graph API](https://docs.langchain.com/oss/javascript/langgraph/graph-api) ## Edges and conditional edges Edges say what runs next. `__start__` and `__end__` are the built-in entry and exit points. A straight line needs only `addEdge`; branching — the reason it is a graph at all — uses `addConditionalEdges` with a function that returns the name of the next node: ```ts const graph = new StateGraph(State) .addNode("classify", classify) .addNode("approve", approve) .addNode("send", send) .addEdge("__start__", "classify") .addConditionalEdges("classify", (state) => (state.needsApproval ? "approve" : "send")) .compile(); ``` Cycles are allowed and expected: an agent loop is a node that routes back to the model until there is nothing left to call. **Go deeper →** [Workflows and agents](https://docs.langchain.com/oss/javascript/langgraph/workflows-agents) ## `.compile()` `.compile()` turns the definition into something runnable, and **what it returns is what skein serves** — the value your `langgraph.json` points at: ```json { "graphs": { "agent": "./src/agent-graph.ts:graph" } } ``` Do not pass a `checkpointer` or `store` to `.compile()` yourself. skein injects both per run; see below. **Go deeper →** [Graph API](https://docs.langchain.com/oss/javascript/langgraph/graph-api) ## Prebuilt agents Most agents are the same loop: call the model, run the tool it asked for, call the model again. `createAgent` is that loop, already written: ```ts import { createAgent } from "langchain"; export const graph = createAgent({ model: "anthropic:claude-sonnet-5", tools: [getWeather], }); ``` It returns a compiled graph like any other, so everything skein does — threads, streaming, interrupts, memory — works against it unchanged. > [!NOTE] > You will meet **`createReactAgent`** from `@langchain/langgraph/prebuilt` in older code. It still > works, but it is `@deprecated` as of `@langchain/langgraph` 1.4: it moved to the `langchain` package > and was renamed. The parameter changed too — `llm` became `model`, which also accepts a > `"provider:model"` string — and a dynamic `prompt` function becomes `dynamicSystemPromptMiddleware`. > skein's examples and scaffolder are all on `createAgent`. **Go deeper →** [Agents](https://docs.langchain.com/oss/javascript/langchain/agents) ## Tools A tool is a function plus the metadata a model needs to decide when to call it: ```ts import { tool } from "@langchain/core/tools"; import { z } from "zod"; const getWeather = tool(async ({ city }: { city: string }) => `It's 21°C in ${city}.`, { name: "get_weather", description: "Get the current weather for a city.", schema: z.object({ city: z.string().describe("City name, e.g. 'Nairobi'") }), }); ``` The `description` and the schema's `.describe()` calls are the model's **only** context for the tool. Write them for a reader who knows nothing else, because that is exactly the situation. **Go deeper →** [Tools](https://docs.langchain.com/oss/javascript/langchain/tools) ## `interrupt()` and commands `interrupt()` pauses the graph from inside a node. The run **ends** on a checkpoint — no connection held, no timer running — and resuming later returns your supplied value from the `interrupt()` call: ```ts import { interrupt } from "@langchain/langgraph"; const answer = interrupt({ question: "Send this email?", draft: state.draft }); ``` This is the concept most dependent on the checkpointer: without one, `interrupt()` has nowhere to park and resume silently no-ops. skein exposes resuming as `command: { resume }` on a normal run create, plus `resume` / `update` / `goto`. **Go deeper →** [Interrupts](https://docs.langchain.com/oss/javascript/langgraph/interrupts) · skein side: [human-in-the-loop](./human-in-the-loop.md) ## Subgraphs A compiled graph can be a node in another graph. That is how you keep a large agent readable — a research step, a drafting step, an approval step, each its own graph with its own state, composed at the top. skein serves the outer graph; the nesting is invisible to the protocol. **Go deeper →** [Subgraphs](https://docs.langchain.com/oss/javascript/langgraph/use-subgraphs) ## Stream modes `graph.stream()` takes a `streamMode` — `values`, `updates`, `messages`, `custom`, `events`, `debug` — and you may request several at once. skein maps each of them onto Agent Protocol SSE frames without translation, which is why the LangChain SDKs and `useStream` work against a skein server with only a URL change. **Go deeper →** [Streaming](https://docs.langchain.com/oss/javascript/langgraph/streaming) · skein side: [streaming](./streaming.md) ## Persistence: the checkpointer and the store LangGraph defines two persistence interfaces, and **skein supplies both** — a checkpointer bound to the thread, and a `BaseStore` bridged from whichever [storage driver](./storage.md) you configured. Your nodes reach the store the usual LangGraph way: ```ts import { getStore, type LangGraphRunnableConfig } from "@langchain/langgraph"; // In a node, the store arrives on the config… async function remember(state: State, config: LangGraphRunnableConfig) { await config.store?.put(["users", userId], "profile", { name: state.name }); } // …and inside a tool, where there's no config argument, reach for getStore(). async function saveName(name: string) { await getStore().put(["users", userId], "profile", { name }); } ``` `getStore()` reads the run currently executing, so call it **inside** the function. At module scope there is no run yet and it throws on import. > [!WARNING] > **Do not construct your own checkpointer or store and pass them to `.compile()`.** It is the > single most common way this breaks: the graph then persists somewhere skein does not know about, > so threads, time travel and interrupt-resume all read the wrong state. Let the injection happen. **Go deeper →** [Persistence](https://docs.langchain.com/oss/javascript/langgraph/persistence) · skein side: [state & context](./state-and-context.md), [storage](./storage.md) ## Who owns what | LangGraph owns | skein-js owns | | ------------------------------------- | --------------------------------------------------- | | State, channels, reducers | Threads, runs, assistants and their versions | | Nodes, edges, subgraphs, `.compile()` | The HTTP surface — Agent Protocol, SSE, the console | | Tools and the agent loop | The run queue, multitask strategies, cancellation | | `interrupt()` and commands | Crons, run-completion webhooks, idempotency | | Stream modes | Storage drivers, and injecting them per run | The line matters when something goes wrong: if it is about what your graph _computed_, it is a LangGraph question. If it is about what got _served, stored or scheduled_, it is ours. ## See also - [Building blocks](./building-blocks.md) — the skein-side map of the same territory - [Your first agent](./your-first-agent.md) — build one from an empty directory - [LangGraph CLI compatibility](./langgraph-cli-compat.md) — what `langgraph.json` supports - [Building a runner](./building-a-runner.md) — serving the protocol from something that isn't LangGraph --- # Features What skein-js can do, in one page. Find the row that matches what you're trying to build, then follow the link. ✅ ships today · ⚠️ preview · 🧪 experimental (works, may change in a minor release) · 🗺️ planned. Comparing against LangGraph Platform specifically? [roadmap.md](./roadmap.md) has the same ground organised by parity instead. ## Run agents | Capability | Status | What it gets you | | --------------------------------------------------------------------------------------- | --------------- | ---------------------------------------------------------------------------------------------------------------------------------- | | [Background jobs](./background-jobs.md) | ✅ | Fire-and-forget work: hand skein a job, get an id back, hear the result by webhook | | [Background runs](./runs.md) | ✅ | Return immediately, let the graph keep working, join the stream later | | [Wait / stream runs](./runs.md) | ✅ | Hold the connection for the answer, or stream tokens as they're produced | | [Stateless & batch runs](./runs.md) | ✅ | One-shot calls with no conversation to keep; up to 100 runs per request | | [Multitask / double-texting](./runs.md#multitask-what-happens-to-the-run-already-going) | ✅ | Decide what a second message does to the run already going — all four strategies | | [Cancel & rollback](./runs.md#cancelling) | ✅ | Stop a run, keeping its writes or discarding them | | [Run timeouts](./runs.md#bound-a-runaway-run) | ✅ | Bound a graph that hangs, instead of losing a worker slot | | [Idempotent run creation](./agent-protocol.md#idempotent-run-creation-idempotency-key) | ✅ | A retrying caller can't start the same run twice. No LangGraph Platform equivalent | | [Cron schedules](./crons.md) | ✅ | Fire a graph on a schedule, exactly once across instances, no leader election | | [A graph as a plain endpoint](./serving-a-single-graph.md) | ✅ | `POST /invoke/:graph_id` — body in, final state out, no threads or runs | | [Deep Agents](./recipes/deep-agents.md) | 🧪 experimental | Serve a LangChain `createDeepAgent` graph — files, per-user memory, approval, and subagents streamed live or run in the background | ## Hold a conversation | Capability | Status | What it gets you | | --------------------------------------------------------------------- | ------ | -------------------------------------------------------------------------------------- | | [State, context & persistence](./state-and-context.md) | ✅ | What the agent knows, what's only for this run, and what survives a restart | | [Threads](./threads.md) | ✅ | Conversations that persist, addressable by your own key (a ticket id, a phone number) | | [Streaming (SSE)](./streaming.md) | ✅ | Tokens, tool calls and reasoning as they happen — including a true `events` mode | | [Reconnect & join](./streaming.md) | ✅ | Resume a dropped stream, or tail the same run from a second client | | [Subgraph streaming](./streaming.md#subgraphs-and-subagents) | ✅ | `stream_subgraphs` streams nested graphs and subagents under `mode\|` names | | [Human-in-the-loop](./human-in-the-loop.md) | ✅ | Pause for approval and resume hours later, from anywhere | | [Time travel](./threads.md#time-travel-re-run-a-turn-a-different-way) | ✅ | Fork from any past checkpoint and run forward — "edit and resubmit" | | [Thread state & history](./threads.md#read-the-state) | ✅ | Read, patch, page, copy or prune a conversation's state | | [Thread TTL](./storage.md#thread-ttl) | ✅ | Expire conversations automatically. LangGraph OSS ignores `ttl`; skein doesn't | | [Assistants & versioning](./assistants.md) | ✅ | Ship a prompt change without redeploying, and roll it back in one call | | [`useStream` and friends](./react-sdk.md) | ✅ | The LangChain client ecosystem works unchanged — React, Vue, Svelte, Angular | ## Remember things | Capability | Status | What it gets you | | ------------------------------------------------------------------------------- | ------ | ----------------------------------------------------------------------- | | [The store, via `getStore()`](./memory.md) | ✅ | Memory that outlives a conversation, reachable from inside a graph node | | [Semantic search](./storage.md) | ✅ | pgvector-backed recall, with optional HNSW indexing | | [Filters & namespace traversal](./storage.md#filtering-and-namespace-traversal) | ✅ | Query memory by value, and walk namespaces with wildcards | | [Store TTL](./storage.md#store-item-ttl) | ✅ | Expire memories, optionally refreshing on read | | [Bring your own store](./storage.md#bringing-your-own-store-storeadapter) | ✅ | Point skein at a LangGraph `BaseStore` or your own implementation | | [Memory patterns](./memory.md) | ✅ | Profile vs collection shapes, the dedup trap, recall, background writes | ## Build workflows across your systems LangGraph owns workflow orchestration; it doesn't own provider integrations. Skein channels supply the authenticated sources and durable destinations around the graph. | Capability | Status | What it gets you | | -------------------------------------------------------------------------- | ------ | ---------------------------------------------------------------------------------- | | [Workflows & channels](./channels.md) | ✅ | Turn provider events into real processes with durable outcomes | | [Run-completion webhooks](./webhooks.md) | ✅ | Be told a run finished — durably, so a receiver's redeploy doesn't lose the news | | [Signed callbacks](./webhooks.md#verify-a-callback-is-really-from-you) | ✅ | Receivers can prove a callback is yours and reject replays. The verifier ships too | | [Deliveries & replay](./webhooks.md#see-what-a-callback-did-and-replay-it) | ✅ | See every attempt, and re-send one by hand when it never landed | | [Custom auth](./agent-protocol.md#authentication--authorization) | ✅ | LangGraph's `Auth` model, drop-in, with ownership filters pushed into the query | | [Telemetry sinks](./observability.md) | ✅ | LangSmith, PostHog, OpenTelemetry — or your own `TelemetrySink` | | [MCP endpoint](./agent-protocol.md#mcp-and-a2a) | ✅ | Expose authorized Assistants as tools over Streamable HTTP | ## Operate it | Capability | Status | What it gets you | | ------------------------------------------------------- | ---------- | ------------------------------------------------------------------------------------ | | [The console](./console.md) | ✅ | Test graphs and track channel wiring, interrupts and deliveries at `/console` | | [Postgres + pgvector](./storage.md) | ✅ | Durable server storage with semantic search and automatic migrations | | [Local apps with SQLite](./local-deployment.md) | ✅ | Desktop, CLI, and daemon embedding with persistent resources, queue, and checkpoints | | [Redis queue & pub/sub](./runs-and-redis.md) | ✅ | Durable run queue and cross-instance streaming | | [Multi-instance](./deploy.md#scaling-past-one-instance) | ✅ | Run independent conversations across replicas, with cross-instance cancellation | | [Errors & logging](./errors-and-logging.md) | ✅ | What a failed run reports, and where — wire, log, and callback | | [Deploy anywhere](./deploy.md) | ✅ | One image; guides for Cloud Run, Fly, Railway, Render, AWS, Kubernetes, a VPS | | [Performance tuning](./performance.md) | ✅ | Every knob in one table, plus symptom → knob triage | | [Bun / Deno runtimes](./deploy.md) | ⚠️ preview | Preview deployment targets; use Node for the default production path | ## Build on it | Capability | Status | What it gets you | | -------------------------------------------------- | ------ | ------------------------------------------------------------------------ | | [Framework adapters](./adapters.md) | ✅ | Express, Fastify, NestJS, Next.js, native Fetch — standalone or embedded | | [Embed a graph in code](./embedding.md) | ✅ | No `langgraph.json`, no CLI — bring a compiled graph and mount it | | [Drop-in LangGraph CLI](./langgraph-cli-compat.md) | ✅ | `skein dev` for `langgraph dev`, with your `langgraph.json` unchanged | | [Scaffolding](./scaffolding.md) | ✅ | `npm create skein-js@latest` — a working project, no API key needed | | [Your own adapter](./building-an-adapter.md) | ✅ | Put the handler table on any HTTP framework | | [Your own agent runtime](./building-a-runner.md) | ✅ | Serve the Agent Protocol from something that isn't LangGraph | ## Deliberately not - **WebSocket transport** — SSE covers the client UX and doesn't affect the React SDK. - **`skein deploy` to a hosted platform** — self-hosted by design; there's no managed target. - **Sub-minute cron schedules**, and **backfilling missed occurrences** — see [crons.md](./crons.md#semantics). - **Exactly-once webhook delivery** — delivery is durable and retried, but at-least-once, so every attempt carries a stable dedup key. See [webhooks.md](./webhooks.md#at-least-once-and-what-your-receiver-owes). Something missing? [File an issue](https://github.com/skein-js/skein-js/issues) — compatibility reports are the most useful feedback we get. --- # Assistants An assistant is a named, versioned configuration of one graph. It's how you ship a prompt change or a model swap without redeploying — and how you roll it back when the change was wrong. ## You already have one Every graph in your `langgraph.json` is registered as an assistant at startup, with `assistant_id` equal to `graph_id`. So this works on a fresh server with nothing configured: ```ts await client.runs.wait(threadId, "agent", { input }); // "agent" is the graph id *and* an assistant id ``` If you never need more than one configuration per graph, you can stop reading here. Assistants become useful when you want **several** configurations of the same graph. ## Several configurations of one graph ```ts const support = await client.assistants.create({ graphId: "agent", name: "support-tone", config: { configurable: { system_prompt: "Be concise and formal." } }, }); await client.runs.wait(threadId, support.assistant_id, { input }); ``` Whatever you put in `config.configurable` arrives in your graph as `config.configurable`. That's the seam: the graph stays one deployment, and the assistant decides how it behaves. This is what makes per-tenant or per-plan behaviour tractable — one assistant per tenant, rather than branching inside the graph on metadata you have to thread through every node. ## Versioning and rollback **Every `PATCH` mints a new immutable version.** The live assistant tracks whichever version is current, so changing a prompt is a `PATCH`, and undoing it is one call: ```ts await client.assistants.update(assistantId, { config: { configurable: { system_prompt: "Be warm and brief." } }, }); // now version 2 await client.assistants.getVersions(assistantId); // newest first await client.assistants.setLatest(assistantId, 1); // back to version 1 ``` Rollback points the assistant at an existing version — it doesn't delete the newer one, so you can go forward again. Nothing is lost by trying a change. **Runs record which assistant they used**, so a thread tells you what configuration produced it. What a version does _not_ capture is your graph's code: rolling back an assistant restores config, not the deployment. If the bad change was in a node, roll back the deploy. ## Inspect what a graph expects Useful when you're building a UI against a graph you didn't write, or checking what a config change is allowed to set: | You want | Call | | --------------------------------- | --------------------------------------------- | | Input/output/state/config schemas | `client.assistants.getSchemas(assistantId)` | | The drawable graph | `client.assistants.getGraph(assistantId)` | | Subgraph schemas by namespace | `client.assistants.getSubgraphs(assistantId)` | The [console](./console.md) renders all three — schemas, graph shape, and version history — so it's usually faster to look there than to call them. ## Housekeeping ```ts await client.assistants.search({ graphId: "agent" }); // filter by graph, name or metadata await client.assistants.delete(assistantId); ``` Two things to know before deleting: - **`delete(id, { deleteThreads: true })` cascades** to the threads that assistant owns. Without it, the threads stay and keep their history. - **Deleting a graph-registered assistant doesn't stop the graph.** It's re-registered on the next boot, since registration is keyed on the graph id. ## What you must get right - **`assistant_id` is not a graph id once you create your own.** Runs take an assistant id; a graph id only works because of the startup registration. Store the id you got back from `create`. - **`config.configurable` is the contract with your graph.** Nothing validates that an assistant's config matches what the graph reads — a typo'd key is silently ignored rather than rejected. Check `getSchemas` if you're unsure of the shape. - **`create` defaults to `if_exists: "raise"`.** Pass `"do_nothing"` if you're seeding assistants at startup and want the call to be idempotent. ## Working example There's no dedicated example yet. The [console](./console.md)'s Assistants view is the practical way to see versioning: edit an assistant's config, run it from the playground, then roll back. ## See also - [Runs](./runs.md) — passing an assistant id to a run - [The console](./console.md) — schemas, graph shape, version history in a UI - [LangGraph CLI compatibility](./langgraph-cli-compat.md) — how `graphs` entries become assistants - [Agent Protocol](./agent-protocol.md#assistants) — every assistant endpoint and field --- # State, context & persistence What your agent knows, where each piece of it lives, and what survives a restart. This is the building block everything else rests on — interrupts, time travel, memory and durability are all consequences of it. ## The four bags, and which one you want This is the single most common source of confusion, so start here. A run carries four separate things, and they behave completely differently: | You pass | Goes to | Persisted? | Use it for | | --------------------- | ------------------------------- | --------------------------------- | --------------------------------------------------------------------------- | | `input` | Your graph's **state channels** | **Yes** — checkpointed per thread | The turn itself: the new message, the document | | `context` | LangGraph's **runtime context** | Stored on the run, not the state | Per-run facts the graph reads but shouldn't remember: a locale, a tenant id | | `config.configurable` | The graph's **config** | Stored on the run, not the state | Knobs: which model, which prompt, a feature flag | | `metadata` | The **run/thread row** | Yes, as searchable labels | Finding and filtering runs; not conversation state | ```ts await client.runs.create(threadId, "agent", { input: { messages: [msg] }, // becomes state context: { locale: "en-GB" }, // available to nodes this run config: { configurable: { model: "fast" } }, // knobs metadata: { ticket: "T-1234" }, // searchable label }); ``` The rule of thumb: **if the agent should still know it next turn, it belongs in `input`.** Everything else is per-run. ## State: what the agent knows right now Your graph declares the shape — the channels — and each node returns a partial update. skein-js doesn't define this; it's LangGraph's, unchanged. If the model is new to you, [Thinking in LangGraph](https://docs.langchain.com/oss/javascript/langgraph/thinking-in-langgraph) is the shortest path to it — and its advice to keep **raw data in state and format prompts inside nodes** is worth following, because everything you put in state is what gets checkpointed. The part that surprises people is that **updates go through reducers**. A channel that appends (the usual `messages` reducer) _adds_ what you return rather than replacing it: ```ts // appends to messages — it does not overwrite the list return { messages: [new AIMessage("done")] }; ``` That matters the moment you edit state by hand, from the API or the [console](./console.md): you're writing through the same reducers, so writing `{ messages: [...] }` to fix a transcript will append, not replace. Use `asNode` to attribute the write to a node whose reducer does what you intend. ## Checkpoints: what makes everything else work LangGraph saves the state to a **checkpoint** at node boundaries — each superstep of the graph. That single mechanism is what gives you: - **Multi-turn conversations** — the next run on the thread resumes from the last checkpoint rather than starting empty. - **[Human-in-the-loop](./human-in-the-loop.md)** — `interrupt()` parks on a checkpoint. The run ends; nothing holds a connection or a timer. Resuming reads the checkpoint back. - **[Time travel](./threads.md#time-travel-re-run-a-turn-a-different-way)** — every checkpoint is addressable, so you can read one, fork it, and run forward from the branch. - **Crash recovery** — a process that dies mid-run leaves a committed checkpoint behind. > [!WARNING] > **No checkpointer, none of the above.** Without one, each run starts empty, `interrupt()` has > nowhere to park, and resume silently no-ops — the single most common way this goes wrong. `skein dev` > configures one for you; in code, `embedInMemoryGraphs` / `embedPostgresGraphs` do. Checkpoint history is readable and pageable — see [threads.md](./threads.md#read-the-state). It's also the largest thing you store, which is why `POST /threads/prune` has a `keep_latest` strategy that drops history while keeping current state. ## Short-term vs long-term: two different stores The distinction people miss:
🧵 ### Short-term — the checkpointer Scoped to **one thread**. Holds the conversation's state and its history. Written automatically by the graph on every step; you rarely touch it directly.
🗄️ ### Long-term — the store Scoped to **whatever namespace you choose**, across every thread. Read and written explicitly by your nodes via `getStore()`, with semantic search and TTL.
A user's name belongs in the store, not the thread — otherwise the agent forgets it the moment they start a new conversation: ```ts import { getStore, type LangGraphRunnableConfig } from "@langchain/langgraph"; // In a node, the store arrives on the config… async function remember(state: State, config: LangGraphRunnableConfig) { await config.store?.put(["users", userId], "profile", { name: state.name }); } // …and inside a tool, where there's no config argument, reach for getStore(). async function saveName(name: string) { await getStore().put(["users", userId], "profile", { name }); } ``` `getStore()` reads the run currently executing, so it has to be called inside the function — at module scope there is no run yet and it throws. Both reach the same store — skein bridges its own into every run as a LangGraph `BaseStore`, so nothing here is skein-specific and the same graph runs unchanged on LangGraph Platform. Full patterns — profile vs collection shapes, the dedup trap, recall — are in [memory.md](./memory.md); the mechanism and drivers are in [storage.md](./storage.md). ## What actually persists, and where | Thing | Stored in | Survives a restart? | | ------------------------- | -------------------------------- | -------------------------------------------------------------------------------------- | | Graph state & checkpoints | The checkpointer | With SQLite/Postgres, yes. In memory, no | | Threads, runs, assistants | The store | With SQLite/Postgres, yes. In memory, no | | Long-term memories | The store | With SQLite/Postgres, yes. In memory, no | | Queued & delayed runs | The queue | Unclaimed jobs persist with SQLite/Redis; active attempts follow driver crash behavior | | Pending webhook retries | The delivery outbox in the store | With SQLite/Postgres; the delivery worker polls persisted records after restart | | Live stream frames | The event bus | SQLite uses process-local frames; Redis frames remain replayable within retention | **By default, `skein dev` uses memory drivers** and snapshots to `.skein/` so your threads survive a restart while you work. **For a shared server**, use Postgres and Redis. **For a local app or one Node owner**, SQLite persists resources and checkpoints in one file. It records abandoned attempts as `ProcessLostError` failures without replaying input; applications decide whether to retry or resume. See [storage.md](./storage.md), [local deployment](./local-deployment.md), and [runs-and-redis.md](./runs-and-redis.md). ## Expiring what you don't need State accumulates. Two TTLs bound it: - **Thread TTL** (`checkpointer.ttl`) — expire whole conversations. A per-thread `ttl` overrides the default, and `null` pins a thread forever. [Details](./storage.md#thread-ttl) - **Store TTL** (`store.ttl`) — expire individual memories, optionally refreshing on read. [Details](./storage.md#store-item-ttl) Both are skein going past LangGraph OSS, which drops `ttl` on the floor. ## What you must get right - **`input` is the only bag the agent remembers.** `context` and `configurable` are per-run. Putting a fact in `context` and expecting it next turn is the classic mistake. - **Reserved `configurable` keys are stripped.** `thread_id`, `run_id`, `checkpoint_id`, `checkpoint_ns`, `langgraph_auth_user`, anything starting with `__` — the server owns these, so a client can't redirect a run to another thread or spoof the authenticated caller. Your own keys pass through untouched. - **The authenticated user arrives in `configurable`**, not in `context` — as `langgraph_auth_user`, `langgraph_auth_user_id` and `langgraph_auth_permissions`, stamped server-side and unspoofable. Present only when [auth](./agent-protocol.md#authentication--authorization) is configured. - **The store is not ownership-scoped by default.** An authenticated caller can read every tenant's items unless an `@auth.on.store` handler narrows it. See [scoping the store](./agent-protocol.md#scoping-the-store). - **Big state is expensive.** Checkpoints hold the whole state per superstep, so a graph that keeps large blobs in a channel multiplies them. Keep artifacts in object storage and a reference in state. ## See also - [Threads](./threads.md) — reading, editing, forking and expiring conversation state - [Memory](./memory.md) — long-term memory patterns - [Storage](./storage.md) — `SkeinStore`, drivers, TTL, bring-your-own - [Human-in-the-loop](./human-in-the-loop.md) — what checkpoints make possible - [Building a runner](./building-a-runner.md) — implementing state for a non-LangGraph runtime --- # Threads A thread is one conversation: its message history, its graph state, and every checkpoint along the way. If you're building a chat product, a thread is what a user would call "a chat" — and it's the thing you address instead of tracking state yourself. ## Start a conversation ```ts const thread = await client.threads.create(); await client.runs.wait(thread.thread_id, "agent", { input: { messages: [msg] } }); ``` You rarely need more than that. Threads persist, so the next run on the same id picks up where the last one left off — the graph sees the accumulated state without you passing any of it back. ## Address a conversation by your own key Most apps already have an identity for the conversation — a phone number, a support ticket, an email thread. You don't need to store skein's thread id alongside it. Pass your own id and make the create a **get-or-create**: ```ts const thread = await client.threads.create({ threadId: ticketId, ifExists: "do_nothing", // returns the existing thread instead of a 409 }); ``` Uniqueness is enforced in storage, so two instances racing the same id cannot both win — no lock of your own required. The default is `ifExists: "raise"`, a 409 on a taken id. The other half of the idiom is on run creation: `if_not_exists: "create"` lets a run bring the thread into existence, so an inbound webhook can start a run in one call. See [runs.md](./runs.md). ## Read the state ```ts const state = await client.threads.getState(threadId); // current values, next nodes, tasks const history = await client.threads.getHistory(threadId, { limit: 10 }); // newest first ``` `getState` is what a UI hydrates from on load. `getHistory` walks the checkpoints backwards — page with `before`, passing the last checkpoint you received. **History is capped, and the cap matters.** You get at most 100 checkpoints when `limit` is omitted, and a `limit` above 1000 is rejected — each element is a whole graph state, so a long thread's full history is one of the largest responses skein can produce. `useStream` asks for 10, which is enough to render the transcript (the newest checkpoint carries the whole message list) and only limits how far back the branch/edit tree reaches. ## Time travel: re-run a turn a different way ```mermaid flowchart LR C1[turn 1] --> C2[turn 2] --> C3[turn 3] C2 --> F[edited copy of turn 2] --> F2[turn 3, differently] class F accent ``` The original branch is left intact — a fork is a new checkpoint, not a rewrite. Fork from any past checkpoint and run forward from there, leaving the original branch intact. This is what "edit and resubmit" in a chat UI is built on, and it's also the fastest way to debug what a node actually saw. ```ts // Run again from an earlier checkpoint await client.runs.create(threadId, "agent", { input, checkpointId }); ``` To change the state before re-running, write a new checkpoint that forks history first: ```ts const forked = await client.threads.updateState(threadId, { values: { messages: editedMessages }, checkpointId, // omit to fork from the tip }); // The SDK returns a checkpoint config; the new id is on `configurable`. const forkedId = forked.configurable?.["checkpoint_id"] as string; // `input: null` + a checkpoint resumes the graph from that point rather than starting over. await client.runs.create(threadId, "agent", { input: null, checkpointId: forkedId }); ``` Things to know: - **The fork target is server-validated.** A client cannot redirect a run to an arbitrary checkpoint through `config` — skein reads `checkpoint_id` only from the top-level field. - **`updateState` 409s while a run is in flight** on that thread. Cancel or wait first. - **A fork adds a checkpoint to the thread's history.** If you want a whole independent copy instead, `client.threads.copy(threadId)` duplicates the thread and its history. - The [console](./console.md) does all of this without code — open a checkpoint, edit, fork, run forward. > [!WARNING] > **Values you write go through the graph's reducers.** A channel that appends — a message list, say — > will add what you write rather than replace it. This surprises everyone once. Use `asNode` to > attribute the write to a node whose reducer does what you intend. ## Find threads ```ts await client.threads.search({ metadata: { graph_id: "my_graph" }, status: "interrupted" }); ``` skein stamps a run's `graph_id` and `assistant_id` onto its thread's metadata, so filtering by graph is just a metadata match. The stamp reflects the thread's most recent run — a thread that has never run carries no `graph_id`. Status filters are how you build a "waiting for you" queue: `interrupted` threads are the ones [paused for a human](./human-in-the-loop.md). ## Import an existing conversation Migrating from another system, you usually want the history there without re-running the graph over it. `POST /threads` accepts `supersteps` — updates written straight into the checkpoint history: ```ts await client.threads.create({ graphId: "agent", // required — a brand-new thread has no run to infer the graph from supersteps: [{ updates: [{ values: { messages: [past] }, asNode: "__start__" }] }], }); ``` Each superstep is one tick and becomes one checkpoint. Bounded at 100 supersteps of 100 updates. Without `graphId` this is a 400 — and it's also what lets the seeded state be **read back**, since `getState` falls back to `metadata.graph_id` for a thread that has never run. ## Clean up | You want | Call | | ---------------------------------------------- | ---------------------------------------------------- | | One thread gone, with its runs and checkpoints | `client.threads.delete(threadId)` | | Many gone, by filter | `POST /threads/prune` with `strategy: "delete"` | | To keep the conversation but drop its history | `POST /threads/prune` with `strategy: "keep_latest"` | | Threads to expire on their own | `checkpointer.ttl` — see below | `keep_latest` is the one people miss: it keeps each thread's current state and discards only the checkpoint history behind it. That's usually where the bytes are, and losing it costs you time travel, not the conversation. ### Expiring threads automatically ```json { "checkpointer": { "ttl": { "default_ttl": 43200, "strategy": "delete", "sweep_interval_minutes": 60 } } } ``` Durations are in **minutes**. `POST /threads` takes a per-thread `ttl` that overrides the default, and an explicit `null` **pins** a thread so no TTL ever collects it. **Expiry means "may be collected", not "gone".** An expired thread still reads normally until the sweeper takes it — hiding it early would make a thread with an in-flight run vanish out from under that run. > [!WARNING] > **Deleting a thread deletes any cron scheduled on it.** A thread-scoped [cron](./crons.md) on an > expiring thread stops firing, silently. Pin such threads with `ttl: null`, or use a stateless cron, > which owns no thread to lose. LangGraph OSS drops `ttl` on the floor (it's a Platform feature there), so this is skein going past what the open-source checkpointers do. Full reference: [storage.md](./storage.md#thread-ttl). ## Working example [`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) puts each issue on its own thread, keyed by the issue id, and re-sweeps without duplicating any of them. ## See also - [Runs](./runs.md) — starting work on a thread - [Human-in-the-loop](./human-in-the-loop.md) — the `interrupted` status and resuming - [The console](./console.md) — browsing threads, checkpoints and forks in a UI - [Storage](./storage.md) — where threads and checkpoints actually live - [Agent Protocol](./agent-protocol.md#threads) — every thread endpoint and field --- # Runs A run is one execution of your graph. Starting one is the single most common thing you'll do against skein, and the shape you pick — wait for it, stream it, or queue it — decides everything about how your client has to behave. ## Pick a run mode | You want | Use | You get | | -------------------------------------------- | ------------------------------------------ | ----------------------------------------- | | The answer, and you can hold a connection | `client.runs.wait(threadId, …)` | The final state, as JSON | | Tokens as they're produced (a chat UI) | `client.runs.stream(threadId, …)` | An SSE stream | | To return immediately and check back later | `client.runs.create(threadId, …)` | The `Run` row; work continues server-side | | A one-shot call with no conversation to keep | The same three, with `null` for the thread | skein creates and owns the thread | | To start many at once | `client.runs.createBatch([…])` | Up to 100 runs per request | ```ts // Wait for it const state = await client.runs.wait(threadId, "agent", { input }); // Stream it for await (const chunk of client.runs.stream(threadId, "agent", { input, streamMode: "messages", })) { // … } // Queue it, come back later const run = await client.runs.create(threadId, "agent", { input }); for await (const ev of client.runs.joinStream(threadId, run.run_id)) console.log(ev); ``` **Background runs need somewhere to run.** They execute on the server after your request returns, so they need a process that stays alive — which is why they don't work on serverless platforms. See [what doesn't work on serverless](./deploy-serverless.md). Streaming is its own topic — the stream modes, reconnecting mid-stream, and joining from a second client are in [streaming.md](./streaming.md). ## Search runs across threads `POST /runs/search` returns run rows across threads without enumerating every thread. For example, find currently failed runs updated in the last hour using your deployment's normal auth headers: ```ts const response = await fetch(`${apiUrl}/runs/search`, { method: "POST", headers: { "content-type": "application/json", ...authHeaders }, body: JSON.stringify({ status: "error", updated_after: new Date(Date.now() - 60 * 60 * 1000).toISOString(), limit: 100, offset: 0, }), }); if (!response.ok) throw new Error(`Run search failed: ${response.status}`); const runs = await response.json(); ``` | Field | Meaning | | --------------------------------- | -------------------------------------------------------------------------------------------------- | | `status` | One run status: `pending`, `running`, `success`, `error`, `timeout`, `interrupted`, or `cancelled` | | `assistant_id`, `thread_id` | Exact identifiers; combine them with any other filter | | `created_after`, `created_before` | Creation-time window: inclusive lower bound, exclusive upper bound | | `updated_after`, `updated_before` | Last-update window with the same half-open bounds | | `limit`, `offset` | Default 100 and 0; limit must be 1–1000, offset a nonnegative integer | Timestamps must include a timezone and are normalized to UTC. Invalid values or inverted windows return 400; equal bounds select no rows. Results are ordered by stored `created_at DESC`, then `run_id DESC`. A deployment's lower page cap still applies. To traverse pages, increase the offset by the number returned and stop on an empty page. The response is a plain `Run[]`, without execution kwargs, retained results, or a total count. Existing thread-scoped listing keeps its behavior. Search authorizes through `threads:search` and matches the ownership metadata stamped on each run. All filters, including server-enforced ownership, run inside the storage query before pagination. Custom auth engines should express access through the declared ownership filters. If a stricter custom `matchesFilters` callback rejects a returned row, the entire request returns 501; Skein does not expose that row or silently shorten the page. Custom stores without `RunRepo.search` also return 501. All bundled stores implement it. The tested SDK versions 1.9.25 and 1.12.6 do not expose `client.runs.search()`. Use HTTP as above; the endpoint and its shared validator appear automatically in the deployment's generated `/docs` reference. Programmatic store/service queries use the exported `RunSearchQuery` with camel-case identifiers and time bounds (`assistantId`, `threadId`, `createdAfter`, `createdBefore`, `updatedAfter`, `updatedBefore`). ## Multitask: what happens to the run already going Send a second message before the first finishes ("double-texting") and something has to give. Pass `multitask_strategy` on the second run to say what: | Strategy | The run already going | The new run | Reach for it when | | ----------- | --------------------------- | ------------------ | ------------------------------------------------------------------------------------------ | | `reject` | Keeps going | **Fails with 422** | The default. A second message is a bug or a double-click | | `interrupt` | Stops, keeps what it wrote | Starts now | The user changed their mind, and the partial answer is still worth keeping | | `rollback` | Stops, its writes discarded | Starts now | The user is correcting themselves — the abandoned turn should read as if it never happened | | `enqueue` | Runs to completion | Waits, then runs | Both messages matter and order is what you want | ```ts await client.runs.create(threadId, "agent", { input, multitaskStrategy: "interrupt" }); ``` Things worth knowing before you pick: - **`reject` is the default, and a `pending` run counts as busy.** A run held by `afterSeconds`, or one sitting in the queue, will reject the next one just as a running one does. That surprises people — but work _is_ scheduled on the thread. - **The 422 is `thread_busy`.** Handle it distinctly from an ordinary validation failure: the client's move is to retry or to switch strategy, not to fix the request. - **`enqueue` is not strict FIFO** unless you run at concurrency 1. Concurrent workers can observe queued runs in different orders. See [thread dispatch](./runs-and-redis.md#dispatch-by-thread). - **It's decided atomically**, so two instances racing the same thread cannot both win. You don't need a lock of your own. A displaced run settles `interrupted` (under `interrupt`) or `cancelled` (under `rollback`), and one that never started executing sends no [webhook](./webhooks.md). ## Cancelling ```ts await client.runs.cancel(threadId, runId); // action: "interrupt", the default ``` | `action` | Effect | | ----------------------- | ----------------------------------------------------------------------------------------------------- | | `interrupt` _(default)_ | Settles the run `cancelled` and **keeps** whatever it wrote | | `rollback` | Also discards its checkpoint writes and deletes the run row — the turn reads as never having happened | > [!WARNING] > **`cancel` takes these positionally, not as an options object.** The signature is > `cancel(threadId, runId, wait?, action?)`, so `cancel(tid, rid, { action: "rollback" })` binds your > object to `wait` and silently performs an ordinary `interrupt` instead. ```ts await client.runs.cancel(threadId, runId, true, "rollback"); // wait for it to stop, discard its writes ``` The third argument returns only once the run has actually stopped, rather than as soon as it's been marked. To cancel in bulk, `POST /runs/cancel` takes `{ thread_id?, run_ids?, status? }`, narrowest selector first: explicit ids, else one thread's inflight runs, else **every** inflight run on the server. Unknown ids are skipped rather than failing the sweep, and the response says what actually happened. ## Bound a runaway run A graph that hangs — a model call with no timeout of its own, a node that loops — holds a worker slot until the process restarts. Set `--run-timeout ` or `SKEIN_RUN_TIMEOUT_MS`; the run aborts and settles as `timeout`. **Off by default, deliberately.** A legitimate research or multi-step tool run takes minutes, so a default would turn slow-but-working into killed — the exact failure the timeout exists to prevent. Pick a number from your own graphs' worst honest case. Individual run creation requests can additionally supply `timeout_ms`, an integer from 1 through 2,147,483,647 milliseconds. The duration starts at execution, so a queued delay does not consume it. The effective limit is the smallest of this duration, the deployment limit, and any remaining native deadline. Native start/resume and BullMQ command inputs inherit the same field. Omit it for the existing deployment behavior. `GET /info.flags.per_run_timeout: true` confirms support. The installed LangGraph SDK does not serialize this additional field. Use a submission-scoped `onRequest` hook to add it to the prepared run body, or send the wire request directly: ```ts await fetch(`${url}/threads/${threadId}/runs`, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ assistant_id: "research", input: {}, timeout_ms: 30_000 }), }); ``` The console exposes this control under **Advanced run options** after capability confirmation. ## Start a run later `afterSeconds` holds a run before it starts — capped at 86400 (a day); for anything longer use a [cron](./crons.md). Use it on a **background** run. The queue holds it, so it costs nothing while it waits and, on Redis, survives a restart. On an inline `wait`/`stream` run the server holds _your connection_ open instead, so a long delay will hit a proxy's idle timeout. A delayed run is cancellable like any other, and counts as inflight the whole time it waits. ## Fields where skein differs from LangGraph Two defaults are deliberately not LangGraph's. If you're migrating, these are the ones to check: | Field | skein | LangGraph | Why | | --------------- | ---------- | --------- | ------------------------------------------------------------------------------------------ | | `on_completion` | `keep` | `delete` | A stateless run's thread stays inspectable afterwards | | `on_disconnect` | `continue` | `cancel` | A proxy timeout is indistinguishable from a real hang-up, and shouldn't kill a healthy run | Pass the LangGraph value explicitly if you want its behavior. A client's explicit choice overrides skein's default. The legacy `@langchain/langgraph-sdk/react` hook requests cancellation on disconnect unless its stream is resumable. Dedicated framework bindings use thread subscriptions: closing a subscription does not cancel execution. Use their explicit stop action when cancellation is intended; see [frontend SDKs](./react-sdk.md#handle-pauses-and-failures). One more worth knowing, though it matches LangGraph: **naming a thread that doesn't exist is a 404** (`if_not_exists: "reject"`). Pass `"create"` to have the run bring the thread into existence instead — which is how you start a run keyed on an external identity (a phone number, a ticket id) without a round trip to create the thread first. ## Don't start a run on a thread that's waiting for a human `interrupt()` leaves a thread parked until someone answers, and the run that parked it is **terminal** — `interrupted` is a finished status, so the thread holds no inflight run. That means `multitask_strategy` cannot protect it: every strategy, including the default `reject`, arbitrates `pending` and `running` only. A plain start on an interrupted thread therefore **succeeds**, and the pending question is discarded with no error and no log line. `if_thread_status` is how you say "only if nobody is waiting": ```ts await fetch(`${url}/threads/${threadId}/runs`, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ assistant_id: "support", input: { messages: [{ role: "human", content: text }] }, if_thread_status: ["idle", "error"], }), }); // → 409 { code: "thread_status_mismatch", details: { status: "interrupted" } } ``` The check happens **inside the driver's atomic create**, alongside the one `multitask_strategy: "reject"` already uses — so two replicas racing the same thread cannot both win. An in-process mutex cannot give you this; the window is between processes. The 409 carries the status it actually observed, so you can branch without a second read that might see a third value. The usual branch for an async chat channel is: refused with `interrupted` → resume the pending interrupt instead of starting a new run. ```ts // The reply hours later is an answer, not a new conversation. body: JSON.stringify({ assistant_id: "support", command: { resume: text } }); ``` It composes with `multitask_strategy` rather than overlapping it — that one guards `pending`/`running`, this one guards everything else. Omit the field and run creation behaves exactly as it always has. A storage driver that does not implement the precondition answers `501 if_thread_status_unsupported` rather than quietly falling back to a read-then-create, because a non-atomic fallback looks like it works right up until two replicas race. Both first-party drivers implement it. ## Don't create the same run twice A retrying caller — Stripe, GitHub, Twilio, or your own sweep — should not start a second run. Send an `Idempotency-Key` header and the original response replays: ```ts // `runs.create` has no `headers` option — a `headers` key in the payload is silently dropped and you // get a brand-new run every retry. A per-key client is the only way to send it. const client = new Client({ apiUrl, defaultHeaders: { "Idempotency-Key": issue.id } }); await client.runs.create(threadId, "agent", { input }); ``` Streaming creates reject the header rather than ignoring it — an SSE response has no body to replay. Details: [agent-protocol.md](./agent-protocol.md#idempotent-run-creation-idempotency-key). ## Reading a run back | You want | Call | | ---------------------------------------- | ----------------------------------------- | | One run's row | `client.runs.get(threadId, runId)` | | A thread's runs | `client.runs.list(threadId)` | | To tail a run already in flight | `client.runs.joinStream(threadId, runId)` | | To block until it settles, then get JSON | `client.runs.join(threadId, runId)` | Joining a run that has **already** settled returns immediately, long after its frames have aged out — the wait is decided by the run row, not by the event stream. So a client that reconnects late still gets its answer. To be told a run finished without holding anything open, use a [webhook](./webhooks.md). ## Working example [`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) dispatches one background run per issue, each on its own thread, with idempotent re-sweeps — and runs with no API key and no network. ## See also - [Streaming](./streaming.md) — stream modes, reconnecting, joining - [Runs & Redis](./runs-and-redis.md) — run concurrency, the queue, scaling past one instance - [Human-in-the-loop](./human-in-the-loop.md) — pausing a run for a person - [Crons](./crons.md) — running a graph on a schedule - [Agent Protocol](./agent-protocol.md#runs--stateless--ephemeral) — every run endpoint and field --- # Background jobs Fire-and-forget work: hand skein a job, get an id back immediately, and hear about the result later. No conversation to maintain, no connection to hold open. This is the shape you want when the caller is another service rather than a person — classification, enrichment, document processing, a queue drained by an agent. There is no separate "tasks" API. A background job **is** a run — you just don't keep the thread. ## Enqueue and return ```ts const run = await client.runs.create(null, "agent", { input: { documentUrl }, onCompletion: "delete", // don't keep the thread once it's done webhook: "https://example.com/hooks/done", // tell me when it lands }); // returns in milliseconds; the graph runs server-side ``` `null` for the thread id means **stateless**: skein creates a thread for the run, and `onCompletion: "delete"` removes it once the run settles. Nothing accumulates. Leave `onCompletion` off (skein's default is `keep`) if you'd rather the job stay inspectable afterwards — you can read its state and its checkpoints for as long as you keep it. ## Hearing the result Three ways, in order of how much they cost you: | Approach | Use when | | ----------------------------------- | -------------------------------------------------------------- | | A [webhook](./webhooks.md) | The default. Nothing to poll, and delivery survives a redeploy | | `client.runs.join(threadId, runId)` | You can hold a connection and just want the answer | | `client.runs.get(threadId, runId)` | You already have a polling loop | The webhook is the one to reach for. It's recorded in the same transaction as the run's terminal status, so a receiver that was restarting doesn't lose the news — and you can [see and replay](./webhooks.md#see-what-a-callback-did-and-replay-it) a callback that never landed. If you kept the thread (`onCompletion: "keep"`), joining works long after the run settled — the wait is decided by the run row, not by a live stream. If you deleted it, the webhook payload is the record; that's exactly why the payload is stored rather than re-rendered at send time. ## Don't run the same job twice A retrying caller — a queue, a cron, a partner's webhook — will re-send. Send an `Idempotency-Key` and the original response replays instead of starting a second run: ```ts // `runs.create` has no `headers` option, so a per-key client is the only way to send it. const client = new Client({ apiUrl, defaultHeaders: { "Idempotency-Key": documentId } }); await client.runs.create(null, "agent", { input, onCompletion: "delete" }); ``` The claim is arbitrated by a uniqueness constraint, so 50 concurrent retries across two instances still produce exactly one run. Details: [agent-protocol.md](./agent-protocol.md#idempotent-run-creation-idempotency-key). ## Many, later, or on a schedule ```ts // Up to 100 at once await client.runs.createBatch(items.map((item) => ({ assistantId: "agent", input: item }))); // Start in 30 seconds (capped at a day) await client.runs.create(null, "agent", { input, afterSeconds: 30 }); // Every five minutes, forever await client.crons.create("agent", { schedule: "*/5 * * * *", input: { source: "queue" } }); ``` A delayed background run is held by the queue, so it costs nothing while it waits and — on Redis — survives a restart. For anything longer than a day, use a [cron](./crons.md). ## How many run at once One instance executes **10** background runs concurrently by default, matching the LangGraph CLI's `--n-jobs-per-worker`. Raise it with `--concurrency` or `SKEIN_RUN_CONCURRENCY` when your jobs are I/O-bound (model calls); add instances when they're CPU-bound. Concurrency is the knob with the widest blast radius — it multiplies memory and Postgres connections at once. Size it against your pool: [runs-and-redis.md](./runs-and-redis.md#run-concurrency). ## What you must get right - **A failed job is not retried.** skein retries webhook _delivery_, not the run. A graph that throws settles `error` and stays there. If your work needs retries, either retry inside the graph, or have your caller re-submit with a fresh idempotency key. - **A crashed worker follows its queue's contract.** Redis can redeliver stalled work, skipping already terminal runs. SQLite preserves unclaimed jobs but marks abandoned attempts as `ProcessLostError` failures; applications choose whether to retry. Memory loses queued jobs. - **Background runs need a process that stays alive.** They execute after your request returns, so they don't work on serverless. See [deploy-serverless.md](./deploy-serverless.md). - **Use durable storage.** Choose SQLite for one local owner, or Postgres/Redis for shared workers. See [local deployment](./local-deployment.md) and [runs-and-redis.md](./runs-and-redis.md). - **Bound a job that hangs.** A graph with no timeout of its own holds a worker slot until the process restarts. Set `--run-timeout` / `SKEIN_RUN_TIMEOUT_MS`; it's off by default. - **With Postgres, each job holds a connection** while it runs, and a second one on that driver for its thread claim. Ten concurrent jobs is not one connection. ## When you want the answer inline instead If the caller can wait and there's genuinely no conversation — a classifier, an extractor — skip runs entirely: ```bash curl -sX POST localhost:2024/invoke/triage \ -H 'content-type: application/json' \ -d '{"text":"Refund charge failed — urgent!"}' ``` `POST /invoke/:graph_id` takes the graph's input as the body and answers with its final state. No threads, no run rows, no webhooks. See [serving-a-single-graph.md](./serving-a-single-graph.md). ## Working example [`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) is exactly this shape: a cron sweep dispatches one durable background run per item, idempotently, so re-sweeping creates nothing. It runs with no API key and no network. ## See also - [Runs](./runs.md) — every run mode and field - [Webhooks](./webhooks.md) — being told a run finished - [Crons](./crons.md) — running a graph on a schedule - [Runs & Redis](./runs-and-redis.md) — the queue, concurrency, scaling out - [Serving a single graph](./serving-a-single-graph.md) — the synchronous non-chat surface --- # Human-in-the-loop Pause a run for a person to approve, edit, or answer something — and resume it later, possibly hours later, from a different client. An interrupted run holds no connection and no timer, only a checkpoint, so your process can restart while it waits. ```mermaid flowchart LR A[create run] --> B["interrupt()"] --> C["run ends
nothing held open"] C --> D["hours, or weeks"] --> E[resume] --> F[done] class C accent ``` ## Pause the graph Call LangGraph's `interrupt()` from a node. Whatever you pass becomes the payload your UI renders: ```ts import { interrupt } from "@langchain/langgraph"; function askForApproval(state: State) { const answer = interrupt({ question: "Send this email?", draft: state.draft, }); return { sent: answer === true }; } ``` The run settles, the thread's status becomes `interrupted`, and the interrupt surfaces in the stream. Nothing is holding anything open. ## Resume it ```ts await client.runs.create(threadId, "agent", { input: null, // not a new turn — pick up where the interrupt parked command: { resume: true }, }); ``` `input: null` plus a `command` is the whole idiom. The run resumes from the checkpoint the interrupt parked on rather than starting over, and the value you pass in `resume` is what `interrupt()` returns inside the node. Three commands are available: | Command | Does | Use for | | -------- | ------------------------------------------- | ------------------------------------------ | | `resume` | Returns a value from the `interrupt()` call | Approve / reject / supply an edited answer | | `update` | Writes state before continuing | Correcting something the graph got wrong | | `goto` | Continues at a named node | Sending the turn down a different path | If you're building on `POST /threads/{id}/commands` directly, it infers the assistant from the thread's last run, so you don't have to track it — and it **409s (`thread_not_interrupted`)** on a thread that isn't paused, rather than quietly starting a new turn. ## In a React UI, this is free [`useStream`](./react-sdk.md) renders the interrupt and resumes it for you — no extra endpoints, no polling. The interrupt arrives on the stream, you render your approval card, and submitting a command continues the same conversation. ## Find what's waiting An approvals queue is a thread search: ```ts const waiting = await client.threads.search({ status: "interrupted" }); ``` The [console](./console.md) ships this view already — its Overview lists what's waiting for you, and Playground and thread inspection show a **Waiting for you** request. Submit the boolean or JSON response your graph expects, then inspect the resumed run. You can test this before building your own UI. ## What you must get right - **A checkpointer is required.** Without one there is no checkpoint to park on, and resume silently no-ops — the single most common way this fails. `skein dev` gives you one; in code, use `embedInMemoryGraphs`/`embedPostgresGraphs` or supply your own. See [embedding.md](./embedding.md). - **Use durable storage for anything real.** With the in-memory driver an interrupted thread is gone on restart, which defeats the point of pausing for a human. [Postgres](./storage.md) keeps it. - **The pause has no timeout.** A thread waits indefinitely. If your product needs "auto-reject after 48 hours", that's a [cron](./crons.md) sweeping `status: "interrupted"` threads — skein won't do it for you. - **Resuming starts a new run.** The interrupted run is terminal; the resume is a fresh run on the same thread. So a webhook fires per run, and the run id you resumed with is not the id you get back. - **A plain start on a waiting thread is not refused for you.** Because the interrupted run is _terminal_, the thread holds no inflight run — so `multitask_strategy`, which arbitrates `pending`/`running`, never sees it, and a fresh start **succeeds** and discards the pending question. Pass [`if_thread_status`](./runs.md#dont-start-a-run-on-a-thread-thats-waiting-for-a-human) when the caller might not know the thread is waiting, which is every asynchronous channel: ```ts // 409 `thread_status_mismatch` instead of silently trampling the interrupt. { assistant_id: "support", input, if_thread_status: ["idle", "error"] } ``` The check runs inside the driver's atomic create, so it holds across replicas — an in-process lock cannot give you this. - **Not available on `POST /invoke/:graph_id`**, which has no thread to park on. See [serving-a-single-graph.md](./serving-a-single-graph.md). ## Interrupting from outside the graph You can also pause without an `interrupt()` call, by naming nodes on run creation: ```ts await client.runs.create(threadId, "agent", { input, interruptBefore: ["send_email"] }); ``` `interruptBefore` / `interruptAfter` take a list of node names, or `"*"` for every node. This is a debugging and stepping tool more than a product feature — `interrupt()` is what you want when the pause is part of the design, because it carries a payload the UI can render. ## Working examples - [`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) — reached by a conditional edge, so only items that need a human park. Runs with no API key and no network; approve from the console. - [`chat-app`](https://github.com/skein-js/skein-js/tree/main/examples/chat-app) — an approval card in a real Next.js UI. ## See also - [Threads](./threads.md) — the `interrupted` status, and editing state before you resume - [Runs](./runs.md) — what a resume run is, and multitask strategies - [Frontend SDKs & `useStream`](./react-sdk.md) — rendering and resuming in React - [The console](./console.md) — approving without building a UI - [Building a runner](./building-a-runner.md) — implementing interrupts for a non-LangGraph runtime --- # Run-completion webhooks Tell another service that a run finished, and have that news survive a receiver that was redeploying when it happened. Pass a `webhook` URL on run creation and skein POSTs the settled run to it — durably, with retries, optionally signed, and replayable by hand when it never lands. ## Quick start ```ts await client.runs.create(threadId, "agent", { input, webhook: "https://example.com/hooks/run", }); ``` That is the whole opt-in. The field is LangGraph's, unchanged; everything below is delivery semantics, not a different payload. It works on every run-creation route that has a run to report on — background, `wait` and `stream`, stateless, batch, and each firing of a [cron](./crons.md). It is not available on [`POST /invoke/:graph_id`](./serving-a-single-graph.md), which has no run row. ## What the callback carries The settled run, as a JSON body: the run's own fields (`run_id`, `thread_id`, `assistant_id`, `metadata`, …) plus | Field | What it is | | ------------------------------- | ----------------------------------------------------------------------------------------- | | `status` | The run's terminal status — `success` · `error` · `timeout` · `interrupted` · `cancelled` | | `values` | The run's final state | | `error` | The failure message, on a failed run only (a string, matching LangGraph) | | `interrupts` | The questions the run is waiting on, on an `interrupted` run only — see below | | `reply` | What the graph declared should be sent back, if it declared one — see below | | `run_started_at`/`run_ended_at` | When the run ran | | `webhook_sent_at` | When **this attempt** was sent — it changes between retries | `status` is read from the delivery row rather than from the stored body, so a callback can never disagree with the run you read back afterwards. A cancel that won the race is reported as the cancel. ### `interrupts` — the question a paused run is asking A run that hits `interrupt()` settles as `interrupted` and waits for an answer. The question itself is **not in `values`** — it is the interrupt's payload, which lives on the thread's pending tasks — so a receiver holding only the callback could not render it, and a question nobody asks is a conversation that waits forever. `interrupts` carries it, keyed by the task that raised it, in the same shape [`GET /threads/{id}`](./threads.md) returns: ```jsonc { "run_id": "…", "status": "interrupted", "values": { "…": "…" }, "interrupts": { "task-1": [{ "id": "…", "value": { "question": "Refund £40?" } }] }, } ``` The field is **absent entirely** on any other status, so a run that succeeded sends exactly the body it sent before this existed. Both `values` and `interrupts` count toward [`max_payload_bytes`](#what-this-stores-and-for-how-long); if the body is over the cap, `values` is replaced by a truncation marker first and `interrupts` only if that was not enough — you can act on a question without the state, but not on the state without the question. **A run that never started executing sends no callback.** A run cancelled while still `pending`, or one displaced by a [multitask strategy](./runs.md#multitask-what-happens-to-the-run-already-going) before it began, has nothing to report — so do not treat "a run was created" as "a callback is owed". Only a run that started does. Every request also carries three headers, whenever the store records deliveries — which both bundled drivers do: | Header | Use | | --------------------- | ----------------------------------------------------------- | | `X-Skein-Delivery-Id` | Stable across every retry — **this is your dedup key** | | `X-Skein-Attempt` | Which attempt this is, counting the first inline one as `1` | | `X-Skein-Signature` | Present when a signing key is configured — see below | The one exception is a custom store that doesn't record deliveries — see [the guarantee](#the-guarantee-why-a-callback-survives-a-crash) below. ## The guarantee: why a callback survives a crash ```mermaid flowchart LR R[run settles] --> TX subgraph TX["one transaction"] ST[terminal status] --- D[delivery row] end TX --> I[inline attempt] I -->|2xx| Done[done] I -->|fails| Q[retry schedule] Q --> Done class TX accent ``` **The callback is recorded in the same transaction as the run's terminal status.** A crash between "the run finished" and "someone was told" cannot lose it, because there is no instant where one is committed and the other is not. That guarantee comes from the store. It is not the retry policy, and it is not something you configure — a run that carries a `webhook` owes a callback whether or not you have tuned anything. skein attempts the first delivery inline, so a healthy receiver hears within milliseconds; only a failure falls through to the retry schedule. A [bring-your-own store](./storage.md) that does not implement the `deliveries` repo degrades to the older best-effort path — one POST, failures logged and swallowed. Both bundled drivers implement it, so this only bites a store you wrote yourself. > [!WARNING] > That fallback POST carries **no `X-Skein-Delivery-Id`, no `X-Skein-Attempt` and no > `X-Skein-Signature`** — there is no delivery row for those to come from. A receiver that dedupes on > the header, or rejects unsigned requests, will find nothing to work with. If you implement a custom > store and want callbacks at all, implement the `deliveries` repo. The body is **stored**, not re-rendered at send time. It has to be: a stateless `POST /runs` deletes the thread the server made for it as soon as the run settles, so an hour later there is no run row and no checkpoint left to render from — and that is exactly the "another service drives skein and wants the answer back" case webhooks exist for. Each POST is aborted after `SKEIN_WEBHOOK_TIMEOUT_MS` (default 5s), sized against the 8s shutdown budget so a slow receiver cannot get the server killed mid-POST. Delivery happens after the thread's execution lock is released, so a slow target never blocks other runs on that thread — which also means two callbacks for one thread are not guaranteed to arrive in run order. ## At-least-once, and what your receiver owes **Delivery is at-least-once, never exactly-once.** A POST that timed out _after_ your receiver committed is indistinguishable from one that never arrived, so it is retried. Dedupe on `X-Skein-Delivery-Id`, which is stable across every attempt. Answer `2xx` only once the work is durably done. Answering early and then failing loses the callback — the sender heard success and cleared the payload. A non-2xx is how you ask for a retry. The API remains the source of truth for run state. A callback is a notification, not a ledger. ## Where the retries actually run Durable stores retain both delivery payloads and scheduling timestamps. A separate delivery queue changes how retries are scheduled, rather than whether those timestamps survive: | Deployment | Retry scheduling | Restart behavior | | ---------------------------- | ----------------------------------------------- | ---------------------------------------------------------------------------- | | Postgres + Redis | BullMQ delayed jobs and recovery sweep | Waiting jobs survive in Redis; persisted outbox rows recover lost scheduling | | Postgres or SQLite, no Redis | Polling persisted due times and delivery leases | Waiting retries and abandoned delivery leases survive in the database | | Memory | Polling the in-memory outbox | Delivery rows and due times are lost with the process | The polling worker needs no SQLite-specific delivery queue. The host starts and stops it through the existing protocol runtime lifecycle. A process killed at exactly the wrong moment can leave a committed delivery with no schedule attached. Nothing is lost: a background sweep picks it up on its next pass. The practical consequence is that such a callback arrives late rather than never — worth knowing if you alert on delivery latency. The default is 12 attempts over roughly 34 minutes, which rides out a rolling deploy with room to spare. Read that as a **time horizon, not a count** — the delays double, so dropping to 6 attempts is ~31 seconds, which does not survive one redeploy. Tune it under [`skein.webhooks`](./langgraph-cli-compat.md#webhooks-skeinwebhooks). ## Verify a callback is really from you Set `SKEIN_WEBHOOK_SECRET` and every callback carries `X-Skein-Signature` in Stripe's format — `t=,v1=` over `"{t}.{raw body}"` — so a receiver can reuse verification code it already has. skein ships the verifier too, from a package that installs with no graph runtime, so a service whose only relationship with skein is _receiving_ callbacks can depend on it for this and nothing else: ```ts import { verifySkeinSignature } from "@skein-js/agent-protocol"; const result = verifySkeinSignature({ header: req.headers["x-skein-signature"], body: rawBody, // the RAW bytes — see below secrets: [process.env.SKEIN_WEBHOOK_SECRET!], }); if (!result.ok) return res.status(401).end(); ``` It refuses with a `reason` of `missing`, `malformed`, `stale` or `mismatch`, compares digests in constant time, and tries every secret you hand it. Because the signature covers the timestamp, a **replay is detectable with no state on your side**: a captured callback stops verifying once it is outside the tolerance window (5 minutes by default, `toleranceSeconds`). The check is absolute, so a callback timestamped in the future is refused too. ### `reply` — letting the graph say what to send back Anything turning a run into an outbound message — a chat reply, a notification, an email — has to know where the answer lives. Only two parties could decide: your receiver, or the graph. A receiver that guesses (`state.answer`? `state.messages.at(-1)`? `state.draft`?) stops working the moment you change the graph's state shape. So let the graph declare it, on the custom stream: ```ts import { replyWith } from "@skein-js/agent-protocol"; // inside a node, with LangGraph's StreamWriter in scope writer(replyWith("Your order ships Tuesday.")); ``` It arrives as `reply` in the callback body, and the run must request the custom stream (`stream_mode: ["values", "custom"]`) for the writer's output to be seen. The field is absent unless the graph declared one, so nothing changes for a graph that doesn't — and it **adds** to the body rather than replacing `values`, so a receiver that already reads state keeps working. Declaring it on the stream rather than in state is deliberate: the reply is a message _about_ the run, not part of it, and putting it in state would make it a channel every reducer and checkpoint carries forever. skein captures it into the stored delivery as the run settles, so — unlike anything read off a live stream — it survives a crash and is replayed verbatim on a retry. Last write wins. ### Verifying somebody else's signature `verifySkeinSignature` only understands skein's own format. If you are on the _other_ side — checking a signature from Twilio, Slack, GitHub or Stripe as it arrives at a route of yours — the one piece worth borrowing rather than writing is the comparison: ```ts import { equalsConstantTime } from "@skein-js/agent-protocol"; const expected = createHmac("sha1", authToken).update(signable, "utf8").digest("base64"); if (!equalsConstantTime(expected, received)) return res.status(401).end(); ``` `===` on a signature is a timing oracle — it returns at the first differing byte, so an attacker who can send many requests and measure your response recovers a valid signature a byte at a time. The length check inside is not decoration either: Node's `timingSafeEqual` **throws** on unequal lengths, so the obvious hand-rolled version turns a forged short signature into a 500 instead of a 401. ### Three things receivers get wrong, in the order they get them wrong 1. **Verifying a re-serialized body.** `JSON.stringify(req.body)` does not reproduce what was signed — key order, whitespace and number formatting are not guaranteed to round-trip — so _every_ genuine callback fails. In Express: `express.raw({ type: "application/json" })` on that route, or `express.json({ verify })`. 2. **Not deduping.** See at-least-once above. `X-Skein-Delivery-Id` is the key; `X-Skein-Attempt` tells you it is a retry. 3. **Answering 2xx before the work is durable.** The sender clears the payload on success. There is nothing left to resend. A runnable receiver doing all three correctly — accepting a genuine callback, refusing a forged one and a replayed one — is `src/webhook-receiver.ts` in [`examples/express-basic`](https://github.com/skein-js/skein-js/tree/main/examples/express-basic) (`pnpm webhook-receiver`). ### Rotating the key Set `SKEIN_WEBHOOK_SECRET` to a comma-separated list, **new key first**: skein signs with the first and receivers accept any they hold. Remove the old key only once every receiver has the new one — the other order refuses every callback until they catch up. > [!WARNING] > If you inject your own `webhookDispatcher`, it must forward `attempt.headers` and send > `attempt.body` verbatim, or callbacks go out unsigned while your config says they are signed. skein > warns at startup when it sees both a signing key and a custom dispatcher. ## See what a callback did, and replay it When a receiver was down for longer than the retry horizon, the delivery is `dead` — and still on disk, which is what makes a by-hand replay possible. | Method | Path | Notes | | ------ | -------------------------------------------------------------------- | ----------------------------------- | | `GET` | `/threads/{thread_id}/runs/{run_id}/deliveries` | `?status=` · `?limit=` · `?offset=` | | `POST` | `/threads/{thread_id}/runs/{run_id}/deliveries/{delivery_id}/replay` | Makes it due immediately | ```bash curl "$API/threads/$TID/runs/$RID/deliveries?status=dead" curl -X POST "$API/threads/$TID/runs/$RID/deliveries/$DID/replay" ``` Each row reports its `status` (`pending` · `delivering` · `delivered` · `dead`), `attempt`, `last_error`, `next_attempt_at` and `payload_truncated`. The list reports a boolean **`replayable`** rather than the payload itself. The payload is up to 256 KiB of the run's final state _per row_, and this is a list — returning it would make an operator's "what failed?" the heaviest response the server can produce, for a field they did not ask for. The question a replay turns on is only whether there is anything left to send. Read the state from the run. Replay makes a delivery due immediately. It **409s** on one that already succeeded (its payload was cleared on delivery, so there is nothing to resend) and **404s** on a delivery id that does not belong to the run in the path — so an id cannot be used to reach a callback on a run you have no access to. Both endpoints sit in the `runs` route group, so `http.disable_runs` and your existing `@auth.on.threads` handler already cover them; no new switch, no new resource. The console shows deliveries on each run page, including failed attempts and a confirmed Replay action. ## What this stores, and for how long A retry has to have something to send, so **the callback body is stored** — including the run's final `values`. That is a new at-rest copy of whatever your graph produced, in the `deliveries` table, and it is worth classifying deliberately if that state carries personal or regulated data: - **On success it is cleared immediately**, so steady-state storage is in-flight deliveries, not every delivery ever made. - **A `dead` delivery keeps it** for `retain_hours` (default 24) — that is what makes replay possible, and it means a callback that never landed leaves the run's final state on disk for a day. - **It is not encrypted by skein.** Use your database's encryption at rest; the column is ordinary `jsonb`. - **Deleting a thread or run does not erase its deliveries** — unlike idempotency records, which are erased explicitly. Lower `retain_hours` if that matters to you. Payloads are capped at `max_payload_bytes` (default 256 KiB). Over the cap, `values` is replaced by a truncation marker **inside the signed body** — so a receiver is told it is looking at a truncated state rather than left to infer it — and the row's `payload_truncated` records it. Raise the cap, or read the state back from the API using the `run_id` in the callback. `values` and `interrupts` are the only fields ever replaced, because they are the only unbounded ones. When both are present `values` goes first: an interrupted run's question is what makes the callback actionable at all, so it is the last thing dropped. skein **redacts the webhook URL's path in logs**, keeping only the scheme and host: for Slack, Discord and Teams the path _is_ the credential, and a failing delivery would otherwise write it into your log sink on every attempt. ## Accepting webhook URLs from untrusted callers `webhook` is a caller-supplied URL, so it is a server-side request to a target they chose — and retrying it turns a one-shot SSRF probe into a repeated one. Set [`skein.webhooks.allowed_hosts`](./langgraph-cli-compat.md#webhooks-skeinwebhooks) if you accept run creates from clients you do not control. It is off by default so that upgrading cannot start dropping your own callbacks; a refused host is recorded `dead` with the reason rather than silently skipped. Set `require_https: true` if callbacks leave your network. Plaintext is permitted by default (an internal receiver is legitimate), but retries mean the body crosses the wire up to `max_attempts` times rather than once — a dozen exposures rather than one. ## Configuration Every knob — `retries.max_attempts`, `retries.initial_delay_ms`, `secret`, `max_payload_bytes`, `retain_hours`, `allowed_hosts`, `require_https` — is documented once, with its default and its reasoning, in [`skein.webhooks`](./langgraph-cli-compat.md#webhooks-skeinwebhooks). Two environment variables sit outside that block: `SKEIN_WEBHOOK_SECRET` (which takes precedence over a `secret` in the config file) and `SKEIN_WEBHOOK_TIMEOUT_MS`. > [!WARNING] > skein does **not** expand `${VAR}` anywhere in `langgraph.json`. `"secret": "${SKEIN_WEBHOOK_SECRET}"` > signs every callback with that literal 23-character string. Use the environment variable. ## See also - [Runs](./runs.md) — the run-creation fields a callback reports on - [Errors & logging](./errors-and-logging.md) — what a failed run puts in the callback body - [Agent Protocol](./agent-protocol.md) — the full endpoint inventory - [Background jobs](./background-jobs.md) — the fire-and-forget shape webhooks were built for --- # Workflows and channels — connect LangGraph to external systems Connect an external event to your graph and deliver its result back to the right person or system. For example, a customer emails a refund request, your graph asks a reviewer on WhatsApp, and the reviewer's answer lets it finish the request and notify the customer. A **channel** verifies and translates provider events. Your **workflow** is the LangGraph graph that makes decisions, calls tools, and pauses for approval. Configure allowed destinations before the graph can choose where to send a result. ## Choose a connection {#when-to-use-it} | Your workflow | Start here | | ------------------------------------------------------------------ | ------------------------------------------------------------------ | | Answer an order question in the same WhatsApp chat | [Coupled replies](./recipes/coupled-channel.md) | | Receive email, request WhatsApp approval, then notify the customer | [Cross-provider workflow](./recipes/decoupled-channel-delivery.md) | | Accept a GitHub or internal-system event | Implement the channel source below and route to your graph | | Notify an existing HTTP service when any run finishes | [Run-completion webhooks](./webhooks.md) may be enough | Skein handles inbound deduplication, thread/run coordination, and durable outbound delivery. You supply provider signature verification, event parsing, business decisions, destination policy, and provider calls. There are no built-in Slack, WhatsApp, or email provider packages to configure. ## Quick start ```bash pnpm add @skein-js/channels ``` Write the provider adapter and bind its source route to a graph: **`src/twilio-channel.ts`** ```ts import { equalsConstantTime } from "@skein-js/agent-protocol"; import type { Channel } from "@skein-js/channels"; export const channel: Channel = { name: "twilio", verify(request) { const expected = sign(process.env.TWILIO_AUTH_TOKEN!, request.url.href, request.form()); if (!equalsConstantTime(expected, request.headers["x-twilio-signature"] ?? "")) return false; return { identity: `channel:twilio:${request.form()["From"]}` }; }, parseEvent(request) { const message = request.form(); if (!message["Body"]) return { kind: "ignore" }; return { kind: "event", event: { threadKey: message["From"]!, idempotencyKey: message["MessageSid"]!, replyTo: { to: message["From"] }, input: { messages: [{ role: "human", content: message["Body"] }] }, resumeWith: message["Body"], }, }; }, async deliver(outcome, target) { const { to } = target as { to: string }; if (outcome.reply !== undefined) await sendWhatsApp(to, String(outcome.reply)); }, }; ``` **`langgraph.json`** ```jsonc { "graphs": { "support": "./src/support.ts:graph" }, "skein": { "channels": { "twilio": { "path": "./src/twilio-channel.ts:channel", "assistant": "support", "public_url": "https://api.example.com", }, }, }, } ``` That serves `POST /channels/twilio`. Point your provider's webhook at it. A complete, runnable version — offline, no Twilio account — is [`examples/whatsapp-agent`](https://github.com/skein-js/skein-js/tree/main/examples/whatsapp-agent). ### Let a workflow route between providers The source that starts a workflow does not have to receive the result. The [`decoupled-delivery`](https://github.com/skein-js/skein-js/tree/main/examples/decoupled-delivery) example sends an email into one LangGraph workflow, routes approval requests to WhatsApp, collects parallel authenticated approvals with LangGraph `interrupt()`, then sends the result by email. Its runnable refund demo uses Skein's source, destination, auth, deduplication, thread/run and delivery paths with offline provider fakes. Skein exposes this as a small composition layer rather than separate provider hierarchies: ```ts import { composeRoutedChannel, declareChannelDestinationDelivery, type ChannelDestinationDelivery, } from "@skein-js/channels"; const destinations = new Map([ [ "whatsapp", async (delivery: ChannelDestinationDelivery) => { const message = whatsappSchema.parse(delivery); await sendWhatsApp(message, { idempotencyKey: delivery.runId }); }, ], ]); export const channel = composeRoutedChannel(emailSource, destinations); ``` ### How LangGraph fills the middle and selects the destination LangGraph passes a custom-stream writer to every node in `LangGraphRunnableConfig`. The Skein helper uses that existing writer to declare the result of the workflow; the node does not call WhatsApp or email itself: ```ts import { Annotation, END, START, StateGraph, type LangGraphRunnableConfig, } from "@langchain/langgraph"; import { declareChannelDestinationDelivery } from "@skein-js/channels"; const RelayState = Annotation.Root({ source: Annotation<"email" | "whatsapp">, sender: Annotation, whatsappNumber: Annotation, subject: Annotation, }); function routeMessage( state: typeof RelayState.State, config: LangGraphRunnableConfig, ): Partial { if (state.source === "email" && state.whatsappNumber) { declareChannelDestinationDelivery(config.writer, { destination: "whatsapp", target: { to: state.whatsappNumber }, payload: { body: `Important email from ${state.sender}: ${state.subject}` }, }); } else { declareChannelDestinationDelivery(config.writer, { destination: "email", target: { to: state.sender }, payload: { subject: "Assistant update", body: state.subject }, }); } // The destination declaration is an output instruction, not graph state. return {}; } export const graph = new StateGraph(RelayState) .addNode("route-message", routeMessage) .addEdge(START, "route-message") .addEdge("route-message", END) .compile(); ``` The workflow connection is deliberately small: 1. Skein's source converts email or WhatsApp into ordinary LangGraph input. 2. LangGraph performs extraction, classification, branching, `Send`, and `interrupt()` as usual. 3. The graph writes one explicit destination declaration through `config.writer`. 4. When the run settles, Skein resolves that declaration and invokes the allowlisted destination through its durable outbox. For approvals, keep using LangGraph's `interrupt()`; no Skein approval abstraction is needed. A node can declare the WhatsApp approval request before it interrupts, and declare the final email result after `interrupt()` returns on resume. The complete parallel HR/Manager/Finance implementation is [`examples/decoupled-delivery/src/relay-graph.ts`](https://github.com/skein-js/skein-js/blob/main/examples/decoupled-delivery/src/relay-graph.ts). The map allowlists destination adapters and remains application-owned, along with provider credentials and routing policy. It does not authorize recipients or operations: derive those from trusted workflow data and enforce tenant/recipient policy before sending, rather than treating an LLM-selected or user-supplied target as permission. Only `declareChannelDestinationDelivery` triggers routing; ordinary `replyWith` values and inferred interrupt/AI-message replies are not dispatched by a routed channel. Construct declarations through the helper, never its internal persisted marker. `target` and `payload` must be plain JSON values so the outbox can persist them. They remain `unknown` to the destination, which must validate its provider boundary—Zod is used above for exactly that. Unknown or malformed declarations and thrown callbacks fail the outbox attempt visibly and are retried. Delivery is at-least-once, so pass a stable key such as `runId` to providers that support idempotency. One run still owns one outbox retry unit; aggregate fan-out does not create independent delivery rows. This helper is for channel-ingested runs. A poller using the Agent Protocol run API still uses its own HTTP callback receiver. Existing `Channel` implementations with `deliver` continue to work unchanged. Configured route keys and explicit channel-name aliases must be unique; collisions now fail at boot instead of sharing thread identity or misrouting a callback. For copy-first workflow implementations, compare the practical [WhatsApp → order lookup → WhatsApp recipe](./recipes/coupled-channel.md) with the [customer refund email → Finance WhatsApp approval → customer email recipe](./recipes/decoupled-channel-delivery.md). ## The pipeline ```text POST /channels/twilio → verify() your signature check → a principal, or 401 → authorize() your Auth block, exactly as on every run-creating route → parseEvent() ignore → 204 · respond → that response · event → on → dedup the provider's event id, claimed over the raw bytes → thread a deterministic id, get-or-create → start | resume decided atomically against the thread's real status → 2xx after enqueueing, never after the run finishes ├→ onSignal() progress, best-effort └→ deliver() the answer, durably ``` Two properties of that shape are worth internalising before you write anything. **The acknowledgement is early, always.** Slack retries after three seconds and shows the user an error; Twilio times out comparably. Your `parseEvent` returns, the run is enqueued, and skein answers 2xx — it never waits for the graph. That is why progress needs `onSignal` and why the answer needs `deliver`: by the time either has something to say, the HTTP request is long gone. **`verify` runs before anything is parsed.** A signature covers the bytes as sent, so any middleware that parses first destroys the thing you need to check. skein hands you the body as text for exactly this reason. ## Implementing `verify` ```ts verify(request: InboundRequest): Promise | ChannelPrincipal | false; ``` Return a **principal**, or `false` for a 401. **Why a principal and not a boolean.** A provider's signature _is_ an authentication scheme; it just is not a bearer token. Producing an identity lets the message flow through the `Auth` block you already configured — `@auth.on.threads` handlers see it, ownership filters apply, multi-tenancy works — instead of needing a route that bypasses authorization to create runs. ```ts return { identity: `channel:twilio:${from}`, // derived from provider-verified data only permissions: ["threads:write"], // optional; becomes AuthContext.scopes metadata: { tenant }, // optional; merged into the user object }; ``` Build `identity` **only from data the signature covered**. A value the caller could have set freely is not an identity, and everything downstream — ownership filters, per-tenant scoping — inherits whatever trust you put in it here. **`verify` is required.** A provider with no signature scheme must still yield a principal some other way: a secret path segment, a shared-secret query parameter, basic auth. Weaker is a decision your deployment gets to make; absent is not, because the route creates runs. ### `InboundRequest` | Member | Notes | | --------- | --------------------------------------------------------------------------- | | `method` | The HTTP method | | `url` | A `URL`. The **public** one — see below | | `headers` | Lower-cased keys, whatever the transport | | `text()` | The body exactly as sent | | `form()` | Decoded `application/x-www-form-urlencoded` | | `json()` | Parsed JSON. Throws on a malformed body, so guard it if the provider varies | The body views are lazy and cached, so reading `form()` in both `verify` and `parseEvent` parses once. **`url` is the public URL, and that matters more than it looks.** Twilio signs the full request URL, so verification depends on knowing what the provider saw — and behind a proxy the server cannot know it. The only inputs are `Host` and `X-Forwarded-Proto`, both attacker-controlled. So skein takes the origin from `public_url` in your config and keeps the router's own path. Set it in production, or signature verification will fail in a way that looks like every request being forged. ### Comparing signatures ```ts import { equalsConstantTime } from "@skein-js/agent-protocol"; ``` Not `===`. It returns at the first differing byte, so an attacker who can send many requests and measure your response recovers a valid signature a byte at a time. The length check inside is not decoration either: Node's `timingSafeEqual` **throws** on unequal lengths, so the obvious hand-rolled version turns a forged short signature into a 500 instead of a 401. ## Implementing `parseEvent` ```ts parseEvent(request: InboundRequest): Promise | ChannelOutcome; ``` Three outcomes, and you will need all three. ### `{ kind: "event", event }` — do something ```ts return { kind: "event", event: { threadKey: message.From, // which conversation input: { messages: [...] }, // what a fresh turn runs with resumeWith: message.Body, // what an interrupt gets, if different idempotencyKey: message.MessageSid, // the provider's own event id replyTo: { to: message.From }, // opaque; handed back to deliver/onSignal onExisting: "resume", // default assistantId: "triage", // only if allowed_assistants lists it metadata: { locale: message.Locale }, // stamped on the thread }, }; ``` | Field | Required | Notes | | ---------------- | -------- | --------------------------------------------------------------------------- | | `threadKey` | ✅ | Stable external identity. Hashed into a thread id | | `input` | ✅ | Graph input for a fresh turn | | `threadId` | | Choose the thread id yourself; skips the derivation entirely | | `resumeWith` | | What `interrupt()` receives. Defaults to `input` — usually wrong, see below | | `idempotencyKey` | | No key, no dedup. See [Retries](#retries-and-idempotency) | | `replyTo` | | Opaque to skein. Omit it and no reply is delivered | | `onExisting` | | `resume` (default) · `enqueue` · `interrupt` · `reject` | | `assistantId` | | Rejected unless the deployment listed it in `allowed_assistants` | | `metadata` | | Merged onto the thread's metadata | ### `{ kind: "ignore" }` — acknowledge, do nothing Answers 204 and starts no run. **Use it liberally.** Slack redelivers your own bot's messages, and a channel that cannot cheaply say "not interesting" ends up answering itself in a loop. Delivery receipts, read receipts, reactions, media-only messages and bot echoes all belong here. ```ts if (payload.bot_id) return { kind: "ignore" }; ``` ### `{ kind: "respond", status, body }` — answer this request directly No run, your response. Required by real providers, not a convenience: ```ts // Slack's very first request is a challenge that must be echoed back. if (payload.type === "url_verification") { return { kind: "respond", status: 200, body: { challenge: payload.challenge } }; } // A slash command wants an immediate ephemeral acknowledgement. return { kind: "respond", status: 200, body: { response_type: "ephemeral", text: "On it…" } }; ``` ## Retries and idempotency Providers retry. Twilio retries on any non-2xx and on a timeout; Slack retries three times. Without dedup the customer gets two answers. Set `idempotencyKey` to **the provider's own event id** — `MessageSid`, Slack's `event_id`, GitHub's `X-GitHub-Delivery`. A retry carries the same one, and skein replays the first response instead of starting a second run. Three things worth knowing about how it behaves: - **The fingerprint is the raw inbound bytes**, not anything derived. A retry that correctly re-reads the thread and resumes instead of starting sends the same key with a _different_ run body — and fingerprinting that derived body would refuse the retry precisely because it did the right thing. - **Same key, different bytes → 422.** Two genuinely different events sharing an id is a provider bug or a forgery; replaying would tell the sender its message was handled while silently dropping it. - **No key means no protection**, deliberately. Without a provider-assigned id nothing identifies two deliveries as the same event, and deriving one from the body would collide when a customer legitimately sends "yes" twice. ## Conversations that wait for a human This is the case channels exist for, and the one every hand-written integration gets wrong. `interrupt()` parks a thread until someone answers. The run that parked it settles as `interrupted`, which is a **terminal** status — so the thread holds no inflight run, no `multitask_strategy` guards it, and a plain start succeeds and discards the pending question. Silently. A channel does the right thing by default: a message arriving on a paused thread **resumes** it. The decision is a [precondition on the create](./runs.md#dont-start-a-run-on-a-thread-thats-waiting-for-a-human), settled inside the driver, so two replicas cannot both resume. ### Set `resumeWith` — you almost certainly need it `input` is what a _fresh turn_ takes. For a message-shaped graph that is an envelope: `{ messages: [{ role: "human", content: "yes" }] }`. But `interrupt()` returns whatever the node asked for — usually a scalar the graph author chose: ```ts const answer = interrupt("Approve this refund?"); if (/^y/i.test(String(answer))) { … } ``` Hand that node the envelope and `String(...)` gives `"[object Object]"`, which does not match — so the graph reads a "yes" as a "no", and nothing errors. skein will not guess between the two shapes, so say which is which: ```ts input: { messages: [{ role: "human", content: body }] }, resumeWith: body, ``` ### Choosing `onExisting` | Value | What happens | When | | ----------- | -------------------------------------------------------- | --------------------------------------------------- | | `resume` | Answers a pending interrupt; otherwise queues a new turn | **Default.** Chat, email, anything conversational | | `enqueue` | Always a new turn, behind whatever is running | A feed of events that each deserve their own run | | `interrupt` | Cancels the run in flight and starts this one | "Actually, stop — do this instead" | | `reject` | 409 unless the thread is genuinely free | An event that is only valid on an idle conversation | Under `resume`, a message arriving while the agent is still thinking **queues** rather than being refused. The server's default `multitask_strategy` is `reject`, which for Twilio would render as a failed message for having typed twice; a conversation wants ordering. When the answer is not an answer — _"actually, never mind, what's my balance?"_ — skein still resumes and the **graph** decides whether to re-ask, pivot or abandon, because the graph holds the question. If you would rather have a blunter rule, use `enqueue`. ## Delivering the reply ```ts deliver?(outcome: RunOutcomeForChannel, target: ReplyTarget): Promise; ``` Optional — a GitHub channel that comments from inside the graph owes no callback. Omit it and no reply is delivered. **It runs inside skein's delivery outbox**, which is where all its guarantees come from. The delivery row is written in the run's finalize transaction, so a crash between "the run succeeded" and "the customer was told" cannot lose the answer. A **throw is a failed attempt**: retried with exponential backoff, recorded, and replayable from the [delivery routes](./webhooks.md#see-what-a-callback-did-and-replay-it). So let it throw — swallowing an error here is how an answer gets silently dropped. ```ts async deliver(outcome, target) { if (outcome.reply === undefined) return; // nothing to say const { to } = target as { to: string }; await sendWhatsApp(to, String(outcome.reply)); // throws → retried } ``` ### What `outcome.reply` is Resolved by skein, in order: 1. **What the graph declared**, via `replyWith` on the custom stream. 2. **The last AI message** in `values.messages` — LangGraph's `MessagesAnnotation` convention, so an ordinary chat agent works with no graph changes at all. 3. **The interrupt's question**, when the run is `interrupted`. 4. Otherwise `undefined`. Resolved centrally rather than per channel on purpose: a channel that guessed at `state.answer` versus `state.draft` would stop being reusable across graphs, which is the premise the whole plugin surface rests on. For a graph whose state is not message-shaped, declare it: ```ts import { replyWith } from "@skein-js/agent-protocol"; // inside a node, with LangGraph's StreamWriter in scope writer(replyWith("Your order ships Tuesday.")); ``` **A failed run says nothing.** `error`, `timeout` and `cancelled` resolve to no reply, because whether an end user hears "something went wrong" is a product decision and leaking internals to a phone number is the wrong default. Send your own message from `deliver` if you want one. ## Progress signals ```ts signals: { kinds: ["progress"], keepaliveMs: 10_000 }, async onSignal(signal, target) { … } ``` Because the acknowledgement is early, progress cannot ride the response. `onSignal` is how a typing indicator gets sent while the graph works. **Its guarantees are deliberately the opposite of `deliver`'s**: best-effort, at most once, never retried, never blocking the run, dropped on any error. Retrying a "typing" indicator four minutes late is nonsense, and a slow provider must never be why the acknowledgement times out. That is why they are two methods rather than one with a flag. - `progress` — fires immediately, then once per frame the run produces. - `keepalive` — fires on `keepaliveMs`, for providers whose indicator expires (most do, in seconds). Declaring a subscription is what lets skein pick the cheapest stream that satisfies it. A channel that asks for nothing costs exactly what an API run costs; one that asks for `progress` gets node-level updates and **never** pays for token streaming it would only throw away. ## Addressing a conversation from outside Thread ids are derived, and the derivation is exported — which is what makes hashing acceptable rather than merely private: ```ts import { threadIdForChannelKey } from "@skein-js/channels"; const threadId = threadIdForChannelKey("twilio", "whatsapp:+254712345678"); await client.threads.get(threadId); ``` The phone number never reaches a primary key, an index or a backup. The raw key **is** stamped into thread metadata, so a search answers the same question from the other side — which is the shape a GDPR erasure request actually arrives in: ```ts await client.threads.search({ metadata: { skein_thread_key: "whatsapp:+254712345678" } }); ``` ### Ending a conversation and starting a new one Two different things people mean by this, and they need different answers. **"Forget everything and start over."** Delete the thread. The next message from that number derives the same id, finds nothing there, and begins fresh: ```ts await client.threads.delete(threadIdForChannelKey("twilio", "whatsapp:+254712345678")); ``` **"Start a new conversation, but keep the old one."** Then the thread key is not just the phone number — it is the phone number _and_ which conversation. `threadKey` is whatever your channel says, so put the session in it: ```ts // A ticket, a billing period, a counter you keep — whatever "a conversation" means to you. threadKey: `${message.From}:${await currentSessionFor(message.From)}`; ``` Old conversations stay readable, each under its own thread, and a search on `skein_thread_key` still finds them because the raw key is what gets stamped. A channel that already has its own conversation ids returns `threadId` on the event instead, and skein does no transformation at all. ## Configuration | Key | Meaning | | -------------------- | --------------------------------------------------------------------- | | `path` | `path:export`, or a package name | | `assistant` | **Required.** Which graph these events run | | `allowed_assistants` | Graphs the channel may route to itself. Omitted means it cannot route | | `public_url` | Your externally reachable origin, when signatures cover the URL | **`assistant` is required and not defaultable.** The binding is deployment knowledge: a community Twilio adapter has no business knowing you named your graph `support`. It takes a graph name or an assistant UUID, resolving exactly as it does for [crons](./crons.md). **`allowed_assistants` is opt-in and bounded** for a sharper reason. A channel is an npm package you installed; without a bound, an `assistantId` derived from untrusted input could reach any graph you serve. Omit the key and the channel cannot route at all. Everything is validated **at boot**: a missing `assistant`, one naming a graph that does not exist, an `allowed_assistants` entry outside your graphs, a `path` whose export is not a channel. Discovering any of those when the first customer texts is the failure this avoids. ## Testing a channel The quickest end-to-end check is the console's **Channels** tab. It shows the effective routes and assistant allowlists that passed boot validation. Copy an inbound path, send a fixture or provider event, then open the resulting thread and run: the run's **Deliveries** panel shows attempts, retry timing, errors, and a confirmed replay action. Sensitive webhook paths and opaque reply targets are redacted in the UI. The same inventory is available as authenticated `GET /channels`; it is mounted only when at least one channel is configured and exposes no module paths, public URLs, credentials or raw config. A channel is a plain object, so `verify` and `parseEvent` are unit-testable with no server: ```ts const request = buildInboundRequest({ method: "POST", url: "https://api.example.com/channels/twilio", headers: { "x-twilio-signature": signature }, text: "From=whatsapp%3A%2B254&Body=hi&MessageSid=SM-1", }); expect(channel.verify(request)).toEqual({ identity: "channel:twilio:whatsapp:+254" }); ``` For the whole path, boot a runtime from a `langgraph.json` and dispatch into the handler — no HTTP server, no ports: ```ts const resolved = await resolveProtocolRuntime({ config: "./langgraph.json" }); const response = await resolved.runtime.handlers.handleInboundEvent(request); expect(response.status).toBe(202); ``` Worth covering, because each has shipped broken in real integrations: a forged signature is refused; a retried delivery produces exactly one run; an interrupt is resumed rather than trampled; a delivery receipt starts no run. [`examples/whatsapp-agent`](https://github.com/skein-js/skein-js/tree/main/examples/whatsapp-agent) does all four offline. ## Things that will bite you - **Mount order, if you also hand-roll routes.** `skeinRouter` installs a JSON body parser for everything routed into it, so a raw-body route registered _after_ it sees an empty body and fails to verify — a 401 that looks like a forged request. Channel routes are handled for you; your own are not. - **`public_url` behind a proxy.** Without it, URL-signing providers fail every request. - **Forgetting `resumeWith`.** The graph reads a "yes" as a "no" and nothing errors. - **Swallowing errors in `deliver`.** A throw is what buys you the retry. - **Returning `false` from `verify` for a _parse_ failure.** That reports a forged request when the truth is a malformed one; prefer `{ kind: "respond", status: 400 }` from `parseEvent`. - **Sending the reply from `parseEvent`.** It will be sent again on every retry. The outbox exists so that it is not your problem. ## What is not here yet - **First-party channels.** There are none: you write the file. Shipping one would make skein responsible for the correctness of someone else's signature scheme, so a flaw becomes a CVE here rather than in your code — a real cost that has to be earned. - **A channel conformance suite**, so a community channel can prove it rejects forged signatures, stale timestamps and duplicate ids without hand review. - **Chunked streaming of the answer** — send it in pieces, crash halfway, and the outbox replays a message the user already partly received. Needs provider-side editing (Slack can, WhatsApp cannot). - **Attachments.** Carry the provider's media URL in `metadata` and fetch it from the graph. - **Polled sources** (IMAP, queue consumers). The pipeline is entered by an inbound HTTP request, and there is nothing to enter it with when you poll — that is closer to [crons](./crons.md). ## See also - [webhooks.md](./webhooks.md) — the durable delivery a channel's reply rides on - [human-in-the-loop.md](./human-in-the-loop.md) — `interrupt()`, and why a reply hours later must resume rather than restart - [runs.md](./runs.md#dont-start-a-run-on-a-thread-thats-waiting-for-a-human) — `if_thread_status`, the precondition underneath `onExisting` - [agent-protocol.md](./agent-protocol.md#authentication--authorization) — the `Auth` block a verified principal flows through --- # Streaming (SSE) Streaming is what makes an agent UI feel alive: tokens appear as the model writes them, model **thinking** streams into a collapsible panel, tool calls and their **structured results** show up as they happen, and a paused run's **interrupt** surfaces for approval. skein-js delivers all of it over one transport — **Server-Sent Events (SSE)** — so the standard clients ([`useStream`](./react-sdk.md), the vanilla SDK, Agent Chat UI) render a rich conversation against a skein-js server with only a URL change. The flagship [`chat-app`](https://github.com/skein-js/skein-js/tree/main/examples/chat-app) example wires the full experience end to end; [`react-usestream`](https://github.com/skein-js/skein-js/tree/main/examples/react-usestream) is a minimal frontend example. Under the hood, skein-js maps LangGraph.js **stream modes** onto Agent Protocol SSE. This one transport powers the `/runs/stream` endpoint, joining an in-flight run (`/runs/{id}/stream`), and thread-scoped streaming (`/threads/{id}/stream`). Reference: LangGraph streaming — ## LangGraph.js stream modes A `CompiledStateGraph.stream(input, { streamMode })` can emit any combination of: | Mode | Emits | | ---------------- | ------------------------------------------------- | | `values` | Full state after each step | | `updates` | State deltas per node (**default**) | | `messages` | Complete messages | | `messages-tuple` | Message chunk + metadata tuples (token streaming) | | `custom` | User-emitted custom events | | `events` | Fine-grained execution events | | `debug` | Detailed debug info | Multiple modes can be requested at once; skein-js preserves that. When `events` is among the requested modes the run engine drives the graph via LangGraph's `streamEvents` (v2) and emits each event as an `events` frame (co-requested modes like `values` still stream alongside); otherwise it uses `graph.stream`. ## Mapping to Agent Protocol SSE Each LangGraph stream item becomes an SSE frame: ``` event: # e.g. messages, updates, values, custom id: # per-run sequence for replay/reconnect data: ``` - **Event id sequencing** — each run assigns monotonically increasing ids so a reconnecting client can resume via `Last-Event-ID` while the requested frames are retained. See [joining and replay](#joining-and-cross-instance-fan-out) for backend limits. - **Terminal frames** — a final `event: end` (or `error`) closes the stream with the run's status. An `error` frame's payload is a `RunError` — `{ error, message, name, cause?, errors? }`, plus `stack` when the server sets `exposeErrorStacks`. See [errors-and-logging.md](./errors-and-logging.md). - **Idle heartbeats** — after 15 seconds without a frame, the server sends an ignorable SSE comment (`: heartbeat`). Configure your proxy and load-balancer timeouts for the expected run duration; heartbeats do not override a platform's total request timeout. Custom adapters can set `toSseEvents(..., { heartbeatMs })`; `0` disables heartbeats. ## Subgraphs and subagents A graph that runs another graph — a compiled subgraph added as a node, or a graph invoked from a tool, which is how [Deep Agents](./recipes/deep-agents.md) runs a subagent — streams only its own frames by default. Set `stream_subgraphs: true` on the run (`streamSubgraphs: true` in the JS SDK) to stream the inner graphs' frames too. Each is named after the mode and the namespace it came from: ``` event: updates # the run's own graph event: updates|tools:6d1f… # a subagent run from a tool call event: messages|tools:6d1f…|model_request:9a2c… # that subagent's model tokens ``` These are LangGraph Platform's names; `@langchain/langgraph-sdk` splits on `|` to tell supervisor and subagent streams apart. Joining a run (`Last-Event-ID` included) replays them unchanged, and a background or cron run keeps the flag. Without the flag you are not blind to subgraphs: `messages` mode already carries their model tokens (each tuple's metadata holds `langgraph_checkpoint_ns`), and `events` mode carries every nested event. What the flag adds is their `updates`, `values` and `custom` frames, and names a client can route on. With it on, the run's own model tokens are namespaced too (`messages|model_request:…`), as upstream — the root graph's frames are the ones with no `|`. Inside Skein, `RunFrame.event` remains the bare mode and `RunFrame.namespace` carries the optional subgraph path. Root frames omit the namespace. Event buses preserve both fields; SSE serialization combines them into the upstream event name shown above. Existing switches over `RunFrame.event` continue to work. Declared replies keep their existing last-write-wins behavior, including nested custom writes that LangGraph propagates. The client's `stream_subgraphs` setting does not change which reply is delivered. ## Slow clients and backpressure A stream is only as fast as the client reading it, and skein paces itself accordingly. Every adapter's write loop honors the response stream's backpressure signal: when the socket's buffer is full it waits for `drain` before pulling the next frame, rather than queueing whatever the graph produces. That matters because the alternative is unbounded. A client on a bad connection — a phone on mobile data, a buffering reverse proxy — that reads more slowly than the graph writes would otherwise be served entirely out of the server's memory, one full copy of the stream per connection. Measured in [`packages/bench`](https://github.com/skein-js/skein-js/tree/main/packages/bench) on the `slow-client` scenario (clients reading at ~25 fps against a 500 fps graph, ~2 MB per stream): | Concurrent slow streams | Unflushed server-side buffer | Per streaming connection | | ----------------------- | ---------------------------- | ------------------------ | | 50, before | 62.9 MB | ~1.26 MB | | 100, before | 125.5 MB | ~1.26 MB | | 50, after | 3.3 MB | ~67 KB | | 100, after | 6.5 MB | ~65 KB | Before, the per-connection cost was the whole stream, so total memory grew with both the number of clients **and** the length of each run. After, it is a constant close to the socket's own 64 KB high-water mark: still linear in connection count, as it must be, but no longer proportional to how much the graph produces. Two consequences worth knowing: - **The graph does not slow down.** The run engine publishes into the event bus and the write loop reads from it, so pacing the reader changes how fast frames leave the bus, not how fast they enter it. A slow client cannot stall the run, or any other client's stream. - **Backpressure itself never discards.** It delays delivery; a slow client receives every frame, just later. What it does is move the queue from the socket into the event bus — and the bus is bounded, so a client slow enough to exceed that bound has its stream **ended** rather than being left hanging. What happens next differs by bus. On **Redis** the frames are still in the durable stream, so reconnecting with `Last-Event-ID` replays them: nothing is lost. On the **in-memory bus** the stream ended precisely _because_ the frames were evicted to stay under `SKEIN_MEMORY_BUS_MAX_FRAMES_PER_RUN`, and that buffer is also the replay log — so a reconnect resumes from what survives, with a gap. See [performance.md](./performance.md#streaming-backpressure-drops-and-recovery) for the bounds, and why that asymmetry is a reason to run Redis for unreliable clients. The Next.js App Router adapter gets this for free: it maps frames onto a `ReadableStream` whose `pull` is demand-driven by the platform. ## Joining and cross-instance fan-out - `GET /runs/{run_id}/stream` lets a late client join a run already in progress. - When a run executes on a **different** worker than the one holding the client connection, [`@skein-js/redis`](./runs-and-redis.md) pub/sub fans the frames across instances so the join still works. In single-process `skein dev`, an in-memory event bus is used instead. - The [SQLite local assembly](./local-deployment.md) also uses a process-local event bus. Records and checkpoints survive reopening, but live token frames do not. Use persisted state/history to restore the UI after restart; SQLite does not provide cross-instance fan-out. ## Why SSE is enough (no WebSocket in v1) The entire LangChain client surface — the vanilla SDK, the [`useStream`](./react-sdk.md) React hook, and Agent Chat UI — consumes **SSE**. The protocol's optional WebSocket upgrade buys bidirectional framing we don't need for v1, so it is deferred (see [roadmap.md](./roadmap.md)). **Deferring WebSocket does not affect the React SDK.** ## Thread event subscriptions The thread protocol separates execution from observation. Submit `{ id, method: "run.start", params: { assistant_id, input } }` to `/threads/{id}/commands`, then observe `POST /threads/{id}/stream/events` with `{ channels: ["values", "messages", "lifecycle", "input"] }`. Reopening that subscription executes nothing. Commands return JSON envelopes; legacy command bodies retain their SSE response. The LangGraph binding uses the public `streamEvents(..., { version: "v3" })` path for protocol runs, including graph-defined transformers and custom channels. Subscriptions stay open across turns and receive idle heartbeat comments. Channel, namespace-prefix and depth filters are reader-specific; static namespace segments also match their runtime `:task-id` suffixes. The bus assigns sequence numbers before filtering, so reconnect cursors work across runs. Request `since` to replay after a cursor; an invalid retained-history boundary requires state restoration and a fresh subscription. With no cursor, the current state is the initial baseline. `MemoryThreadEventBus` bounds each payload history to 10,000 entries, retaining at most 50 payload histories. `RedisThreadEventBus` uses capped streams (approximately 10,000 entries) with a one-hour payload expiry. Sequence and deletion-generation records survive payload expiry. Deleting a thread clears payloads and closes its readers; execution captures its generation so a late publisher cannot repopulate history after deletion or ID reuse. Release the Redis bus with `dispose()` when assembling infrastructure by hand; runtime assemblers own that cleanup automatically. Legacy `GET /threads/{id}/stream` joins one run by default. Pass explicit `stream_mode` to follow events across turns. The official categories are `run_modes` (run frames plus completion metadata), `lifecycle` (start/end `metadata` events), and `state_update` (`{ thread_id, state }` checkpoint snapshots after execution or direct state updates). Recorded snapshots are replayed without substituting a later thread state. Individual frame modes and optional `thread_state` thread-row events remain available. The stream starts with live events unless `Last-Event-ID` resumes after an earlier one. This default differs from Agent Server; see the [compatibility matrix](./langgraph-compatibility.md) for the verification scope. A subscription without `since` replays retained events so late selectors can recover custom channels and media, then receives a current state baseline and live events. An explicit `since` resumes after that publication sequence. Sequences span runs; payload expiry does not reset them. Expired or future explicit cursors return `thread_cursor_expired` and require state restoration. Deleting and recreating a thread ID does not make old events part of the new conversation. SQLite's event bus is process-local; its state and run records remain durable across restart. --- # Errors & logging What happens when a graph throws — where the failure shows up, what a client sees, and what lands in your logs. Errors and logging live in one doc because a failure goes to both, on deliberately different terms: **the log gets everything; the wire gets what is safe to hand a caller.** ## Diagnose a failed request If an order assistant says nothing after calling your order service, inspect the existing run before sending the request again. Check its status and error in the [console](./console.md#runs), then find the same run in the server logs. A graph failure, a disconnected browser, and a failed delivery need different fixes. Confirm what happened before retrying an action that could change an order. ## When a local process disappears Reopening a SQLite deployment records abandoned executing/claimed attempts and pending inline requests as `error` with a `ProcessLostError` payload. Unclaimed queued jobs keep their due times; terminal outcomes and real approval interruptions are preserved. This startup bookkeeping does not replay input, resume checkpoints, or synthesize completion callbacks and telemetry. Applications decide what to do next. See [local crash behavior](./local-deployment.md#what-survives-a-process-ending). ## When a graph throws A node that throws does not crash the server. The run engine catches it, and the failure appears on these surfaces: | Surface | What it gets | | --------------------- | ---------------------------------------------------------------------------------------------- | | **The server log** | An `error`-level line with the full stack and `cause` chain. Always — see [Logging](#logging). | | **The SSE stream** | A terminal `event: error` frame carrying a [`RunError`](#the-runerror-payload). | | **The run row** | `status: "error"` plus `error` — the same `RunError`, durable. | | **The thread row** | `status: "error"` plus `error` (the message). A _mirror of the latest turn_. | | **`POST /runs/wait`** | `200` with `{ "__error__": { … } }` in place of the graph's values. | | **The webhook** | `error` as a plain message string, alongside the settled run. | The run row is the durable record. The thread's `status` and `error` are cleared as soon as a later run on that thread succeeds — they describe the thread _now_, not its history. If you need to know why a particular run failed, read the run. ```bash curl localhost:2024/threads/$THREAD/runs/$RUN ``` ```json { "run_id": "63086ee4-…", "status": "error", "error": { "error": "Error", "name": "Error", "message": "model call failed", "cause": { "error": "Error", "name": "Error", "message": "MISSING_KEY is undefined" } } } ``` ### The `RunError` payload One shape (`RunError`, from `@skein-js/core`) is used by the SSE frame, the persisted `Run.error`, and the `__error__` wait body — so the stream and a later `GET` can never disagree. | Field | | | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `error` | The error's constructor name. LangGraph Platform's name for this field. | | `message` | The error's message. The one field every client renders. | | `name` | Always identical to `error` — see [compatibility](#langgraph-platform-compatibility). | | `cause` | The `Error.cause` chain, one `RunError` per link, up to five deep. | | `errors` | An `AggregateError`'s members (up to ten) — LangGraph throws one when several nodes fail in the same superstep, and its envelope message alone tells you nothing. | | `stack` | **Only when the server opts in.** See below. | `toRunError(thrown, { includeStack })` builds one from anything thrown. It is cycle-safe, depth capped, and never throws — it runs on the failure path, where a second failure would hide the first. ### `exposeErrorStacks` A stack names server file paths, dependency versions, and sometimes argument values. So it goes to your logs unconditionally, and to the **client** only when you ask: ```ts const deps: ProtocolDeps = { /* … */ exposeErrorStacks: true }; ``` - **`skein dev` turns it on**, unconditionally — not behind `--verbose`. Needing a flag to find out why your graph crashed is the problem this exists to remove. - **`skein start`, and every embedded server, leave it off.** LangGraph Platform never puts a stack on the wire either, so with it off the frame is a strict superset of the platform's. Turning it off never costs you information as an operator: the server log has the full stack either way. It only affects what the _client_ can see. ## Logging skein logs through a four-method interface — no framework, no transport, no color: ```ts interface Logger { debug(message: string, meta?: unknown): void; info(message: string, meta?: unknown): void; warn(message: string, meta?: unknown): void; error(message: string, meta?: unknown): void; } ``` Every adapter takes a `logger` option, and it reaches **the run engine** — not just the transport. It is the one knob that decides whether a crashed graph is visible at all. ### What each adapter does by default | | Default | Why | | ------------------- | -------------------- | ------------------------------------------------------------------ | | `@skein-js/nestjs` | Nest's own `Logger` | A facade over `app.useLogger()` / `NestFactory.create({ logger })` | | `@skein-js/fastify` | `fastify.log` (pino) | The host's own instance, honoring its config | | `@skein-js/express` | **nothing** | Express owns no logger — pass `createConsoleLogger()` | | `@skein-js/nextjs` | **nothing** | Same | The split is deliberate. Where the framework owns a logger the host has already configured, skein _borrows_ that decision — including the decision to be silent (`NestFactory.create({ logger: false })`, `Fastify({ logger: false })`). Where it doesn't, defaulting on would mean a library writing into someone's stdout uninvited, so skein stays quiet until asked: > **The two standalone servers are the exception, and differ from each other.** `createNestServer` > owns its bootstrap and keeps Nest's banner off, which silences the global facade — so it defaults to > its own `ConsoleLogger` and `app.useLogger()` does _not_ redirect it; pass `logger` instead. > `createFastifyServer` leaves pino off exactly as Fastify does, so it stays silent until you enable > it with `{ fastify: { logger: true } }` or pass a `logger`. ```ts import { createConsoleLogger, createExpressServer } from "@skein-js/express"; createExpressServer({ config: "./langgraph.json", logger: createConsoleLogger() }); ``` `createConsoleLogger({ level, prefix })` is plain uncolored output — `level: "warn"` suppresses the per-run summaries while still reporting every failure. (The colored, code-framed failure block is `skein dev`'s; it knows your source root and can safely read from it.) ### Precedence | Situation | Result | | ---------------------------- | ----------------------------------------------------- | | `deps.logger` is set | that logger, always — the adapter never overwrites it | | otherwise, `logger` a Logger | that logger | | otherwise, `logger: false` | nothing — no default installed | | otherwise | the adapter's default from the table above | An injected `deps.logger` outranking the adapter option is deliberate, and it is what makes the rest safe: the adapter only ever _fills_ `deps.logger`, never replaces it. That in turn lets it hand the engine your actual deps object rather than a copy — which is what keeps post-mount configuration (`deps.exposeErrorStacks = true` after mounting) working, since the invoke surface re-reads its deps on every request. ### Bridging to your own logger Each bridge ships with its adapter, so meta lands in the shape that logger actually wants: ```ts import { createNestLogger } from "@skein-js/nestjs"; // Point skein at a specific LoggerService rather than the globally configured one. SkeinModule.forRoot({ config, logger: createNestLogger({ logger: myPinoNestLogger }) }); import { createFastifyLogger } from "@skein-js/fastify"; // pino is object-first, so a failed run's ids become queryable fields and `err` gets pino's // error serializer — not a pre-flattened string. app.register(skeinPlugin, { config, logger: createFastifyLogger(app.log.child({ svc: "graphs" })), }); ``` Anything else is four lines against the interface above. ### What is always logged A **failed run** is always reported at `error` level, whatever `logRunActivity` says. The `meta` is a `RunFailureReport` — plain data plus the original `Error`, so a console logger can render a stack and a code frame while a JSON logger can serialize the fields. Recognize it with `isRunFailureReport`. Also always logged: background-run lifecycle summaries (at `error` level, with a `run_error` field, when the run failed), webhook delivery failures, rollback failures, and queue-shutdown problems. That is the whole steady-state volume — failures, plus one line per background run. The noisy per-run chatter is behind `logRunActivity` and stays off. One cost worth knowing: naming the node that threw takes a checkpointer read, so the engine skips building the report when nothing is listening. With a logger configured — which, under NestJS and Fastify, is now the default — a **failed** run pays that read. Successful runs are unaffected. ### Sending failures somewhere else The log is one of **three** surfaces a failure reaches. The third is a telemetry sink — LangSmith, PostHog, OpenTelemetry, or your own — which receives a `run.finished` event carrying the `RunError`, the node that threw, and the original `Error`: ```ts const sink: TelemetrySink = { name: "sentry", onRunEvent: (event) => { if (event.type === "run.finished" && event.cause) Sentry.captureException(event.cause); }, }; ``` A sink is **server-side, like the log** — so it always gets the full `Error`, stack and `cause` chain included, **regardless of `exposeErrorStacks`**. That flag governs only what reaches a client. See [observability.md](./observability.md). ### What `--verbose` adds `skein dev --verbose` sets `ProtocolDeps.logRunActivity`, which adds per-run _chatter_: run start/finish with duration and frame count, each tool call and tool result, and interrupt prompts. It costs nothing when off — the engine skips the stream inspection entirely. It does **not** gate failures. A graph that throws is logged either way. ### The failure block The CLI — `skein dev` and `skein start` alike — renders a graph failure as a fenced block naming the run, the assistant, the thread, the node that threw, and a code frame pointing at the line: ``` error: Graph run failed: model call failed ──────────────────────── GRAPH RUN FAILED ──────────────────────── run 63086ee4-1729-4744-8c8e-2cbdd48aff80 assistant boom thread 76258f06-a401-44c0-a6d7-7e189a3f8b36 node call_model 3 | function callModel(): never { 4 | const apiKey = process.env["MISSING_KEY"]; > 5 | throw new Error("model call failed", { | ^ 6 | cause: new Error(`MISSING_KEY is ${String(apiKey)}`), Error: model call failed at RunnableCallable.callModel (src/boom-graph.ts:5:9) caused by: Error: MISSING_KEY is undefined ────────────────────────────────────────────────────────────────── ``` The rules and blank lines are plain text, not color — piped logs, CI output, and `NO_COLOR` all disable color, and a crash needs to stand out precisely there. Two parts are best-effort and are simply omitted when unavailable, never faked: - **The `node` row.** LangGraph rethrows a node's error verbatim, so the error object never names the node. skein reads it from the post-failure state snapshot instead, where the runner records an `__error__` write against the failing task. A failure before the graph started yields nothing, and a failure inside a subgraph names the _parent_ node. - **The code frame.** It needs the stack to point at readable source inside the project. Under `skein dev` it does — vite's module runner source-maps stacks back to your `.ts` files, and the frame is bounded to the workspace root vite serves from. Under `skein start` the bundled artifact usually ships without original sources, so the frame is normally absent and the stack still prints. The frame is deliberately conservative about where it reads from, because an error _message_ is frequently attacker-influenced — a raw model response, a fetched document, or simply ``throw new Error(`bad mode: ${input.mode}`)`` over a client-supplied input. Since `Error.stack` is `${name}: ${message}` followed by the frames, a newline in a message produces a line that parses exactly like a stack frame and sits ahead of every genuine one. skein parses frames only from the region after the header, and reads only files resolving inside the project root — so a crafted message cannot steer it into reading an unrelated file. ### The load-failure block A graph can fail the other way too — never loading at all, because importing its module threw. That is the failure a fresh project hits first, and almost always for one reason: the graph builds a model client at module scope and the API key is not set. `skein dev` and `skein start` import every declared graph once at startup, **after** the banner, and report each failure as its own block: ``` error: graph "agent" failed to load ────────────────────── GRAPH FAILED TO LOAD ────────────────────── graph agent source src/agent-graph.ts:68 66 | const apiKey = process.env.GOOGLE_API_KEY; 67 | if (!apiKey) { > 68 | throw new Error( | ^ GOOGLE_API_KEY is not set — the "agent" graph needs it. Uncomment it in .env (get a key at https://aistudio.google.com/apikey) and save; the dev server picks it up on reload. The "echo" graph needs no key. SkeinConfigError: Failed to import graph module ".../src/agent-graph.ts". caused by: Error: GOOGLE_API_KEY is not set — the "agent" graph needs it. … ────────────────────────────────────────────────────────────────── ``` The headline is the **root** of the `cause` chain, not the wrapper: `SkeinConfigError` says _where_ the failure happened, and only its cause says _what_ went wrong. The code frame follows the root's stack for the same reason — the wrapper's stack is nothing but skein frames. The full chain still prints underneath. One graph that cannot load never takes the server down; the rest keep serving. Under `skein dev` the project's `.env` is watched, so filling in the missing key and saving reloads the graphs — a key that is _newly added_ takes effect, while changing one that is already set still needs a restart (the ambient environment outranks the file, by design). ## Errors at the edges Two typed errors, both carrying a `cause` that the CLI prints: - **`SkeinHttpError`** (`@skein-js/core`) — carries the HTTP status a handler wants, plus optional `code` and `details`. Adapters map it to `{ status, message, code?, details? }`; anything else becomes an opaque `500` with the real error sent to the logger. See [building-an-adapter.md](./building-an-adapter.md). - **`SkeinConfigError`** (`@skein-js/config`) — a bad `langgraph.json`, an unknown graph, or a graph module that failed to import. Its `cause` is the actual import failure and its `details` are the Zod issues, so read past the top-level message. A graph that fails to _load_ during a **run** surfaces through the same failure path as one that throws, with the config error's `cause` chain intact — unchanged. On the surfaces that answer over HTTP directly — thread state and history, assistant introspection, the single-graph invoke handler — it is mapped to a `SkeinHttpError` carrying `code: "graph_load_failed"`, so it stops being an unhandled fault. How much of _why_ reaches the caller is governed by [`exposeErrorStacks`](#exposeerrorstacks), the same switch that governs stacks: | | Body | | --------------------------------- | ------------------------------------------------------------------------------------------------------------------- | | **`skein dev`** (on) | `{"status":500,"message":"Graph \"agent\" failed to load: GOOGLE_API_KEY is not set …","code":"graph_load_failed"}` | | **`skein start`**, embedded (off) | `{"status":500,"message":"Graph \"agent\" failed to load.","code":"graph_load_failed"}` | The reason is withheld in production because a load failure's message is not skein's: `Cannot find module '/srv/app/dist/tools.js'` and `connect ECONNREFUSED 10.0.3.14:5432` are both ordinary ones, and they name server paths and internal hosts. The stack never travels either way. **The operator loses nothing.** Adapters log every `5xx` — including this one, typed or not — with the full `cause` chain, so the reason is always in your logs even when it is not on the wire. `rootCause(thrown)` / `rootCauseMessage(thrown)` (`@skein-js/core`) are what collapse a chain to one sentence, if you want the same in your own handler. Cycle-safe and depth-capped, like `toRunError` — which you want instead when a client should see _every_ layer. ## LangGraph Platform compatibility Verified against `@langchain/langgraph-api` and `@langchain/langgraph-sdk`: - The platform's error frame is `{ error, message }`, and the SDK's `ErrorStreamEvent` declares exactly that. Its `StreamError` reads `data.name ?? data.error`. skein emits **both** `error` and `name` with the same value, so the SDK, the platform's own clients, and older skein clients (which saw only `name`) all agree. Extra keys are ignored by the SDK. - `Run.error` is a skein extension — the SDK's `Run` records only _that_ a run failed. It is optional, so a client that ignores it is unaffected. This and the `"cancelled"` run status are the only two places skein deliberately steps outside the SDK's wire contract. - `Thread.error` **is** an SDK field that the JS platform leaves empty; skein populates it (and keeps the older `metadata.error` alongside it for existing readers). - `ThreadTask.error` is a JSON string, because `useStream` reads thread history by `JSON.parse`ing this field and rebuilding a `StreamError` from it. - `POST /runs/wait` answers a failed run with `{ "__error__": … }`, the platform's key. skein does not reproduce the platform's double-encoding of that payload. - The webhook's `error` is a plain message string, matching the platform exactly. --- # Frontend SDKs / `useStream` compatibility Connect your frontend to a running skein-js server with the standard LangChain SDKs. Use `useStream` to submit turns, read streamed messages, show errors, and respond to human-approval requests. You build the interface; the hook supplies the conversation state and actions. ## Connect a React UI {#usestream-against-skein-js} Install the dedicated React package in your frontend: ```bash npm install @langchain/react @langchain/core ``` Start your skein-js server, then use its API URL and a registered Assistant ID. This example assumes the scaffolded `echo` graph and sends a known input: ```tsx "use client"; import { useStream } from "@langchain/react"; export function Chat() { const thread = useStream({ apiUrl: "http://localhost:2024", assistantId: "echo", // replace with your graph's Assistant ID }); return (
{thread.messages.map((message) => (

{typeof message.content === "string" ? message.content : ""}

))} {thread.error != null &&

{String(thread.error)}

} {thread.isLoading && }
); } ``` Choose **Send hello** and expect `echo: hello` in the conversation. A browser on a different origin needs the [server's CORS policy](./recipes/serving.md#cors-for-a-browser-client) to allow your frontend. For custom authentication, follow the [authentication recipe](./recipes/authentication.md#browser-cookies-and-cors). The [`nextjs-app`](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-app) example serves both UI and protocol from one origin. For a complete application with tools, reasoning output, and approval cards, see [`chat-app`](https://github.com/skein-js/skein-js/tree/main/examples/chat-app). ## Handle pauses and failures When your graph calls `interrupt()`, render its request from `thread.interrupt` and respond with the value your graph expects. For a graph expecting `{ approved: boolean }`, an approval button can call: ```tsx ``` Use the matching rejection value on another test thread and verify that the action is not performed. See [human-in-the-loop](./human-in-the-loop.md) for the graph and SDK examples. Show `thread.error` to the user and inspect the run before retrying an ambiguous request. Call `thread.stop()` when the user wants to cancel. Unmounting the UI or reopening a conversation does not cancel or start a run by itself. Test restoration and multiple turns with your application's thread-selection controls. ## Choose a client {#the-clients-skein-js-must-satisfy} | Your application | Client | | ------------------------------------ | -------------------------------------------------------------- | | Node, a service, or a test | `Client` from `@langchain/langgraph-sdk` | | React | `useStream` from `@langchain/react` | | An existing legacy React integration | `@langchain/langgraph-sdk/react` remains supported | | A ready-made chat frontend | [Agent Chat UI](https://github.com/langchain-ai/agent-chat-ui) | | Interactive graph development | The [console](./console.md) or LangGraph Studio | For a JavaScript SDK smoke test with assertions and cleanup, see [Testing your project](./testing-your-project.md#check-the-running-server). ## Vue, Svelte, and Angular {#not-just-react-vue-svelte-and-angular} Use the dedicated framework packages, not SDK subpaths such as `@langchain/langgraph-sdk/vue`: | Framework | Package | Entry points | Tested peer range | | --------- | -------------------- | ----------------------------------------------- | ----------------- | | React | `@langchain/react` | `useStream`, `StreamProvider` | React 18–19 | | Vue | `@langchain/vue` | `useStream`, `provideStream`, `LangChainPlugin` | Vue 3 | | Svelte | `@langchain/svelte` | `useStream`, `provideStream`, `getStream` | Svelte 5 | | Angular | `@langchain/angular` | `injectStream` | Angular 18–21 | Use each package's framework-specific setup. The [compatibility reference](./langgraph-compatibility.md) lists the versions exercised with skein-js and known media limitations. There are no standalone skein-js Vue, Svelte, or Angular example applications yet. ## Version and streaming reference {#dedicated-framework-packages} The React examples use `@langchain/react` 1.2.7 and SDK 1.12.7. React, Angular, Vue, and Svelte 1.2.6 are also verified combinations. Treat these as tested versions, rather than a guarantee about every future client release. Svelte 1.2.6's root `useImages` selector returns an empty projection; use the scoped selector for the subgraph namespace containing media. Vue 1.2.6 proxies media assets, so unwrap an asset before reading its `blob`: ```ts import { toRaw } from "vue"; const blob = await toRaw(images.value[0]).blob; ``` Dedicated framework bindings use thread event subscriptions and commands. The legacy React hook uses run-based streaming. Both use HTTP/SSE; no WebSocket configuration is required. See [streaming](./streaming.md#thread-event-subscriptions) if you implement a custom client. ## Working example {#verification-harness} [`react-usestream`](https://github.com/skein-js/skein-js/tree/main/examples/react-usestream) is a minimal Next.js application. Start its graph server and UI using its README, send two turns, reopen the conversation, and check that state is restored without another execution. Repeat with your own graph before building a larger interface. ## References - [React `useStream` reference](https://reference.langchain.com/javascript/langchain-react/useStream) - [LangGraph JavaScript SDK](https://github.com/langchain-ai/langgraphjs/tree/main/libs/sdk) - [Vue bindings](https://github.com/langchain-ai/langgraphjs/tree/main/libs/sdk-vue) - [Svelte bindings](https://github.com/langchain-ai/langgraphjs/tree/main/libs/sdk-svelte) - [Angular bindings](https://github.com/langchain-ai/langgraphjs/tree/main/libs/sdk-angular) --- # LangGraph Agent Server compatibility Use this reference before pointing an existing LangChain client at skein-js. Start with your current `langgraph.json` and SDK, run a known input, and verify the behavior your application uses with [project smoke tests](./testing-your-project.md#check-the-running-server). ## Verified client combinations | Client or surface | Verified combination and behavior | | ----------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | JavaScript SDK | Legacy SDK 1.9.25/1.9.31 and thread-protocol SDK 1.12.6; examples also use 1.12.7 | | React | Dedicated `@langchain/react` 1.2.6 and 1.2.7; legacy SDK React hook remains supported | | Angular, Vue, Svelte | Dedicated packages 1.2.6, exercised in Chromium | | Thread subscriptions | Multiple turns, restoration, interrupt/resume, cancellation, and reconnect; reconnect does not start another run | | Cross-instance reconnect | Express with Redis queues and thread event buses | | Memory and retention options | Memory, SQLite, and Postgres store indexing, write TTL, and per-read refresh controls | | MCP | Official MCP SDK 1.32.1, stateless Streamable HTTP | | A2A | Official A2A SDK 1.3.0, retained task output and interrupt handling | | Compiled UI and custom applications | UI compiler 1.4.2; Node 24 production artifact with SQLite | | Diagnostics | Checkpoint payload accounting with memory, SQLite, and Postgres; Prometheus and OTLP JSON output | These combinations establish the listed behavior, rather than universal compatibility with every version or backend. The workspace uses LangGraph.js 1.4.7 and a packed consumer uses 1.4.12. Thread-event execution requires LangGraph's public `streamEvents({ version: "v3" })` capability; legacy execution remains available for older graphs and custom agents. For frontend setup, use [the SDK guide](./react-sdk.md). For configuration and exact wire contracts, use [CLI compatibility](./langgraph-cli-compat.md) and the [protocol reference](./agent-protocol.md). ## Client limitations - **Svelte 1.2.6:** root `useImages` returns an empty projection. Use a scoped media selector for the subgraph namespace containing the media. - **Vue 1.2.6:** unwrap a media asset with `toRaw(asset)` before reading `blob`; see the [frontend workaround](./react-sdk.md#dedicated-framework-packages). - **Development UI watchers:** follow the upstream watcher's process lifetime. Your host owns shutdown. ## Intentional differences - HTTP/SSE is supported; WebSockets and gRPC are outside the compatibility target. - Stateless runs keep their threads by default. Set `on_completion: "delete"` to remove them. - Store ordering filters compare numbers only. - Local SQLite persists resources and checkpoints; event replay is process-local and is lost on restart. - Skein adds idempotent run creation and durable, signed, retried webhook delivery. LangGraph also supports ordinary run-completion webhooks. - Managed hosting, billing, evaluations, and application identity policy remain application or service choices. Contributor investigation and dated test evidence are recorded in the [testing guide](https://github.com/skein-js/skein-js/blob/main/docs/testing.md#recorded-compatibility-evidence). --- # Storage Choose storage for the lifetime and deployment of your project. Your graph can continue using LangGraph's `getStore()` interface when you change the backend. | Your project | Choose | What to verify | | ------------------------------------------- | ------------------- | ------------------------------------------------------------------- | | Development or isolated tests | Memory | Graph behavior; development snapshots are not production durability | | Desktop app, CLI, daemon, or one Node owner | SQLite | Reopen the same file and read the thread, checkpoints, and memory | | Shared server | Postgres | Resources and checkpoints survive restart | | Multiple server instances | Postgres plus Redis | Queueing and stream observation work across instances | SQLite supports text/filter search. Semantic indexing uses Postgres with pgvector or an external store. Start with [SQLite embedding](./local-deployment.md), [Postgres deployment](./deploy.md), or the driver reference below. Graph checkpoints hold conversation state/history; store items hold values across conversations. Choose the correct lifetime before adding memory: see [state and persistence](./state-and-context.md) and [agent memory](./memory.md). ## Verify your storage Create a test thread, run a known input, and read its state. Write an item in a test namespace. With SQLite or Postgres, restart using the same storage configuration and read both again. For multi-instance servers, repeat the read through another instance. An embedded memory backend loses its state on process exit; the CLI's development snapshots are a separate convenience. Clean up the test thread and item afterward. Follow the [project persistence checks](./testing-your-project.md#check-state-across-turns-and-restarts) for the expected outcomes and local crash-recovery limits. ## Drivers ### `@skein-js/storage-memory` (dev/tests — and the Redis-less production path) - In-process maps; zero external dependencies. - Paired with an in-memory queue and a `MemorySaver` checkpointer for `skein dev`. - `store` semantic search falls back to a naive scan/embedding compare. - **Not only a dev driver.** `embedPostgresGraphs` uses this queue and event bus whenever no Redis URL is configured, so its retention bounds apply to real traffic — see [embedding.md](./embedding.md#going-to-production) for `SKEIN_MEMORY_BUS_MAX_FRAMES_PER_RUN` and `SKEIN_MEMORY_BUS_MAX_RETAINED_RUNS`. ### `@skein-js/storage-postgres` (prod) - Backed by `pg`; owns tables for assistants (+ assistant_versions)/threads/runs/store items + migrations. `store.migrate()` applies them on boot — idempotent, tracked in a `skein_migrations` table, and serialized by an advisory lock so concurrent boots queue rather than collide. The SQL is compiled into the package (no filesystem access at runtime), so the driver bundles with zero externals — see [bundling.md](./bundling.md). - Uses **`@langchain/langgraph-checkpoint-postgres`** (`PostgresSaver.fromConnString`) for graph checkpoints — we wrap it rather than reimplement checkpointing. - **pgvector** for semantic store search, configured from `langgraph.json`'s `store.index.{embed, dims, fields}` (see [langgraph-cli-compat.md](./langgraph-cli-compat.md)). pgvector is **opt-in**: the base schema needs no extension, so skein runs on a stock managed Postgres out of the box. Only when `store.index` is set does `migrate()` run `CREATE EXTENSION IF NOT EXISTS vector` and add the `embedding` column — which requires a Postgres that ships pgvector (see the provider table in [deploy.md](./deploy.md#1-a-postgres)). #### Indexes, and the one thing to watch on upgrade Migration `0005_performance_indexes` adds the indexes the list/search paths need: composite `(created_at, thread_id)` and `(updated_at, thread_id)` on `threads` matching the `ORDER BY … , ` the queries actually emit, `(status, created_at)` for the status filter, GIN `jsonb_path_ops` on `threads.metadata` and `assistants.metadata` for the `metadata @> …` containment the auth ownership check performs on **every** request, `runs (thread_id, created_at)`, and `store_items (created_at, key)`. `runs_thread_id_idx` is dropped, superseded by the composite with the same leading column. They are built with **`CREATE INDEX CONCURRENTLY`**, so the boot migration does not hold a write-blocking lock while indexing an existing table — a plain `CREATE INDEX` on a large `threads` would stall writes for minutes, at boot, during a rolling deploy. Concurrency has one cost: such a migration cannot be transactional, so a failure partway leaves some indexes created and the ledger row unwritten, and the next boot retries the whole migration. Every statement is `IF NOT EXISTS` for that reason. **If an index build is interrupted** (the pod is killed mid-migration, say), Postgres leaves an _invalid_ index behind — and `IF NOT EXISTS` will then skip it forever, so queries silently never use it. Nothing breaks; it just stays slow. To check and fix: ```sql -- Any invalid indexes? SELECT c.relname FROM pg_class c JOIN pg_index i ON i.indexrelid = c.oid WHERE NOT i.indisvalid; -- Drop each one; the next boot rebuilds it. DROP INDEX CONCURRENTLY ; ``` #### Semantic search: exact by default, HNSW opt-in With `store.index` configured, semantic search ranks by cosine distance over an **unindexed** `embedding` column — an exact scan of every row. Correct, and fine until the store is large. Add `"hnsw": true` to opt into an HNSW index: ```json { "store": { "index": { "embed": "openai:text-embedding-3-small", "dims": 1536, "hnsw": true } } } ``` Off by default deliberately: HNSW is an **approximate** nearest-neighbour index, so turning it on changes which rows a search returns. That is a semantic change, not something to inherit from an upgrade. Enabling it also **pins the column to `vector(dims)`** — pgvector cannot index a dimensionless `vector`, which is how the column is created so the base schema works without knowing `dims`. If rows already exist at a different dimensionality (an embedder or model change), boot fails with an error saying so rather than a raw Postgres one; re-embed or clear those rows before enabling it. Three things to know before turning it on: - **The first boot is not free.** Pinning the column rewrites the table under `ACCESS EXCLUSIVE`, which blocks reads and writes on `store_items` for the duration and queues behind any long-running query already touching it. The index build that follows is concurrent and does _not_ block, but it does hold boot until it finishes. If pgvector warns that the graph no longer fits in `maintenance_work_mem`, raise it — that is the biggest lever on build time. - **Namespace-filtered search needs pgvector ≥ 0.8.** HNSW selects a fixed candidate set (`hnsw.ef_search`) and the namespace predicate is applied _after_ it, so a prefixed search can return fewer rows than exist — or none. skein sets `hnsw.iterative_scan = strict_order` on every connection to prevent that, which requires pgvector 0.8. On an older server it warns once at startup and the post-filter behaviour stands; leave `hnsw` off there. - **Turning it back off does not unpin the column.** `hnsw: false` skips the `ALTER`, it does not reverse it, so a later `dims` change still fails on the pinned column. Undo it by hand: `ALTER TABLE store_items ALTER COLUMN embedding TYPE vector;` An interrupted build leaves an invalid index; the next boot detects it, drops it concurrently and rebuilds. That check exists because pinning a column whose index is invalid rebuilds that index _inline_ and non-concurrently, holding `ACCESS EXCLUSIVE` for the whole build. ```ts import { PostgresSaver } from "@langchain/langgraph-checkpoint-postgres"; const checkpointer = PostgresSaver.fromConnString(process.env.POSTGRES_URI!); await checkpointer.setup(); // idempotent migrations for checkpoint tables ``` ### Bringing your own store (`store.adapter`) Long-term memory is the one repo you can swap without implementing the other five. Point `store.adapter` at a `"path:export"` and skein uses it for `/store/*` **and** for the `getStore()` handed to every graph run; assistants, threads, runs, crons and idempotency keep using the configured driver. ```jsonc { "store": { "adapter": "./src/my-store.ts:store" } } ``` ```ts // src/my-store.ts — LangChain's own JS long-term-memory guide builds on this exact class. import { PostgresStore } from "@langchain/langgraph-checkpoint-postgres/store"; export const store = PostgresStore.fromConnString(process.env.POSTGRES_URI!); await store.setup(); // top-level await: `store.adapter` imports a *ready* store ``` The export may be **either** a LangGraph [`BaseStore`](https://docs.langchain.com/oss/javascript/langgraph/persistence) — `PostgresStore`, `InMemoryStore`, your own — or a skein `StoreRepo`. They are told apart structurally (`BaseStore` has `batch`), and a mis-shaped export fails at **startup** naming the missing method, rather than at the first request that reaches it. Prefer `BaseStore`: it is the wider ecosystem, and `PostgresStore` brings pgvector with HNSW _and_ IVFFlat plus a `"text" | "vector" | "hybrid" | "auto"` search mode — hybrid search being a capability skein's own driver does not have. **skein re-applies its own semantics rather than forwarding them**, because forwarding would be wrong: `InMemoryStore`'s `search` matches namespace prefixes as a raw _string_ (so `["users"]` also matches `["users2", …]` — a cross-tenant read the moment a prefix is derived from a principal), ignores `query` without a vector index, and coerces filter operands with `Number()`. So the adapted store supplies a candidate set and its vector ranking; skein applies filtering, ordering and paging on top. The price is over-fetching; measure this cost with your dataset before switching adapters. Four things worth knowing before you switch: | | | | ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **`put` does a read-back** | `StoreRepo.put` returns the stored item; `BaseStore.put` returns `void`. So the adapter writes then reads, costing a round trip — and it is **racy**: a concurrent write to the same key returns the other writer's item. | | **TTL is partly expressible** | `store.ttl.default_ttl` works — the adapter stamps it onto every write. But `refresh_on_read` cannot be expressed through `BaseStore.get`, and it **defaults to enabled**, so `store.ttl` is refused unless you set `refresh_on_read: false` and accept write-based expiry. A store with no `sweepExpiredItems()` (`InMemoryStore`) refuses `store.ttl` outright; `PostgresStore` has it. A silently ignored retention policy is worse than a boot error. | | **A prefix-less search fans out** | Some stores reject an empty namespace prefix (`PostgresStore` does), so `search({})` enumerates namespaces and searches each. Scores are re-sorted globally afterwards, so semantic ranking still holds — but give a prefix when you can. | | **The store may have its own rules** | `PostgresStore` rejects namespace labels containing `.`, `%`, `_` or `\`, and a root label of `"langgraph"`. Items written under such a namespace by another driver are not reachable through it. | **Your store owns its items.** skein imports _into_ an adapter — `skein import-langgraph`, a restored `.skein/dev-state.json`, the one-time `langgraph dev` auto-import — but never snapshots back out of it, because writing whatever store you brought into `.skein/dev-state.json` every few seconds is not something to do to somebody's database. So an adapted `InMemoryStore` loses its items across a `skein dev` restart while a `PostgresStore` keeps them. `skein dev` prints this at startup rather than letting you find it by losing items. **`store.index` is refused alongside an adapter.** It configures pgvector on the store the adapter replaces, so it could only ever have no effect on search. Configure the index on your own store instead — `PostgresStore` and `InMemoryStore` both take one in their constructor. In code (no `langgraph.json`), the same seam is `ProtocolDeps.storeItems` — wrap a `BaseStore` with `fromBaseStore` first. Pass it as its own field rather than composing `{ ...store, store: mine }`: the bundled drivers expose `maxPageSize` and `durable` as class **getters** over **private** fields, so a spread silently loses them and a prototype clone throws on read. `withStoreItems` is the supported way to do it by hand. ## Checkpointer selection | Storage selection | skein-js uses | | -------------------------------- | --------------------------------------------------- | | Memory (the default `skein dev`) | `MemorySaver` | | Postgres | `PostgresSaver` | | SQLite store/queue pair | `SqliteSaver` sharing the owned resource connection | You rarely wire these drivers by hand. [`@skein-js/runtime`](https://github.com/skein-js/skein-js/tree/main/packages/runtime) assembles the `PostgresSkeinStore` + `PostgresSaver` + Redis queue/bus (and their `dispose()`) for you, two ways: **`buildRuntime`** from a `langgraph.json` (the `skein dev`/`skein up` path), and **`embedPostgresGraphs`** from a graph you hold in code (the durable sibling of `embedInMemoryGraphs` — see [embedding.md](./embedding.md#going-to-production)). Protocol resources (`SkeinStore`) stay separate from LangGraph checkpoints, which is what allows an in-memory dev experience with no database while the checkpoint format stays 100% LangGraph-native — so history endpoints and interrupt/resume behave exactly as LangGraph expects. For SQLite, use `embedSqliteGraphs` from `@skein-js/local`, or the CLI's explicit SQLite pair. It assembles resources, the queue, and the upstream saver with one file owner. See [local deployment](./local-deployment.md) for lifecycle and native packaging. ## Local SQLite storage [`@skein-js/storage-sqlite`](./local-deployment.md) implements the full resource store and a persisted run queue in one exclusively owned file. `@skein-js/local` assembles it with LangGraph's SQLite saver. Unclaimed work persists; abandoned attempts become failures without automatic input replay. ## Per-operation indexing and retention The bundled memory, SQLite and Postgres stores implement optional `StoreRepo.putWithOptions`: ```ts await store.putWithOptions!(namespace, key, value, { index: ["title"], ttl: null }); await store.get(namespace, key, { refreshTtl: false }); await store.search({ prefix: namespace, query: "meeting", refreshTtl: true }); ``` `index: false` stores the item without making it a text/vector search candidate. A field list limits indexed content; `$` includes the whole value. Exact key reads still return the complete item. An explicit nullable TTL disables the configured default for that write. Existing `put` and its `ttl?: number` options remain source compatible. Unsupported custom stores reject requested advanced controls. The LangGraph bridge forwards indexing, write TTL and read/search refresh options, including batch operations. An external BaseStore must expose the corresponding controls; Skein does not emulate persistent expiry with process-local timers. Thread TTL retains deletion as the default. An opt-in `keep_latest` strategy prunes old history without deleting the thread, root checkpoint, referenced child state or pending writes. The retention window slides when state changes; the sweep skips an active run and rearms its next check after pruning. A null TTL pins the thread. The SQLite and Postgres migrations preserve existing pins: older rows without a recorded TTL remain pinned rather than acquiring an unexpected expiry. Pruning and rollback use LangGraph’s public saver methods. A scratch copy is written under a random, operation-specific ID, checked for existing checkpoints, before erasure. Fixed names such as `::skein-rollback` and `::skein-prune` are never used as scratch IDs because they may belong to real threads. If replay fails, the backup's actual ID is included in the error for operator recovery; retain that copy until recovery is complete. These operations require the same thread execution gate as state writes. They preserve namespace coordinates when copying history. Checkpoint pruning retains current root/child namespaces, explicit checkpoint references and pending writes. Delta channels may also require ancestor writes until a full snapshot; those necessary ancestors remain readable after prune and copy. `keep_latest` removes expendable history, not checkpoint data required to reconstruct current state. Exact payload accounting is available at `GET /threads/{thread_id}/storage`; see [diagnostics](./observability.md#http-diagnostics). ### Retention readback Bundled memory, SQLite, and Postgres drivers enrich thread resources with optional `ttl` (minutes), `expires_at`, and `ttl_strategy`. Store `Item` and `SearchItem` resources expose optional `ttl` and `expiresAt` in repository results; HTTP responses serialize expiry as `expires_at`. An absent field means the driver cannot report it; `null` means known no expiry. Thread expiry means eligible for sweeping, rather than a promise of immediate deletion. Refresh-enabled reads return the new expiry. These fields reuse existing storage columns, require no migration, and add no required custom-driver methods. Passive tools can use `refreshTtl: false` to inspect without extending a sliding lifetime. ## `SkeinStore` interface A single interface, implemented by each driver, covering the protocol resources: ```ts interface SkeinStore { // assistants (derived from langgraph.json graphs, plus user-created) assistants: AssistantRepo; // threads: metadata + status (idle | busy | interrupted | error), plus the latest turn's error threads: ThreadRepo; // runs: status + queue rows (pending | running | success | error | cancelled | timeout), // and — for a failed run — why (see errors-and-logging.md) runs: RunRepo; // schedules that fire runs on a cadence crons: CronRepo; // long-term memory: namespace/key items with optional semantic search store: StoreRepo; // recorded responses for `Idempotency-Key`, so a provider's retry replays instead of re-running idempotency: IdempotencyRepo; } ``` Each repo exposes CRUD + list/search shaped to the [Agent Protocol](./agent-protocol.md) endpoints. Memory, SQLite, and Postgres implement the same resource contract. Their queue ownership and crash semantics differ; see [local deployment](./local-deployment.md#what-survives-a-process-ending). ### Page bound (`SKEIN_MAX_PAGE_SIZE`) Every list and search path is **bounded — including when the caller passes no `limit` at all**. The default is **1000 rows**. This is a memory bound: a thread row carries the thread's mirrored graph state, so an unbounded `POST /threads/search` on a large deployment materializes the table twice over (the rows, then the JSON response string) inside one request. | Surface | Bound | | --------------------------------------- | ----------------------------------------------------------- | | `limit` on a search request | rejected above 1000 by the wire schema | | `limit` omitted, or a `list()` call | the first `SKEIN_MAX_PAGE_SIZE` rows (default 1000) | | `assistants.count()`, `threads.count()` | **not** bounded — they answer "how many match" in total | | `runs.listByThread()` | **not** bounded — run rows carry no graph state | | `runs.latestForThread()` | one row by construction — the thread state path uses this | | `runs.listActiveRuns()` (all threads) | bounded — the whole-server sweep behind `POST /runs/cancel` | | `POST /threads/{id}/history` | 100 checkpoints by default, 1000 max — a separate bound | A custom driver's `runs.latestForThread()` must return the newest run by `created_at` descending, tie-broken on `run_id` descending. This ordering makes ties deterministic within the driver. Sizing guidance is in [performance.md](./performance.md#sizing). Set `SKEIN_MAX_PAGE_SIZE` to change the driver bound (`maxPageSize` on the store constructor and on `embedPostgresGraphs` do the same in code). Lowering it is the useful direction on a small container. Raising it widens what an omitted `limit` returns, but **not** the wire cap: a client-supplied `limit` is still rejected above 1000, deliberately, so a single request can't be made arbitrarily expensive from outside. Truncation is **not** signalled on the response today — a short page is indistinguishable from the end of the results, so page with `offset` until you get fewer rows than you asked for. Lowering the bound below 1000 clamps a client-supplied `limit` silently for the same reason, so page by what you _received_ rather than by what you requested. ### Server-enforced metadata (`enforcedMetadata`) `ThreadSearchQuery` carries a second metadata subset alongside `metadata`, AND-ed with it. It is set by the server, never read from a request body, and exists so the auth ownership filter is a `WHERE` clause instead of a full read plus an in-process pass — see [agent-protocol.md](./agent-protocol.md#authentication--authorization). Two subsets rather than one merged object, because a merge would silently drop one side on a key collision: a caller filtering `owner: "bob"` while the ownership filter requires `owner: "alice"` must match **nothing**, not one or the other. Custom drivers must apply both subsets with the same containment semantics as `metadata`. ### Assistant versioning `AssistantRepo` carries full CRUD plus a **version history** (LangGraph parity — see the [assistants endpoints](./agent-protocol.md#assistants)). The model is deliberately simple so nothing else has to change: - Each assistant keeps an append-only list of immutable **version snapshots** (`graph_id`/`name`/`description`/`config`/`context`/`metadata` at that version). - The **live assistant row mirrors the currently-active version** and carries its `version` number. So every existing reader — `get`, `list`, the run engine's graph resolution, thread history — keeps working unchanged; a run always resolves the assistant's _active_ version. - `create` seeds version 1. `update` mints a new version (`max + 1`) and makes it active. `setLatest(version)` rolls the live row back to an existing snapshot **without** minting a new one. `listVersions` returns history newest-first (filterable by metadata, paginated). Deleting an assistant cascades its versions. Keep migrations enabled when upgrading a persistent deployment. ### Long-term memory in the graph (`getStore()`) Use LangGraph's `getStore()` inside a node to read and write values across threads. For example, a support assistant can remember a customer's preferred language between conversations. Skein supplies the store to each run; your application chooses the namespace and access rules. Follow the [memory guide](./memory.md) for profile and collection patterns and checks that a second thread can retrieve the saved preference. Semantic search requires a configured embedding index; SQLite supports text and filter search. ### Filtering and namespace traversal Two ways to narrow a store read: by **content** (`search`'s `filter`) and by **namespace shape** (`listNamespaces`' `prefix`/`suffix`/`maxDepth`). Both are honoured over HTTP and through `getStore()`, across the bundled storage drivers. ```ts await store.search(["users", userId], { filter: { topic: "coffee", score: { $gte: 3 } } }); await store.listNamespaces({ prefix: ["users", "*"], suffix: ["facts"], maxDepth: 3 }); ``` **`filter`** applies to the **top-level** keys of an item's `value`, never a nested JSON path — `"a.b"` is the literal key `"a.b"`. Keys are ANDed, as are multiple operators on one key. The operators are LangGraph's — `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$in`, `$nin` — and a bare scalar means equality. It is applied **before** paging, so a page is a page of matches. | Case | Behaviour | | ------------------------------- | ---------------------------------------------------------------------------------------------------- | | Ordering (`$gt`/`$gte`/…) | **Numbers only** on both sides. A string, boolean, `null`, object, array or absent key never matches | | Key absent from `value` | `$ne` and `$nin` match; `$eq`, a bare scalar, `$in` and every ordering operator do not | | Present JSON `null` | Distinguishable from absent — `{ deleted: null }` means "present and null" | | Empty operator bag `{}` | States no conditions, so it matches everything | | Unknown `$op`, non-scalar value | **400** at the boundary, not a silent narrowing | The numbers-only ordering rule is a **deliberate departure from LangGraph**, which coerces both sides with `Number()` and so treats `"5" > 3` as true. Postgres cannot reproduce that (`'abc'::numeric` throws), and faithful reproduction would also make `true > 0` and `null >= 0` true. The divergence is limited to numeric strings. **Namespace matching** is positional. `"*"` stands for exactly one segment, `prefix` anchors at the first and `suffix` at the last, and a path longer than the namespace never matches. A path _shorter_ than the namespace still matches, which is what makes `prefix` select a subtree: `["users","*"]` matches `["users","1"]` and `["users","1","memories"]` alike. `maxDepth` truncates each match to that many segments and de-duplicates, before sorting and paging — so `{ maxDepth: 1, limit: 10 }` is ten _roots_, not the roots of the first ten namespaces. Namespaces come back in ascending, element-wise order, shorter first on a shared prefix. Per-segment ordering follows the **driver's** collation (Postgres's database collation; UTF-16 code units in memory), which agree for ordinary segments and can differ on exotic ones like `"a-b"` vs `"ab"`. The contract is the element-wise order, not a byte-exact one. > **Driver authors:** `StoreRepo.listNamespaces` accepts a `StoreNamespaceQuery` > (`{ prefix, suffix, maxDepth, limit, offset }`). Apply the driver's page bound when `limit` is omitted. #### Multi-tenancy is yours to define > `prefix`, `suffix` and `filter` are _request parameters_, not access control. An omitted prefix > matches every namespace, so **any caller who can reach `POST /store/items/search` can read every item > in the store** — no wildcard required. skein offers no store-scoping mechanism of its own, deliberately: who owns what, and how a namespace encodes it, is the policy that varies most between deployments. Scope it with an `@auth.on.store` handler that rewrites `value.namespace` — the pattern, and its two limits, are in [agent-protocol.md](./agent-protocol.md#scoping-the-store). ### Store item TTL Store items can expire, matching LangGraph's store TTL. Configure it in `langgraph.json` under `store.ttl` (all durations in **minutes**): ```json { "store": { "ttl": { "default_ttl": 1440, "refresh_on_read": true, "sweep_interval_minutes": 60 } } } ``` - `default_ttl` — lifetime applied to a `put` that doesn't pass its own `ttl`. A `PUT /store/items` body may include a per-item `ttl` (minutes) that overrides the default for that item. - `refresh_on_read` (default `true`) — a `get` extends a live item's expiry by its own TTL. - `sweep_interval_minutes` (default `60`) — how often the background sweeper deletes expired rows. Expiry is enforced two ways: **lazily** (an expired item reads as absent from `get`/`search`/ `listNamespaces` even before it's swept) and by the **sweeper** (a periodic `DELETE`). With no `store.ttl` set, items never expire. The sweeper runs in the production runtime (`skein up`/`build`, and `skein dev --store postgres`); pure in-memory `skein dev` still enforces expiry lazily on read. ### Thread TTL Threads can expire too, configured under `checkpointer.ttl` — the shape LangGraph Platform documents, so the same `langgraph.json` works under both. Durations in **minutes**: ```json { "checkpointer": { "ttl": { "default_ttl": 43200, "strategy": "delete", "sweep_interval_minutes": 60 } } } ``` - `default_ttl` — lifetime applied to a thread created without its own `ttl`. `POST /threads` accepts a per-thread `ttl` (minutes) that overrides it, and an explicit `null` **pins** the thread so no TTL ever collects it. `PATCH /threads/{id}` can change or clear it later; a new value restarts the clock. - `strategy` — `"delete"` (default) removes the expired thread; `"keep_latest"` retains current state, referenced namespace checkpoints and necessary ancestor writes. - `sweep_interval_minutes` (default `60`) — how often the sweeper runs. An expired thread remains readable through `GET /threads/{id}` and search until the sweeper collects it. With the `delete` strategy, collection stops any active run and removes the thread, its runs, and checkpoints. > **Deleting a thread also deletes its thread-scoped schedules.** Pin a thread with `ttl: null` > when its [cron](./crons.md) must keep firing, or use a stateless cron. Per-thread TTLs are collected even without a `checkpointer.ttl` configuration block. Threads without a configured or per-thread TTL live until explicitly deleted. --- # Agent memory A travel assistant should remember “I prefer aisle seats” when the user starts a new conversation. A support agent should remember facts from this ticket without mixing them into another customer's history. Use thread state for the current conversation and the store for facts that need to outlive it. Write and retrieve those facts in your graph with LangGraph's `getStore()`. Configure skein-js storage for the required lifetime, and scope each user's namespaces through your authorization policy. Start with the patterns below; [multi-tenant memory](#multi-tenant-memory) covers isolation. ## What you already have | Need | Use | From | | ------------------------------- | ------------------------------------------------------------ | ---------------------------------------------------------------------------- | | Durable thread state | the upstream checkpointer — `SqliteSaver` or `PostgresSaver` | skein selects it for the chosen storage ([storage.md](./storage.md)) | | Trimming a long conversation | `trimMessages`, `filterMessages`, `RemoveMessage` | `@langchain/core` | | Summarizing a long conversation | `summarizationMiddleware`, `contextEditingMiddleware` | the `langchain` v1 package | | Cross-thread memory store | `getStore()` — namespace/key items, vector search | skein injects it into every run | | Semantic search over memories | `store.index` (pgvector, HNSW optional) | skein ([storage.md](./storage.md#drivers)) | | Per-owner isolation | an `@auth.on.store` handler that roots the namespace | you ([agent-protocol.md](./agent-protocol.md#authentication--authorization)) | | Scheduled background extraction | a cron firing an extractor graph | skein ([crons.md](./crons.md)) | | Memory _shapes_, tools, dedup | this page | your graph and application | ## Short-term memory is already solved **Durability** is the checkpointer. `SqliteSaver` or `PostgresSaver` persists a thread's state and history, so a conversation survives restarts and resumes from an interrupt; thread TTL collects the ones that go stale. Select durable storage when you need conversations to survive a restart. Choose [SQLite](./local-deployment.md) for one local Node owner, or Postgres for a server. SQLite supplies text/filter search; semantic recall requires Postgres indexing or an external semantic store. **Context management** — keeping a long thread inside the model's window — is upstream too: With your existing `model` and `tools`, add summarization middleware: ```ts import { createAgent, summarizationMiddleware } from "langchain"; const agent = createAgent({ model, tools, middleware: [ summarizationMiddleware({ model, trigger: { tokens: 4000 }, keep: { messages: 20 } }), ], }); ``` Keep a conversation summary in **graph state** rather than the store: state is checkpointed, thread-scoped, and rolls back with the checkpoint, none of which is true of a store item. ## Long-term memory: profile and collection Cross-thread memory is `getStore()` — namespace/key JSON documents, the same store LangGraph Platform auto-provides. Two shapes cover almost everything, and the choice is really "one document or many": **Profile** — one continuously-updated document. Good for stable facts with one current value: timezone, tone, home airport. Read it whole, merge a patch, write it back. **Collection** — many documents accumulated over time. Good for open-ended facts: "prefers morning meetings", "allergic to shellfish". Append, and retrieve by relevance rather than reading all of them. ```ts import type { BaseStore } from "@langchain/langgraph"; // Profile: read-modify-write one document. async function mergeProfile(store: BaseStore, userId: string, patch: Record) { const current = (await store.get([userId], "profile"))?.value ?? {}; await store.put([userId], "profile", { ...current, ...patch }); } ``` This example uses a shallow merge: the patch wins, arrays and nested objects are replaced, and `null` is stored as a value. Choose explicit deletion behavior if your profile needs it. Two things to know: - **There is no compare-and-swap.** `BaseStore` has none, so read-modify-write is last-writer-wins. On skein, runs on one thread are serialized by the execution lock, so updates from other threads or direct store clients can still race. Do not build an optimistic-retry counter — without CAS it does not close the race and implies a guarantee that is not there. - **Keep collection fields top-level and scalar.** `search`'s `filter` reads the _top-level_ keys of an item's `value` and takes scalars, so `{ content, kind, createdAt }` filters and `{ meta: { kind } }` does not. `tags: string[]` is storable but not filterable (`$in` asks the opposite question). Full operator semantics in [storage.md](./storage.md#filtering-and-namespace-traversal). ## The dedup trap If your agent saves a fact on every turn, repeated facts can overwhelm useful memories. Do not use a similarity threshold until you have checked what search scores mean for your backend: | Substrate | `score` on a text search | | ------------------------------------------------------ | ------------------------ | | `@skein-js/storage-memory` without an index | **`1` for every hit** | | `@skein-js/storage-postgres` **without** `store.index` | **`1` for every hit** | | `@skein-js/storage-postgres` **with** `store.index` | real cosine similarity | | LangGraph `InMemoryStore` without an index | no `score` field at all | For a backend that returns `1` for every hit, `score >= 0.9` would treat unrelated facts as duplicates and discard them. Two rules: **1. Dedup on content, not similarity.** A content-addressed key makes exact dedup free, with no read at all — saving the same sentence twice upserts one row instead of appending a second: ```ts async function memoryKey(content: string): Promise { const normalized = content.trim().toLowerCase().replace(/\s+/g, " "); const digest = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(normalized)); return `mem-${[...new Uint8Array(digest).slice(0, 8)].map((b) => b.toString(16).padStart(2, "0")).join("")}`; } ``` This example uses the Web Crypto API available on supported Node.js versions, Bun, and Deno. **2. Treat semantic dedup as opt-in, requiring `store.index`.** And check the scores are meaningful before trusting them — at least one candidate scoring **strictly below 1** — falling back to exact dedup otherwise. State the safe direction out loud: **dedup errs toward writing.** A duplicate is recoverable; a dropped memory is not. ## Recall Retrieval is `store.search` with the namespace and filter already right: ```ts const relevant = await store.search([userId, "memories"], { query: latestUserMessage, limit: 5 }); ``` **When to recall is your graph's decision, and skein has no opinion.** Two patterns, both fine: - **Auto-inject** — fetch relevant memories before each model call and fold them into the system message. Personalization does not depend on the model remembering to ask. This is what [`examples/chat-app`](https://github.com/skein-js/skein-js/tree/main/examples/chat-app) does. - **A recall tool** — the model calls `search_memory` when it decides it needs to. Fewer tokens per turn, but it only works when the model chooses well. If you expose memory as a **tool**, bind the namespace server-side and give the model no namespace parameter at all: ```ts // The model may choose *which collection*, from an enum — never which namespace. const userId = config.configurable?.langgraph_auth_user_id; // never model output ``` That is the pattern people get wrong: a namespace the model can name is a namespace the model can escape. ## Multi-tenant memory Decide whether memory belongs to a user, a team, or the whole application. For per-user memory, verify that a second user cannot read the first user's namespace. `prefix`, `suffix` and `filter` are **request parameters, not access control**. An omitted prefix matches every namespace. The control is an **`@auth.on.store` handler** that rewrites `value.namespace` to root the caller — LangGraph's own idiom, and skein honours it. See [agent-protocol.md](./agent-protocol.md#authentication--authorization). It covers the HTTP surface. **`getStore()` inside a graph has no request and no principal**, so nothing guards it for you. Build its namespace from `config.configurable.langgraph_auth_user_id` when using skein's authentication integration, never from model output. Use the same safe identity encoding as your HTTP authorization handler; see [store scoping](./agent-protocol.md#scoping-the-store) for its constraints. To verify your setup, save a preference in one thread and read it from a new thread for the same user. Then repeat as another user and expect no access. With durable storage, repeat after a restart. See [project persistence checks](./testing-your-project.md#check-state-across-turns-and-restarts). ## Writing memories in the background Extracting memories inline costs the user's turn a model call. Two ways to move it off the hot path, both use existing graph and run APIs: **A cron.** Point a [schedule](./crons.md) at an extractor graph that you write and register: ```jsonc { "graphs": { "agent": "./src/agent.ts:graph", "extract-memories": "./src/extract.ts:graph" } } ``` **A delayed follow-up run.** After a conversation turn succeeds, your application can schedule extraction after a short delay. With an existing SDK `client` and source `threadId`: ```ts await client.runs.create(null, "extract-memories", { afterSeconds: 30, input: { thread_id: threadId }, metadata: { extractor: true }, }); ``` The extractor must accept `thread_id` and read that conversation. Each call creates a separate run; the delay does not debounce repeated requests. If you need cancellation, retain the returned run's IDs and cancel that run rather than searching the source conversation's runs. Use durable storage and queue configuration when scheduled work must survive a restart. Verify that the extractor writes the expected fact once and that the next conversation recalls it. Your application chooses when extraction should happen after failures or cancellation. ## See also - [storage.md](./storage.md) — the store itself: drivers, `getStore()`, filters, TTL, scoping, BYO store - [crons.md](./crons.md) — schedules, for the cron extraction pattern - [`examples/chat-app`](https://github.com/skein-js/skein-js/tree/main/examples/chat-app) — a working research assistant with memory and recall - [LangChain's memory guide](https://docs.langchain.com/oss/javascript/concepts/memory) — the profile/collection framing this page builds on --- # Runs & Redis Choose a queue for background runs and configure concurrency for your deployment. Use memory while developing, SQLite for one local Node owner, and Redis when workers or streaming clients span server instances. For submitting individual runs, start with [Runs](./runs.md). `@skein-js/redis` supplies the queue and cross-instance event delivery. It is not a graph checkpointer; Postgres or another configured saver holds checkpoint history. ## Run modes Choose wait, streaming, or background execution using the [run-mode guide](./runs.md#pick-a-run-mode). Runs on the same thread are serialized. Run concurrency below controls how many different threads one instance executes at once. ## Inspect execution {#run-engine} Keep the thread ID and run ID returned by submission. Read the run's status or open it in the [console](./console.md#runs). A failed run retains its error for later inspection; see [Errors & logging](./errors-and-logging.md). Joining a stream observes existing work rather than submitting another run. ## Queue drivers | Deployment | Queue | Recovery | | --------------------------------- | ------ | ---------------------------------------------------------- | | Development and isolated tests | Memory | Pending queue work is process-local | | One local Node owner | SQLite | Unclaimed jobs survive; abandoned attempts become failures | | Shared server or multiple workers | Redis | Delayed jobs survive and stalled work can be redelivered | Choose the matching store/queue configuration in your CLI or embedding helper. An `after_seconds` delay starts work no earlier than its due time. Its survival across restart depends on the selected queue, as described below. Make external effects idempotent when retries are possible. ### In-memory (dev) - Single-process queue + event bus. No external services. - Used by `skein dev` so nothing beyond Node is required locally. - An `after_seconds` delay is a local timer, so it is lost on restart — like every other run already sitting in this queue, which is in-process too. ### SQLite (local applications) - `@skein-js/storage-sqlite` persists the queue in the same file as protocol resources, owned by one process. `@skein-js/local` adds the upstream checkpointer and process-local event bus. - Run creation and background-job admission commit together. Unclaimed jobs retain their original due times across reopening; duplicate enqueue does not shorten a delay. - Claimed or executing attempts abandoned by a process become `ProcessLostError` failures. Skein does not replay their input or resume checkpoints automatically. The application decides. - Live stream frames stay in memory. This is a Node 22/24/26 option for local apps and single-owner services, with no distributed worker support. See [local deployment](./local-deployment.md). ### `@skein-js/redis` (shared servers) - **Job queue ([BullMQ](https://docs.bullmq.io))** — background runs are enqueued in Redis; worker processes across instances consume and execute them. BullMQ provides retries, backoff, and concurrency out of the box. - **Crash recovery** — a stalled job (its worker died mid-run) is moved back to the queue by BullMQ's stalled-job check and retried, so runs survive restarts. - **Delayed runs** — an `after_seconds` delay is handed to BullMQ's own delayed set, which lives in Redis and promotes the job when it comes due, so a scheduled run outlives the process that created it. - **Cross-instance pub/sub** — run stream frames are published to a Redis Stream + channel so a client connected to instance B can join a run executing on instance A (see [streaming.md](./streaming.md)). - **Cross-instance cancellation** — a `RunAbortChannel` over Redis pub/sub carries the _stop now_ signal to whichever instance is executing a run, so a cancel routed to the wrong replica still stops the graph. The cancel itself is durable in the run row before the message is published, so a dropped message costs promptness rather than correctness. See [deploy.md](./deploy.md#scaling-past-one-instance). ### Stream retention {#what-a-frame-costs} Redis retains stream frames so a client connected to another instance can join a run or reconnect. Set bounds for both retained events and each observer's buffer: Two knobs bound what this costs: | Variable | Default | Purpose | | ---------------------------- | ------- | --------------------------------------------------------------------------------------- | | `SKEIN_REDIS_STREAM_MAXLEN` | 10000 | Approximate cap on a run's stream, in frames. `0` disables trimming (TTL only). | | `SKEIN_STREAM_BUFFER_FRAMES` | 512 | Frames one subscriber may queue before it is judged too far behind and its stream ends. | `MAXLEN` exists because the 1-hour TTL bounds a stream in _time_ but not in size, and a chatty graph can produce a very large one well inside an hour. Trimming is approximate (`~`), which lets Redis trim whole nodes instead of walking the stream on every append. A subscriber whose buffer overflows has its stream ended rather than being allowed to grow without bound behind a reader that cannot drain it; the client reconnects with `Last-Event-ID` and replays from the stream, which is what the stream is for. See [streaming and reconnect](./streaming.md) for client behavior. ## Run concurrency How many **queued** (background) runs one instance executes at once. It defaults to **10**, matching the LangGraph CLI's `--n-jobs-per-worker`, so a project moving off `langgraph dev` keeps its throughput. Inline `wait`/`stream` runs never touch the queue and are unaffected. Concurrency is the knob with the widest blast radius: it multiplies memory, Postgres connections, and in-flight graph state at once. Size it together with `PG_POOL_MAX` — see [performance.md](./performance.md#sizing), and note `skein start` warns at boot when the two disagree. **One worker, N concurrent runs.** skein runs a _single_ background worker whose consumer executes up to N runs at a time — which is why the startup banner says `Starting 1 worker, up to 10 concurrent runs` rather than `langgraph dev`'s `Starting 10 workers` (it really does spawn 10 loops; we don't). The observable behavior is the same. > **Upgrading from ≤ 0.9.0?** This default changed: background runs used to execute strictly one at a > time. Nothing about your code or config needs to change, but each instance now does up to 10 runs > concurrently — so check that your Postgres pool has headroom (see > [pool sizing](./deploy.md#connection-budget)), and note that background runs on one thread > using `multitask_strategy: "enqueue"` no longer execute in strict enqueue order. Set > `SKEIN_RUN_CONCURRENCY=1` (or `--concurrency 1`) to restore the previous behavior exactly. Three ways to set it, highest precedence first: | Surface | How | | -------------- | ---------------------------------------------------------------------------- | | CLI flag | `skein dev --concurrency 4` / `skein start -n 4` (`--n-jobs-per-worker` too) | | Environment | `SKEIN_RUN_CONCURRENCY=4`, or the LangGraph-compatible `N_JOBS_PER_WORKER=4` | | Adapter option | `createExpressServer({ deps, worker: { maxConcurrency: 4 } })` | An explicit value wins, but the environment is still validated — so a typo'd `SKEIN_RUN_CONCURRENCY` fails loudly at boot instead of being silently ignored. The environment is the path that reaches a container: add it to the `skein up` compose `environment:` block or your PaaS config. **Per-thread ordering is unaffected.** Two runs on the same thread are serialized by the engine's execution lock at _every_ concurrency, so the per-thread guard above holds regardless. ### Dispatch by thread Bundled memory, SQLite and Redis queues retain a busy thread's pending head while dispatching work for independent threads. A busy execution claim does not hold a worker execution slot or spend a processor-failure attempt. The execution claim remains responsible for serialization, including against inline runs and other instances. The default `multitask_strategy` remains `"reject"`; `"enqueue"` is the strategy that builds a thread backlog. For a custom queue, optional `RunQueue.consumePartitioned` acknowledges handled work with `true`. Return `false` only when execution did not start; the run stays at its thread's head without spending a failure attempt. Optional `ThreadExecutionGate.tryAcquire` returns a lease or `undefined` when busy; backend failures reject. Older queues and gates retain blocking dispatch. Preserve original due times, deduplication, and the execution lease when implementing these public extension contracts. Coordinate worker upgrades so all instances use the same dispatch behavior. Existing queued jobs can remain in place; an upgrade does not require flushing Redis. SQLite retains queued job due times and records abandoned attempts as failures without automatic replay. **Ordering across background `"enqueue"` runs is not guaranteed above concurrency 1.** If your application needs a strict sequence, finish one run before submitting the next, or express the sequence inside your graph. A busy thread should not prevent unrelated conversations from running. **Concurrency vs. replicas.** Raise concurrency when you have many independent threads and runs are I/O-bound (model calls). Add instances when runs are CPU-bound, or when threads are long-lived and serialized. Note each concurrent run holds a Postgres connection — see the pool-sizing note in [deploy.md](./deploy.md#connection-budget) — and, on the Postgres driver, a second one for its per-thread execution claim, held for the whole run. ## Deployment topology (`skein up`) ```mermaid flowchart TB C(["clients · SSE"]) --> A["instance A"] C --> B["instance B"] A --> R[("Redis
queue + pub/sub")] B --> R A --> P[("Postgres
checkpoints · resources
pgvector")] B --> P class R accent ``` `skein up` brings this stack up via Docker Compose. Horizontal scaling is verified by starting a run on instance A and joining its SSE stream from instance B through Redis (see [testing.md](https://github.com/skein-js/skein-js/blob/main/docs/testing.md)). To run the same topology on a hosted platform, the generated image is PaaS-friendly (binds the injected `$PORT`, non-root, `/ok` health probe, graceful `SIGTERM`) — see [deploy.md](./deploy.md) for what every platform needs, plus per-platform guides for Cloud Run, Railway, Fly.io, Render, AWS, Kubernetes and a plain VPS. --- # Cron / scheduled runs Schedules that fire a run on a cadence — LangGraph Platform's **Crons** resource, served by skein-js and driven by the official `@langchain/langgraph-sdk` client. > **Compatibility note:** crons are **not** part of the open > [Agent Protocol](https://github.com/langchain-ai/agent-protocol) spec — its `openapi.json` has no > cron paths. They are a LangGraph Platform / LangSmith Deployment extension, and a paid-tier one. > The OSS [`@langchain/langgraph-api`](https://github.com/langchain-ai/langgraphjs/tree/main/libs/langgraph-api) server > registers the routes but every handler throws `500 Not implemented`, and its `GET /info` reports > `crons: false`. skein implements them against the LangSmith Deployment OpenAPI spec plus the SDK's > TypeScript types — the same oracle the rest of the wire surface uses ([reuse.md](https://github.com/skein-js/skein-js/blob/main/docs/reuse.md)). **A cron is a _cadence_, not a delay.** To run something once, a little later, pass [`after_seconds`](./agent-protocol.md) on an ordinary run create — no cron row, nothing to clean up afterwards. Reach for a cron when the run should keep happening. ## Quick start ```ts import { Client } from "@langchain/langgraph-sdk"; const client = new Client({ apiUrl: "http://localhost:2024" }); // Every weekday at 09:00 New York time, on a fresh thread each fire. const cron = await client.crons.create("agent", { schedule: "0 9 * * 1-5", timezone: "America/New_York", input: { messages: [{ role: "user", content: "Summarize yesterday's issues." }] }, }); // What has it produced? A stateless cron makes a thread per fire, each tagged with the cron id. const threads = await client.threads.search({ metadata: { cron_id: cron.cron_id } }); const runs = await client.runs.list(threads[0].thread_id); await client.crons.update(cron.cron_id, { enabled: false }); // pause await client.crons.delete(cron.cron_id); ``` `assistant_id` accepts a UUID **or a graph name** from your `langgraph.json` — a graph name resolves to the assistant skein auto-registers for it. ## Stateless vs thread crons | | Stateless (`crons.create`) | Thread (`crons.createForThread`) | | --------------- | --------------------------------------------- | ------------------------------------------ | | Thread | A fresh one per fire | The one you named, reused every fire | | State | Starts clean each time | Accumulates across fires | | Default cleanup | `on_run_completed: "delete"` | n/a — the thread is yours | | Concurrency | Cannot collide (each fire has its own thread) | `multitask_strategy` defaults to `enqueue` | Use a **stateless** cron for independent jobs (a nightly digest). Use a **thread** cron when each run should see what the previous ones did (a long-running monitor). > **`on_run_completed` defaults to `"delete"`**, matching LangGraph — and deliberately _unlike_ > skein's own `on_completion` default of `"keep"` for one-off stateless runs. A schedule is the case > where the difference bites: a five-minutely cron keeping every thread accrues over a hundred > thousand of them a year, with nobody watching. Pass `on_run_completed: "keep"` if you want them, > and give the checkpointer a TTL. ## Schedule format **Standard 5-field cron only** — minute, hour, day-of-month, month, day-of-week: ``` ┌───────── minute (0-59) │ ┌─────── hour (0-23) │ │ ┌───── day of month (1-31) │ │ │ ┌─── month (1-12) │ │ │ │ ┌─ day of week (0-6, Sunday = 0) │ │ │ │ │ 0 9 * * 1-5 ``` Six-field (seconds-first) expressions, `@daily`-style nicknames, and `L`/`?` are **rejected with a 422**, matching LangGraph's _"cron must be a standard 5-field expression"_. skein's parser ([croner](https://github.com/hexagon/croner)) would accept them, so the five-field rule is enforced before the expression reaches it: silently running `0 0 9 * * *` at a different time than the author's crontab says is worse than refusing it. `timezone` is an optional IANA name (`America/New_York`); absent or `null` means UTC. DST is handled: an occurrence in a spring-forward gap shifts forward into the same day rather than being skipped, and a repeated fall-back hour fires once. The finest resolution is **one minute**. Sub-minute schedules are not expressible and are a [non-goal](./roadmap.md). ## Endpoints | Method | Path | Notes | | -------- | --------------------------------- | ----------------------------------- | | `POST` | `/runs/crons` | Stateless cron → `Cron` | | `POST` | `/threads/{thread_id}/runs/crons` | Thread cron → `Cron` | | `POST` | `/runs/crons/search` | → `Cron[]`, `x-pagination-total` | | `POST` | `/runs/crons/count` | → a **bare integer** | | `GET` | `/runs/crons/{cron_id}` | → `Cron` | | `PATCH` | `/runs/crons/{cron_id}` | → `Cron` | | `DELETE` | `/runs/crons/{cron_id}` | → **200** with a JSON body, not 204 | All seven answer the full `Cron` object. Two response shapes are deliberately unusual because the official SDK requires them: `count` returns a bare integer rather than `{ count }`, and `DELETE` returns 200 with a body (the SDK skips `response.json()` only for 202 and 204, so an empty 200 makes it throw). `search` accepts `assistant_id`, `thread_id`, `enabled`, `metadata` (subset match), `limit`, `offset`, `sort_by`, `sort_order`, and `select`. `select` is validated and then ignored — skein always returns the whole row. Set `"http": { "disable_crons": true }` in `langgraph.json` to remove the resource. That also stops the scheduler, so a disabled deployment does not keep firing schedules nobody can see or cancel. ## Semantics **Pausing.** `enabled: false` stops a cron without deleting it; the row stays readable and `next_run_date` becomes `null`. Re-enabling recomputes it. **`next_run_date === null` means dormant** — disabled, past `end_time`, or an unreachable expression (`0 0 30 2 *`). Inspect the saved schedule before deciding whether it should fire. **`end_time`** is inclusive: an occurrence falling exactly on it still fires. On `PATCH` it is tri-state — omit to leave it, send `null` to clear it, send a value to set it. Same for `timezone`. **Metadata merges** on `PATCH` rather than being replaced. The run **payload** is replaced wholesale when a patch touches any run field, because a half-replaced request (new `input`, stale `config`) is not a request anyone asked for. **Catch-up.** If the server is down when occurrences pass, an overdue enabled cron fires once on return and resyncs to the next future occurrence. It does not backfill missed occurrences. **Traceability.** Every fired run — and every thread a stateless cron creates — carries `cron_id` in its metadata. For a **stateless** cron that makes `client.threads.search({ metadata: { cron_id } })` the way in, since each fire has its own thread; for a **thread** cron the thread is already known, so `client.runs.list(thread_id)` lists the runs and each carries the `cron_id` that produced it. The tested SDK provides per-thread listing; use [HTTP run search](./runs.md#search-runs-across-threads) when you need a cross-thread query. **Failures.** If a fire fails (a deleted assistant, a store error) the cron is logged at `error`, counted, and **left enabled** — the next occurrence is the retry. skein never auto-disables a schedule: turning a thirty-second rolling-deploy blip into a silently stopped cron needing a human is the worse failure. **Deleting a thread deletes its thread crons**, by foreign key. A cron cannot outlive the thread it runs on. ## Driver requirements Crons work on every supported store/queue combination, including `skein dev` on memory with no Docker. | Store | Queue | Crons | Posture | | -------- | ------ | ------------------------------------------------- | ------------------------- | | memory | memory | Work; snapshotted to `.skein/` by `skein dev` | Development | | postgres | memory | Durable; single-instance delivery | Fine for small production | | postgres | redis | Durable; at-least-once delivery; multi-instance | Shared servers | | memory | redis | Work, but **lost on restart** outside `skein dev` | Warned about at startup | | sqlite | sqlite | Durable schedules and unclaimed queued jobs | One local Node owner | SQLite requires the store/queue pair. Claimed or executing attempts abandoned by a process become failures without replaying their input; the schedule remains enabled for future occurrences. See [local crash behavior](./local-deployment.md#what-survives-a-process-ending). The scheduler logs one warning at startup on a non-durable store. A schedule that quietly stops is worse than one that was never created, and an embedded memory store in production loses every cron on every deploy. ## Execution and recovery {#how-it-works} Multiple instances can share schedules without creating a separate run for the same occurrence. Delivery to the execution queue is retried, so design your graph's external effects to tolerate retries. Use durable storage when schedules must survive restart; the driver requirements above explain which combinations provide that behavior. Scheduling resolution is one minute. Check the saved timezone, next occurrence, enabled state, and end time when a schedule does not fire. For an end-to-end check, create a test schedule with a known input, observe its run in the console, then pause or delete the schedule. ## Operating a scheduler **The one metric to alert on is cron lag** — how overdue the most overdue enabled cron is. Healthy, it stays below one tick; if the scheduler dies it climbs without bound. It is the only signal that distinguishes "nothing is due" from "nothing is running the crons". Each tick reports it, along with counts of fired / claims lost / failed / re-enqueued runs. Claims lost are **not** errors: on N instances, N−1 lose every occurrence by design. | Setting | Default | Notes | | -------------------- | ------- | ------------------------------------------------------------- | | `SKEIN_CRON_TICK_MS` | `30000` | Lower does not help below one minute; higher trades lateness. | **Serverless caveat.** Nothing fires if no process is running. On Cloud Run that means `--no-cpu-throttling` and `--min-instances=1` — see [deploy-cloud-run.md](./deploy-cloud-run.md). Scale-to-zero and cron are incompatible, and no queue changes that. ## Authentication Crons are a first-class auth resource, so `@auth.on.crons.create`, `.read`, `.update`, `.delete`, and `.search` all work — matching the SDK's own `Auth` class. **If you have not registered a cron handler, crons inherit your `threads` scoping.** This matters, because `@auth.on` callbacks are matched by exact event key: a deployment that wrote `.on("threads", …)` has no callback named `crons`, and without the fallback the whole resource would be served unscoped. Since a schedule is really "runs on a thread, later", `threads` is both the safe default and the honest one — cron routes fall back to it, and `create` specifically falls back to `threads:create_run`. **Attaching a schedule to a thread is always authorized as `threads:create_run`**, separately from the cron handler. A cron writes into a thread indefinitely, so it is gated by thread ownership even when your cron handler is more permissive than your thread handler. A thread you cannot read answers 404, exactly as it does everywhere else. A cron fires with no HTTP request behind it, so skein remembers the **creating principal** (beside the row, never on the wire) and replays it into every run. The ownership filters are re-derived from your `@auth.on` handler at fire time rather than frozen at create time, which means: - Editing your auth handler applies to existing crons immediately. - Revoking a principal stops their crons producing runs; re-granting resumes them. - A stateless cron's per-fire thread is owner-stamped, so the user who created the cron can actually find its results in their own `POST /threads/search`. See [agent-protocol.md](./agent-protocol.md#authentication--authorization) for the request lifecycle. --- # The skein console Use the console to test your graph, inspect its state, and understand why a run succeeded, failed, or paused. It runs alongside your skein-js server at `/console/`. ## Open your project {#running-it} From your project directory, with `skein-js` installed and a `langgraph.json` in place, start the development server: ```bash npx skein dev ``` Open the **Console** URL printed by the server, usually `http://localhost:2024/console/`. No separate UI process or account is needed. If you do not have a project yet, follow [Your first agent](./your-first-agent.md). With custom authentication, use your normal API key in the connection control, or follow [development UI access](./console-reference.md#development-authentication). If you embed skein-js in your own server, [mount the console](./console-reference.md#mounting-it-yourself) first; framework adapters do not mount it automatically. ## Test your graph 1. Open **Playground** and choose an **Assistant**. An Assistant is a saved configuration of a graph; each registered graph has one by default. Check the graph ID beneath its name. 2. Use **Chat** when your graph accepts a `messages` channel. Use **JSON** for structured input, following the selected graph's schema and example. 3. Enter an input whose expected result you know, then choose **Send message** or **Run input**. 4. Compare the response or final state with that expected result. For a scaffolded project, choose **echo**, send `hello`, and expect `echo: hello`. This graph needs no model credentials. For your own graph, use an example from its tests or documented input schema. Sending a message to the echo graph does not test your model-backed graph; select that graph's Assistant to test it separately.
![Playground with graph selection, input, and run inspection](/images/console/playground-light.png)
![Playground with graph selection, input, and run inspection in dark mode](/images/console/playground-dark.png)
For example, suppose you are testing a support agent. Send “I was charged twice for my order.” Check that it chooses your billing path and looks up the order before drafting a response. If the reply sounds right but the wrong tool ran, inspect Activity and State before changing the prompt. For a structured classifier, submit the same complaint using the fields in its JSON input schema. ## Understand the result {#runs} The inspector helps you check more than the final answer: | View | Use it to check | | ------------ | ---------------------------------------------------------------------------------- | | **Activity** | Reported node output, tool calls and results, pauses, and failures | | **State** | The latest reported values, including structured output | | **Graph** | Which nodes reported output; highlighting does not identify the node executing now | For a failed run, open the thread's **Runs** tab and select the run to inspect its activity and raw events. Compare the reported error with the server logs; see [Errors & logging](./errors-and-logging.md). A connection problem and a failed graph execution are different states. After a disconnect or an ambiguous submission error, inspect the thread before sending the input again: the run may already exist. **Stop run** requests cancellation on the server. Leaving the page, reloading, or closing an inspection stream does not cancel execution. Reopen the same thread to observe its latest run; opening it does not submit another input. ## Try another case {#threads-and-what-is-waiting-on-you} Continue on the same thread to test a conversation with existing state. Choose **New thread** to test from a fresh conversation. **Recent threads**, or **Threads → Open in Playground**, returns you to an earlier conversation. Each thread keeps its own Chat and JSON drafts in this browser tab. Change your graph and save it to trigger development hot reload, then run a new case. In the echo example, change the reply prefix to `you said:` and check that a new message uses it. Start a fresh thread when earlier state would affect the result.
![Thread inspection with Conversation, Runs, State, and Checkpoints tabs](/images/console/thread-detail-light.png)
![Thread inspection tabs in dark mode](/images/console/thread-detail-dark.png)
## Test pauses and alternative paths {#time-travel} If your graph uses [human approval](./human-in-the-loop.md), run an input that reaches `interrupt()`. Read the **Waiting for you** request and submit the value your graph expects. **Send true** and **Send false** send literal booleans; use the editable response for another JSON value. Check that the resumed run follows the intended path. Repeat on a fresh thread with a different response. For a refund workflow, check that approval reaches your test refund tool and rejection leaves the charge untouched. If the thread has checkpoint history, open **Checkpoints** to inspect an earlier state: - **Fork here** writes your edited values as a new checkpoint. - **Fork and run** writes the edits and runs forward. - **Run from here unchanged** runs forward from the selected checkpoint. Edits use your graph's reducers: an append channel adds values rather than replacing its history. Review the selected checkpoint and the reducer warning before running. These actions can execute your tools again; use test inputs and destinations when exploring side effects. ## Inspect the rest of your project {#overview} Use these screens when your project uses the corresponding capability: | Task | Where to go | | -------------------------------------- | ------------------------------------------------------------------------------------------------------------ | | Find paused or failed conversations | **Overview** or filter **Threads** by status | | Compare saved graph configurations | **Assistants**; see [Assistant versions](./assistants.md#versioning-and-rollback) | | Check long-term memory | **Store**; see [memory](./memory.md) and [storage](./storage.md) | | Check when a scheduled run should fire | **Schedules**; see [cron schedules](./crons.md) | | Inspect configured inbound routes | **Channels**; changes are made in [code/configuration](./channels.md) | | Diagnose a run's outbound delivery | The run's **Deliveries** tab; see [delivery and replay](./webhooks.md#see-what-a-callback-did-and-replay-it) | Lists refresh on demand; observed runs stream live. Overview's recent activity covers recent threads, not every run on the server. For server-wide queries, use [run search](./runs.md#search-runs-across-threads). The console operates on your connected server's real resources with your permissions. Store writes, schedule changes, deletion, and delivery replay affect that server. It does not edit graph code or manage your application's permissions. ## Make your checks repeatable The console is useful for exploring an input and diagnosing a result. Turn important cases into [automated project tests](./testing-your-project.md) so you can check them after future changes. ## Optional walkthrough {#watch-the-console-walkthrough} This silent recording uses offline demo graphs to show running an Assistant, inspecting its thread, and returning to Playground. The steps above work with your own registered graphs. ## Setup reference See the [console setup reference](./console-reference.md) for connections, authentication, [production enablement](./console-reference.md#turning-it-on-in-production), [manual mounting](./console-reference.md#mounting-it-yourself), and advanced run options. --- # Console setup reference Configure how the console connects to your server, enable it in production, or mount it inside your own application. For running and inspecting your graph, start with the [console guide](./console.md). ## Connect to a server `skein dev` serves the console by default. Pass `--no-console` to disable it. Use the connection control in the top right to check the selected server and enter an API key. Keys are sent as `x-api-key`, saved in this browser's `localStorage`, and bound to the server where you entered them. Changing the connection or key resets this tab's drafts and navigation workspace. Drafts otherwise last for the browser tab's lifetime; they are not a backup of your project. Account changes outside the console's connection control are not detected automatically. If you switch users through your application's sign-in flow, close the console tab and open it afresh to clear its drafts. If browser storage is unavailable, the console warns that drafts may not survive a reload. The default mount is same-origin, so it needs no separate CORS configuration. For another server, use `?baseUrl=https://your-deployment`; the choice is remembered. Check the destination before confirming a foreign-server connection. Cross-origin servers must allow the console's origin in [their CORS policy](./recipes/serving.md#cors-for-a-browser-client). ## Development authentication With custom auth, run `skein dev --dev-ui-access` or enable it in `langgraph.json`: ```jsonc { "auth": { "path": "./src/auth.ts:auth", "dev_ui_access": true, // skein dev only; stripped from production builds }, } ``` Both forms print a temporary API key shared by the console and LangGraph Studio. Enter it in the console's connection control or add `x-api-key` under Studio's **Custom Headers**. The key skips your custom `authenticate` callback; your `@auth.on.*` authorization callbacks still run with the `skein-dev-ui-user` identity. It rotates every 15 minutes, with a one-minute grace period for the previous key. The new key appears in the server output; enter it again in each UI. It works only when the dev server binds to loopback (`127.0.0.1`, `::1`, or `localhost`). Without either opt-in, both UIs need normal credentials. Dev UI access allows Studio's `https://smith.langchain.com` origin when no `http.cors` policy is configured; an explicit policy must allow that origin itself. Keep the printed key private; anyone holding it can call the API as that dev identity. ## Turning it on in production **Off by default** under `skein start` and the production image. The console can read and change resources allowed by your server's authorization policy: ```jsonc { "http": { "console": true, // serve at /console // or: "console": "/admin/console" — any path but "/" }, } ``` Requests from the console go through the **same** [`auth`](./agent-protocol.md#authentication--authorization) path as any other client. The temporary UI key is unavailable under `skein start`. On a server with custom auth, use the console's connection control (top right) to supply an API key — it is sent as `x-api-key` and held in `localStorage`, since a static bundle has no server of its own to set a cookie. Two things worth knowing before you enable it on a public host: - The **assets** are served unauthenticated (they are a UI shell; every byte of data behind them is authorized). If that is not acceptable, put the mount path behind your ingress' own auth. - `http.console` is honoured by the **Express** transport, which is `skein start`'s Node runtime. On Bun/Deno the flag logs a warning and does nothing — mount it yourself (below). ## Advanced run options Open **Advanced run options** in Playground when a test needs something beyond the current thread. | Option | When to use it | | -------------------- | -------------------------------------------------------------------------------------------------------- | | Existing thread | Continue a conversation using its exact ID | | Stateless run | Let the server create a thread for this execution; choose completion retention | | Batch | Submit 1–100 JSON objects for the selected Assistant | | Delay | Start later; the value is in seconds | | Busy-thread strategy | Reject, enqueue, interrupt, or roll back when another run is active | | Execution timeout | Set integer milliseconds, within the deployment limit; available only when the server advertises support | | Idempotency key | Protect a repeated submission from creating another run | | Stream modes | Choose the output captured while observing the run | See [run modes and options](./runs.md) for the public contracts. Delayed and keyed submissions run in the background. Closing observation leaves execution running. After an ambiguous failure, inspect the thread before retrying. A failed batch may already have created some runs. ## Resource changes The console uses the server's authorization for every operation. Inspecting a resource does not make changes such as deletion or replay safe to repeat. - **Assistants:** deletion keeps threads by default; cascading deletion is explicit. A default Assistant registered from graph configuration returns at startup. Deleting it does not remove the graph from `langgraph.json`. - **Threads:** copy, prune checkpoints, set retention, or delete. Retention is in minutes; `null` means no expiry, while missing readback means unavailable. Expiry makes a thread eligible for cleanup; active work may defer cleanup. See [thread retention](./threads.md#expiring-threads-automatically). - **Store:** saving replaces the value at the same namespace/key and resets expiry using the chosen lifetime or deployment default. Browsing requests non-refreshing reads. If a custom adapter cannot support them, **Load using normal reads** may extend sliding expiry. See [store TTL](./storage.md#store-item-ttl). - **Schedules:** use five-field cron and the selected timezone. Editing scheduling fields preserves the saved run input and Assistant configuration. An enabled schedule with no future occurrence shows **no upcoming run**. See [schedules](./crons.md). - **Deliveries:** dead deliveries can be replayed. Delivery is at least once, so receivers must deduplicate by the stable delivery ID. See [webhook receivers](./webhooks.md). - **Channels:** inventory is read-only. Edit code or `langgraph.json` and restart to change routes. The screen is available only when channels are configured. See [channels](./channels.md). ## Inspection limits Lists are paged and refresh on demand. Overview samples recent threads. Run details retain a bounded buffer of events and report dropped history; use your logs or tracing backend for a durable diagnostic record. Execution status and observer connection status are separate. Closing an observer never cancels a run; choose **Stop run** to request cancellation. Thread and run inspection tabs can be shared through their hash URLs. A fresh Playground link uses `#/playground?new=1`, optionally with `assistant=`; `#/playground/` continues that thread. ## Mounting it yourself Install `@skein-js/console` in your server project and serve it alongside the protocol. Framework adapters do not mount the console automatically. For an Express application: ```ts import { consoleAssetHeaders, resolveConsoleRequest } from "@skein-js/console"; // app is your Express application; register before a catch-all route. app.use((req, res, next) => { if (req.method !== "GET" && req.method !== "HEAD") return next(); const resolution = resolveConsoleRequest(req.path, { mountPath: "/console" }); if (resolution.kind === "miss") return next(); if (resolution.kind === "redirect") return res.redirect(302, resolution.location); res.set(consoleAssetHeaders(resolution.asset)); return res.status(200).send(Buffer.from(resolution.asset.bytes)); }); ``` Two rules the resolver enforces so you do not have to: - **A bare mount path redirects to its slashed form.** Assets are referenced relatively (so one build works at any mount point); at `/console` the browser would resolve them against the parent directory. - **An unknown path is a `miss`, not a fallback to `index.html`.** Hash routing means a real deep link never arrives, so answering 200-with-HTML would only turn a broken asset reference into a blank page. The SPA derives the API base by dropping the **last segment** of its own path: `/console/` → `/`, `/api/console/` → `/api`. Mount it at `/console` and it finds the server with no configuration. To point it somewhere else entirely, append `?baseUrl=https://your-deployment` (the choice is remembered), which is also how you would host the bundle statically. --- # Observability How to see what your agents are doing in production — traces in LangSmith or Langfuse, metrics and spans in OpenTelemetry, product analytics in PostHog — and how to write a sink for anything else. For **logs** and what happens when a graph throws, see [errors-and-logging.md](./errors-and-logging.md). This doc is about the other two surfaces. ## Three surfaces skein reports what a run did in three places, deliberately carrying different amounts of detail: | Surface | Who reads it | What it gets | | ------------ | ----------------- | ------------------------------------------------------------- | | **The wire** | your API client | `RunError` — a stack only when you set `exposeErrorStacks` | | **The log** | you, the operator | everything: the original `Error`, its stack and `cause` chain | | **A sink** | you, the operator | everything, same as the log — it's server-side too | That last row is the rule worth remembering: **`exposeErrorStacks` governs the wire, not telemetry.** A sink always receives the real `Error`. See [errors-and-logging.md](./errors-and-logging.md). Telemetry itself splits in two, and one interface covers both: - **Traces** — spans for what happens _inside_ a graph: each LLM call, tool, and chain. These come from LangChain's callback system, so any callback-based tracer works (LangSmith, Langfuse, Braintrust, OpenLLMetry). - **Events** — the run's own lifecycle: started, settled, how long, how many frames, what failed. skein emits these itself, from the one code path every run mode goes through. ## Turning it on Three ways, highest precedence first. **1. In code** — pass a sink as `ProtocolDeps.telemetry`. Works everywhere, CLI or not: ```ts import { createPostHogTelemetry } from "@skein-js/posthog"; import { embedInMemoryGraphs } from "@skein-js/server-kit"; const telemetry = createPostHogTelemetry(); const deps = embedInMemoryGraphs({ agent }, { overrides: telemetry ? { telemetry } : {} }); ``` Each `create*Telemetry()` returns `undefined` when its backend isn't configured, so that ternary is the whole "is it enabled" story. Pass an **array** to feed several backends at once. **2. In `langgraph.json`** — for `skein dev` / `skein start`: ```json { "graphs": { "agent": "./src/agent.ts:graph" }, "telemetry": { "langsmith": true, "posthog": { "host": "https://eu.i.posthog.com" }, "otel": true, "paths": ["./src/my-telemetry.ts:sink"] } } ``` `true` enables with defaults, `false` **hard-disables** even when the environment says otherwise, and an object is passed through to the adapter. This is a skein extension; it's additive, so a config carrying it still loads under `langgraph dev`. **3. From the environment** — a provider the config doesn't mention turns itself on when its variables are present: | Provider | Detected from | | ------------- | ------------------------------------------------------------------------------------------ | | LangSmith | `LANGSMITH_TRACING=true` **and** `LANGSMITH_API_KEY` (or the `LANGCHAIN_` equivalents) | | PostHog | `POSTHOG_API_KEY` | | OpenTelemetry | `OTEL_EXPORTER_OTLP_ENDPOINT` · `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` · `OTEL_SERVICE_NAME` | With nothing configured there is no telemetry and no cost — the engine skips building any context at all. A provider that is only _detected_ stays quietly off when it can't be built — its package isn't installed, or its API key is absent. One you **declared** in `langgraph.json` instead **fails at startup**, naming what is missing, because a silent no-op for something you explicitly asked for is worse: the first you'd hear of it is an incident with no traces to investigate. Remove the entry to run without it. > **Why LangSmith needs two variables.** Turning tracing on uploads every prompt, tool argument, and > model response — your users' messages — to a third party. A stray `LANGSMITH_API_KEY` (common in > environments that use LangSmith for something else) is not consent for that. `LANGSMITH_TRACING=true` > is LangChain's own opt-in, so that is the signal skein takes; declaring `telemetry.langsmith` in > `langgraph.json` is the other way to say yes. > **Building an image?** `skein build` can't see a dynamically imported adapter, so declare the > provider in `langgraph.json` and it gets pinned into the image. Environment-detected providers are > deliberately **not** pinned: the build machine's environment isn't the runtime's. ## LangSmith ```bash pnpm add @skein-js/langsmith langsmith export LANGSMITH_API_KEY=lsv2_pt_... export LANGSMITH_TRACING=true ``` Both variables, deliberately — see the note above. Declaring `{"telemetry": {"langsmith": true}}` in `langgraph.json` switches tracing on for you, so the key alone is enough on that path. **What this actually fixes.** LangSmith's tracer already instruments everything inside your graph. What it can't know is what a "run" is on your server, so without help every run lands as an anonymous root trace: no thread, no assistant, no user. This adapter supplies that identity — above all **`session_id`**, the key LangSmith's **Threads** view groups on, so a conversation reads as one thread there just as it does through the Agent Protocol. It deliberately attaches **no tracer of its own**. `@langchain/core` installs a global tracer when `LANGSMITH_TRACING` is on; a second one would send every span twice — you'd pay double and couldn't trust the numbers. The adapter enriches that one tracer instead. Credentials are **read from the environment, never written to it**: skein will not copy an API key into `process.env`, where every module and every child process would inherit it. Set `LANGSMITH_API_KEY` (and `LANGSMITH_ENDPOINT` for self-hosted) in the environment — that is where LangChain's tracer reads them from. | Set on every trace | | | ------------------ | --------------------------------------------------------------------------- | | `metadata` | `session_id` (thread id), `run_id`, `thread_id`, `assistant_id`, `graph_id` | | `metadata` | `ls_user_id` when auth is on, `skein_trigger` | | `tags` | `skein`, `graph:`, `trigger:`, `assistant:` | | run name | the graph id | Full options in the [package README](https://github.com/skein-js/skein-js/blob/main/packages/telemetry-langsmith/README.md). ## PostHog ```bash pnpm add @skein-js/posthog posthog-node export POSTHOG_API_KEY=phc_... ``` Two layers, correlated by `$ai_trace_id` (= skein's run id): - **`skein_run_started` / `skein_run_finished`** — status, duration, queue wait, frame count, and the graph, assistant, and thread. The operational picture. - **`$ai_generation`** — PostHog's LLM Analytics schema, per model call: `$ai_model`, `$ai_provider`, input/output/total tokens, `$ai_latency`, `$ai_is_error`. This is what fills PostHog's LLM dashboards — cost per user, tokens per conversation. Token counts come from the message's `usage_metadata` where the provider supplies it, falling back to the older `llmOutput.tokenUsage`. When neither is present the token fields are **omitted** rather than reported as zero — a missing number is more honest than a wrong one. Runs are attributed to the authenticated user's identity, falling back to the thread id so anonymous traffic groups per conversation instead of collapsing into one distinct id. Override with `distinctId`. Turn the LLM layer off with `captureGenerations: false`. See the [package README](https://github.com/skein-js/skein-js/blob/main/packages/telemetry-posthog/README.md). ## OpenTelemetry ```bash pnpm add @skein-js/otel @opentelemetry/api ``` `@skein-js/otel` records through the OTel **API** without creating an SDK or exporter. Optional HTTP metric collection uses official serializers. Your app owns the SDK and decides where data goes, exactly as it already does for the rest of your service. That's why one small package covers every OTLP backend. If no SDK is registered, the API's no-op implementation takes over and this costs almost nothing. **Span** `skein.run `, one per run, with `skein.run.id` / `.thread.id` / `.assistant.id` / `.graph.id` / `.trigger` / `.status` / `.queue_ms` / `.frames` / `.failing_nodes`, plus the `gen_ai.*` semconv equivalents. A failure records the exception and sets an `ERROR` status. **Metrics** `skein.runs` (counter), `skein.run.duration` (histogram, ms), `skein.run.queue.duration` (histogram, ms), and `skein.run.frames` (histogram), dimensioned by graph, trigger, and status — deliberately **low-cardinality**, with no run, thread, or user id, so they stay cheap in Prometheus and friends. ### Spans inside the graph skein reports the _run_. For LLM- and tool-level spans, add a LangChain instrumentation — [`@traceloop/node-server-sdk`](https://github.com/traceloop/openllmetry-js) or [`@arizeai/openinference-instrumentation-langchain`](https://github.com/Arize-ai/openinference). The OTel sink makes the Skein run span active while LangGraph executes. Instrumentation that respects the OTel context therefore nests model, tool, and chain spans under the run span, including across awaits; the run and thread attributes remain available for correlation and filtering too. ### Datadog, Grafana, Honeycomb, Jaeger, New Relic, Sentry All of these are the OTel adapter plus their own exporter — no skein-specific code. Configure the SDK the way that vendor documents, then: ```json { "telemetry": { "otel": true } } ``` For most, exporting `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_HEADERS` at their collector is the entire integration. ## Other backends ### Langfuse Langfuse ships a LangChain callback handler, so it needs only a small sink: ```ts import { CallbackHandler } from "langfuse-langchain"; import type { TelemetrySink } from "@skein-js/core"; export const sink: TelemetrySink = { name: "langfuse", callbacks: (context) => [ new CallbackHandler({ sessionId: context.threadId, // groups a conversation in Langfuse userId: context.userId, metadata: { run_id: context.runId, graph_id: context.graphId }, }), ], }; ``` ```json { "telemetry": { "paths": ["./src/langfuse.ts:sink"] } } ``` ### Braintrust Same shape — Braintrust's LangChain handler in `callbacks`, with the thread id as the span's parent identifier. ### Sentry Sentry reads OTel spans, so `@skein-js/otel` plus Sentry's SDK covers tracing. For error _events_ specifically, a lifecycle sink is more direct: ```ts import * as Sentry from "@sentry/node"; import type { TelemetrySink } from "@skein-js/core"; export const sink: TelemetrySink = { name: "sentry", onRunEvent: (event) => { if (event.type !== "run.finished" || !event.cause) return; // `event.cause` is the original Error — stack and cause chain intact. Sentry.captureException(event.cause, { tags: { graph_id: event.context.graphId, status: event.status }, contexts: { skein: { run_id: event.context.runId, thread_id: event.context.threadId } }, }); }, }; ``` ### Your existing logger For skein's **own** reports — failed runs, webhook failures — the seam is `ProtocolDeps.logger`, already wired to the host's logger under NestJS and Fastify. See [errors-and-logging.md](./errors-and-logging.md#logging). For **run lifecycle events** in your structured logs, a sink is a ten-line bridge over `onRunEvent`, the same hook the Sentry example uses. ## Writing your own sink The whole interface, from `@skein-js/core`. Every method is optional — implement the half you need: ```ts export interface TelemetrySink { name: string; /** Run lifecycle. Fire-and-forget: never awaited. */ onRunEvent?(event: RunTelemetryEvent): void; /** LangChain callback handlers, so a tracer's spans nest under the run. */ callbacks?(context: RunTelemetryContext): unknown[]; /** Extra metadata / tags stamped on the graph call. */ traceMetadata?(context: RunTelemetryContext): Record; traceTags?(context: RunTelemetryContext): string[]; /** Make a backend context active while the graph executes. */ withRunContext?(context: RunTelemetryContext, body: () => Promise): Promise; /** Drain buffered data — called on shutdown. */ flush?(): Promise; shutdown?(): Promise; } ``` `RunTelemetryContext` carries `runId`, `threadId`, `assistantId`, `graphId`, `userId`, `trigger` (`wait` / `stream` / `background` / `invoke`), `streamModes`, and the run's `metadata`. `run.finished` adds `status`, `durationMs`, `frameCount`, and on failure `error` (the JSON-safe `RunError`), `failingNodes` (which graph node threw), and `cause` (the original `Error`). Two things to know: - **`assistantId` is absent for `trigger: "invoke"`.** The `POST /invoke/:graph_id` surface addresses a graph directly, with no assistant in between. It emits lifecycle telemetry using a synthetic run identity for correlation, but deliberately creates no persistent run row. - **Sinks may not throw or block.** Every call is guarded, so a throw is logged and swallowed rather than failing a run — but a sink doing inline I/O still slows every run it observes. Buffer, and implement `flush()`; skein calls it on shutdown so buffering is the safe default. Point `langgraph.json` at it: ```json { "telemetry": { "paths": ["./src/my-telemetry.ts:sink"] } } ``` The export may be the sink itself or a function returning one, so your module can read its own configuration. ## Cost and safety - **Off by default, free when off.** With no sink configured the engine builds no context and reads no metadata. There is no "disabled telemetry" tax. - **A broken sink can't break a run.** Every method is wrapped; a throw is logged through `ProtocolDeps.logger` and dropped. One sink failing doesn't stop the others. - **Flushed on shutdown.** The run worker and the runtime both drain sinks when stopping, so batching exporters don't lose the tail of a process — which is exactly the telemetry you want after a crash. - **Metric cardinality.** skein's own OTel metrics carry no run, thread, or user id. If you write a sink, resist adding them as metric dimensions — they belong on spans and events, not counters. - **What leaves your server.** Traces carry your graph's inputs and outputs, which for an agent means user messages. Check that against your data-handling obligations before pointing them at a hosted backend; self-hosted LangSmith, Langfuse, and any OTLP collector are all supported. ## Learn more - [Errors & logging](./errors-and-logging.md) — the other two reporting surfaces - [Runs & Redis](./runs-and-redis.md) — the run engine these events come from - [Deploy](./deploy.md) — running skein in production ## HTTP diagnostics `GET /openapi.json` describes the actual mounted routes and request validators. `GET /docs` serves an offline API reference using that document. Disabled route groups are omitted, and disabling the meta group removes both endpoints and `/metrics`. Deployment-provided `auth.openapi` security schemes describe existing authentication; custom policy remains in the host. `GET /metrics` collects an explicitly supplied host metric reader. With `createOtelTelemetry`, pass `metricReader` already registered on your `MeterProvider`; install the optional official `@opentelemetry/exporter-prometheus@^0.222.0` and/or `@opentelemetry/otlp-transformer@^0.222.0` serializers for the formats you expose. The default response is Prometheus text; `?format=json` (or an Accept header for JSON) returns standard OTLP JSON. Skein owns neither provider lifetime nor exporter/backend selection. A deployment without a collector returns 501; collection failures return 503. Alongside existing run/queue instruments, the sink records `skein.worker.active_runs`, `skein.http.requests`, and `skein.http.duration`. HTTP duration measures handler response latency; stream consumption may continue afterwards. HTTP dimensions contain method, handler and status, without thread IDs, caller IDs or credentials. Internal MCP/A2A service calls do not count as additional HTTP requests. Authorized `GET /threads/{thread_id}/storage` reports exact stored saver payload sizes: checkpoints, channel versions, pending writes and their total. Memory sums encoded buffers, SQLite uses stored blob lengths, and Postgres measures stored JSONB and BYTEA datums, including database compression, in one snapshot. Inline channel values count in checkpoint bytes. Counts exclude indexes, table/page overhead and filesystem allocation; this is payload accounting rather than a disk-space estimate. Custom savers without an inspector return 501. --- # Recipes Choose a task, follow its recipe, and run the linked example to compare the result with your own project. Each recipe uses existing public APIs. For a terse API reference, see [Using skein-js](../using-skein.md). ## Read working code first [`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) covers crons, background runs, idempotency, human-in-the-loop, long-term memory and time travel in **one graph** — and runs with **no API key and no network** (bundled fixtures, deterministic fallback classifier). ```bash pnpm --filter @skein-js/example-triage-agent dev # console at http://127.0.0.1:2024/console/ pnpm --filter @skein-js/example-triage-agent seed # register the schedule + sweep once ``` ## The recipes | Page | Covers | | ---------------------------------------------------------- | --------------------------------------------------------------------------- | | [Serving](./serving.md) | Pick an adapter, serve a graph as a plain endpoint, CORS for a browser | | [Running agents](./running-agents.md) | Background runs, crons, idempotency, human-in-the-loop, run timeouts | | [Coupled WhatsApp workflow](./coupled-channel.md) | A workflow with the same provider as source and destination | | [Cross-provider workflow](./decoupled-channel-delivery.md) | Email source → LangGraph workflow → allowlisted WhatsApp/email destinations | | [Memory](./memory.md) | `getStore()`, semantic search, and the dedup trap | | [Deep Agents](./deep-agents.md) _(experimental)_ | Files, per-user memory, streamed subagents, approval and async subagents | | [Authenticating requests](./authentication.md) | Better Auth, Clerk, Supabase, Firebase, OIDC/JWT, and authorization policy | | [Production](./production.md) | Auth, run-completion webhooks, durable storage and deploying | | [Local apps and SQLite](../local-deployment.md) | Desktop, CLI, and daemon hosts with one owned database file | ## Which example shows what | Example | Shows | | -------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | | [`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) | Crons, background runs, idempotency, HITL, memory, time travel | | [`decoupled-delivery`](https://github.com/skein-js/skein-js/tree/main/examples/decoupled-delivery) | Routed channels: email source → approval workflow → WhatsApp/email destinations | | [`chat-app`](https://github.com/skein-js/skein-js/tree/main/examples/chat-app) | Full-stack chat: streaming, thinking, tool cards, HITL, memory | | [`deep-agent`](https://github.com/skein-js/skein-js/tree/main/examples/deep-agent) | Research briefing assistant on Deep Agents _(experimental)_: memory, subagents, approval | | [`invoke-endpoint`](https://github.com/skein-js/skein-js/tree/main/examples/invoke-endpoint) | Non-chat graphs as plain HTTP endpoints | | [`embed-graph`](https://github.com/skein-js/skein-js/tree/main/examples/embed-graph) | A graph you already have, served with no `langgraph.json` | | [`migrated-langgraph`](https://github.com/skein-js/skein-js/tree/main/examples/migrated-langgraph) | The drop-in proof — a stock LangGraph project under `skein dev` | | [`react-usestream`](https://github.com/skein-js/skein-js/tree/main/examples/react-usestream) | A minimal `useStream` frontend against any skein server | | [Daybook — Electron](https://github.com/skein-js/skein-js/tree/main/examples/desktop-electron) | Meeting context tools, streaming, recap review, and Markdown revisions | | [Reading Room — Tauri](https://github.com/skein-js/skein-js/tree/main/examples/desktop-tauri) | Local documents, library tools, source excerpts, and reviewed reading briefs | | [Dayplan — CLI](https://github.com/skein-js/skein-js/tree/main/examples/local-cli) | Local task files, previous plans, and approval before saving | --- # Serving Getting the protocol in front of clients: which adapter, the non-chat surface, and browser access. ## Pick a framework adapter Every adapter serves the identical protocol and takes the same `{ config } | { deps }` seam — pick the framework you already run. Each has a **standalone** entry (a dedicated server) and an **embedded** one (mount beside your existing routes). | Framework | Package | Standalone | Embedded | Examples | | --------- | ------------------- | --------------------- | -------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Express | `@skein-js/express` | `createExpressServer` | `skeinRouter` | [express-basic](https://github.com/skein-js/skein-js/tree/main/examples/express-basic), [embed-graph](https://github.com/skein-js/skein-js/tree/main/examples/embed-graph) | | Fastify | `@skein-js/fastify` | `createFastifyServer` | `skeinPlugin` | [fastify-basic](https://github.com/skein-js/skein-js/tree/main/examples/fastify-basic), [fastify-app](https://github.com/skein-js/skein-js/tree/main/examples/fastify-app) | | NestJS | `@skein-js/nestjs` | `createNestServer` | `SkeinModule.forRoot` | [nestjs-basic](https://github.com/skein-js/skein-js/tree/main/examples/nestjs-basic), [nestjs-app](https://github.com/skein-js/skein-js/tree/main/examples/nestjs-app) | | Next.js | `@skein-js/nextjs` | route handlers | `createSkeinRouteHandlers` (App) · `createSkeinPagesHandler` (Pages) | [nextjs-app](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-app), [nextjs-basic](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-basic) | Bun and Deno use [`@skein-js/fetch`](../deploy.md), selected with `skein build --runtime`. For a framework skein doesn't ship, the adapters are ~40-line shims over one handler table — see [building-an-adapter.md](../building-an-adapter.md). Mount snippets: [using-skein.md](../using-skein.md#mount-it-on-your-framework). ## Serve a graph as a plain endpoint For a classifier, extractor, or a workflow another service calls — no threads, no runs. The request body **is** the graph input; the response **is** its final state. ```ts const { router } = await skeinInvokeRouter({ deps: embedInMemoryGraphs({ triage }) }); app.use(router); // curl -X POST localhost:2024/invoke/triage -d '{"text":"…"}' ``` Working version: [`invoke-endpoint`](https://github.com/skein-js/skein-js/tree/main/examples/invoke-endpoint) — two non-chat graphs, no model, no API key. Details: [serving-a-single-graph.md](../serving-a-single-graph.md). ## CORS for a browser client CORS is **off by default**. Same-origin needs nothing — see [`nextjs-app`](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-app), which serves the protocol and the UI from one app. ```jsonc // langgraph.json — matches the LangGraph CLI { "http": { "cors": { "allow_origins": ["http://localhost:3000"] } } } ``` Or pass `cors` to any adapter (`true` for permissive dev, `false` to force off). Cross-origin example: [`react-usestream`](https://github.com/skein-js/skein-js/tree/main/examples/react-usestream). --- # Running agents Getting work started, keeping it from duplicating, pausing it for a human, and bounding it. ## Background runs, join and cancel Kick off a long run, return immediately, stream it from anywhere. ```ts const run = await client.runs.create(threadId, "agent", { input }); // returns immediately for await (const ev of client.runs.joinStream(threadId, run.run_id)) console.log(ev); await client.runs.cancel(threadId, run.run_id); ``` Cross-instance join needs the Redis queue + bus ([runs-and-redis.md](../runs-and-redis.md)). Concurrent runs on **one** thread follow `multitask_strategy`; how many run at once across **different** threads is [run concurrency](../runs-and-redis.md#run-concurrency) (default 10). Working version: [`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) — one background run per issue, each on its own thread. Details: [runs.md](../runs.md); for fire-and-forget work with no conversation to keep, [background-jobs.md](../background-jobs.md). ## Scheduled runs (crons) Fire a graph on a schedule. Schedules live in the store, so they survive restarts and fire exactly once across instances — no leader election. ```ts await client.crons.create("agent", { schedule: "*/5 * * * *", input: { source: "queue" } }); ``` Cron delivery is **at-least-once**: an occurrence committed but never queued is re-enqueued by the scheduler's sweep. Working version: [`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) — a sweep that dispatches one run per issue. Details: [crons.md](../crons.md). ## Don't create the same run twice A retrying caller — Twilio, Stripe, GitHub, or your own sweep — should not start a second run. Send an `Idempotency-Key` and the original response replays. ```ts // `runs.create` has no `headers` option — a `headers` key in the payload is silently dropped and you // get a brand-new run every retry. A per-key client is the only way to send it. const client = new Client({ apiUrl, defaultHeaders: { "Idempotency-Key": issue.id } }); await client.runs.create(threadId, "agent", { input }); ``` The claim is an insert arbitrated by a uniqueness constraint, so 50 concurrent retries across two instances still produce exactly one run. Keys are scoped per principal, failures are never recorded, and the **streaming** creates reject the header rather than ignoring it — an SSE response has no body to replay. **LangGraph Platform has no equivalent.** Working version: [`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) — re-sweeping creates nothing. Details: [agent-protocol.md](../agent-protocol.md#idempotent-run-creation-idempotency-key). ## Human-in-the-loop (interrupt / resume) Pause for approval, resume later — possibly hours later, from a different client. An interrupted run holds no connection and no timer, only a checkpoint. ```ts import { interrupt } from "@langchain/langgraph"; async function approve(state) { const decision = interrupt({ question: "Send this email?", draft: state.draft }); return { sent: decision === "yes" }; } ``` The thread's status becomes `interrupted` and the interrupt surfaces in the stream; resume by submitting a `command`. [`useStream`](../react-sdk.md) renders and resumes it for free. Working versions: [`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) (reached by a conditional edge) and [`chat-app`](https://github.com/skein-js/skein-js/tree/main/examples/chat-app) (approval card in the UI). Details: [human-in-the-loop.md](../human-in-the-loop.md). ## Bound a runaway run A graph that hangs — a model call with no timeout of its own, a node that loops — holds a worker slot until the process restarts. Set `--run-timeout ` or `SKEIN_RUN_TIMEOUT_MS`; the run aborts and settles as `timeout`. **Off by default, deliberately.** A legitimate research or multi-step tool run takes minutes, so a default would turn slow-but-working into killed — the exact failure the timeout exists to prevent. Pick a number from your own graphs' worst honest case. ## Re-run a turn a different way Fork from any past checkpoint and run from there, leaving the original branch intact — useful for retrying a decision with different input, or debugging what a node saw. ```ts await client.runs.create(threadId, "agent", { input, checkpointId }); ``` The fork target is server-validated, so a client cannot redirect a run to an arbitrary checkpoint through config. Visible in the [console](../console.md); details: [threads.md](../threads.md#time-travel-re-run-a-turn-a-different-way). --- # Build a LangGraph WhatsApp workflow with a coupled channel Use a coupled channel when a workflow receives messages from a provider and its outcome naturally goes back through the same provider: WhatsApp → LangGraph → WhatsApp, Slack → LangGraph → Slack, or email → LangGraph → email. One `Channel` owns both translations, while Skein supplies authentication, deduplication, thread resolution, interrupt/resume, and durable delivery. Consider a customer who sends **“Where is order GT-1042?”** to a shop's WhatsApp number. The graph looks up the shipment and replies **“GT-1042 left our Nairobi warehouse and arrives tomorrow.”** The sender, conversation, and expected return provider are already the same, so splitting the source and destination would add ceremony without adding control. ## When coupling is the simpler design Coupling is useful, not a limitation, when the inbound event already contains the correct reply address. A WhatsApp sender expects a WhatsApp response, and keeping both provider mappings together means fewer names, maps, and failure paths. Use a [cross-provider workflow](./decoupled-channel-delivery.md) when LangGraph must choose a different destination—for example, an email source that should alert a manager over WhatsApp. ## Implement the WhatsApp channel The event's `replyTo` is opaque to Skein. It is stored with the run and passed back to the same channel's `deliver` method after the run settles. ```ts import type { Channel } from "@skein-js/channels"; import { z } from "zod"; const whatsappReplyTargetSchema = z.object({ to: z.string().min(1) }); export const channel: Channel = { name: "whatsapp", verify(request) { if (!verifyWhatsAppSignature(request)) return false; const message = request.form(); return { identity: `channel:whatsapp:${message["From"]}` }; }, parseEvent(request) { const message = request.form(); if (!message["Body"] || !message["From"]) return { kind: "ignore" }; return { kind: "event", event: { threadKey: message["From"], idempotencyKey: message["MessageSid"], input: { customerMessage: message["Body"] }, resumeWith: message["Body"], replyTo: { to: message["From"] }, }, }; }, async deliver(outcome, target) { if (outcome.reply === undefined) return; const { to } = whatsappReplyTargetSchema.parse(target); await sendWhatsAppMessage({ to, body: String(outcome.reply) }, { key: outcome.runId }); }, }; ``` Provider retries cannot create duplicate runs when `idempotencyKey` is stable. Delivery is at-least-once, so use `outcome.runId` as a provider idempotency key when the provider supports one. ## Return a reply from LangGraph A LangGraph node can declare the exact value delivered by the channel with Skein's existing `replyWith` helper. If it does not, Skein can fall back to the last AI message in message-shaped state. ```ts import { Annotation, END, START, StateGraph, type LangGraphRunnableConfig, } from "@langchain/langgraph"; import { replyWith } from "@skein-js/agent-protocol"; const State = Annotation.Root({ customerMessage: Annotation }); async function answerOrderQuestion(state: typeof State.State, config: LangGraphRunnableConfig) { const orderId = /GT-\d+/.exec(state.customerMessage)?.[0]; const shipment = orderId ? await orderSystem.findShipment(orderId) : undefined; const answer = shipment ? `${orderId} ${shipment.status} and arrives ${shipment.estimatedArrival}.` : "I could not find that order. Please send the order number, for example GT-1042."; config.writer?.(replyWith(answer)); return {}; } export const graph = new StateGraph(State) .addNode("answer-order-question", answerOrderQuestion) .addEdge(START, "answer-order-question") .addEdge("answer-order-question", END) .compile(); ``` Bind the channel to that graph: ```jsonc { "graphs": { "order-support": "./src/graph.ts:graph" }, "skein": { "channels": { "whatsapp": { "path": "./src/whatsapp-channel.ts:channel", "assistant": "order-support", "public_url": "https://api.example.com", }, }, }, } ``` The result follows one durable path: ```text WhatsApp webhook → channel.parseEvent → LangGraph → channel.deliver → WhatsApp ``` For a runnable implementation with `interrupt()` and resume, see [`examples/whatsapp-agent`](https://github.com/skein-js/skein-js/tree/main/examples/whatsapp-agent). --- # Build a cross-provider workflow with Skein channels Use a routed workflow when a source starts work in one system and LangGraph must choose a different destination: email → LangGraph → WhatsApp, WhatsApp → LangGraph → email, or one source routed to several application-owned provider adapters. The technical mechanism is decoupled channel delivery: Skein supplies the source and destination, while LangGraph fills the middle with the workflow. Here, a workflow means the complete business process, not merely forwarding a message: validate the request, collect approvals, pause and resume, decide the result, and notify the customer. The [workflows and channels guide](../channels.md#what-workflow-means-here) explains why these responsibilities are split across Skein and LangGraph. The source still uses Skein's normal channel pipeline. The only extra layer is an allowlisted destination map plus one explicit instruction written by the graph. The example below handles a concrete request: a customer emails **“Please refund KES 27,500 for order GT-1042; I was charged twice.”** LangGraph asks Finance for approval over WhatsApp, the Finance reply resumes the interrupted graph, and the customer receives the decision by email. ## See the refund approval workflow The recording follows the complete example: an email starts the workflow, LangGraph pauses for the three required approvals, each authenticated WhatsApp response resumes its own interrupt, and the approved result is delivered back to the customer by email. ## When a workflow should use separate source and destination Separate source and destination providers are appropriate when routing is a workflow decision rather than an inherent response: - an ERP order event alerts a sales team over WhatsApp; - an email refund request asks HR, a manager, and Finance for approval; - a personal assistant reads calendar or email events and sends a WhatsApp briefing. If a WhatsApp message simply needs a WhatsApp reply, use the smaller [coupled WhatsApp workflow recipe](./coupled-channel.md). Both forms use the same `Channel`, run, thread, LangGraph, and outbox primitives; decoupled delivery is an additional routing shape, not a replacement. ## 1. Define the workflow's inbound source A source verifies and parses only. `composeRoutedChannel` arms the existing durable callback path, so the source does not need a fake email `deliver` method. Both inbound providers use one trusted, tenant-scoped workflow ID when they intentionally converge on the same thread: ```ts const tenantId = "acme"; // Deployment configuration, never inbound request data. export function workflowThreadId(refundId: string): string { return `relay:${tenantId}:${refundId}`; } ``` Here `refundId` is issued by the application or verified provider metadata; do not derive an authoritative thread ID directly from email text or another user-controlled field. ```ts import type { Channel } from "@skein-js/channels"; import { z } from "zod"; import { workflowThreadId } from "./workflow-thread-id.js"; const refundEmailSchema = z.object({ messageId: z.string().min(1), refundId: z.string().min(1), from: z.string().email(), orderId: z.string().min(1), amountKes: z.number().positive(), reason: z.string().min(1), }); export const emailSource = { name: "email-source", verify(request) { if (!verifyEmailWebhook(request)) return false; return { identity: "channel:email:inbound" }; }, parseEvent(request) { const email = refundEmailSchema.parse(request.json()); return { kind: "event" as const, event: { threadKey: email.refundId, threadId: workflowThreadId(email.refundId), idempotencyKey: email.messageId, input: { source: "email", refundId: email.refundId, customerEmail: email.from, orderId: email.orderId, amountKes: email.amountKes, reason: email.reason, financeWhatsapp: "whatsapp:+254700000013", }, }, }; }, } satisfies Pick; ``` ## 2. Allowlist workflow destinations Destination callbacks own credentials, provider validation, authorization, and idempotency. Names are explicit and local to this composed channel. The map allowlists provider adapters, not recipients: authorize every target from trusted application data rather than treating an LLM-selected or user-supplied phone number as permission to send. ```ts import { composeRoutedChannel, type ChannelDestinationDelivery } from "@skein-js/channels"; import { z } from "zod"; const whatsappDeliverySchema = z.object({ target: z.object({ to: z.string().min(1) }), payload: z.object({ body: z.string().min(1) }), }); const emailDeliverySchema = z.object({ target: z.object({ to: z.string().email() }), payload: z.object({ subject: z.string(), body: z.string() }), }); export const destinations = new Map([ [ "whatsapp", async (delivery: ChannelDestinationDelivery) => { const message = whatsappDeliverySchema.parse(delivery); await assertAuthorizedRecipient(delivery.threadId, "whatsapp", message.target.to); await sendWhatsAppMessage( { to: message.target.to, body: message.payload.body }, { key: delivery.runId }, ); }, ], [ "email", async (delivery: ChannelDestinationDelivery) => { const message = emailDeliverySchema.parse(delivery); await assertAuthorizedRecipient(delivery.threadId, "email", message.target.to); await sendEmail( { to: message.target.to, subject: message.payload.subject, body: message.payload.body, }, { key: delivery.runId }, ); }, ], ]); export const channel = composeRoutedChannel(emailSource, destinations); ``` The map is copied at construction and is the complete adapter allowlist. Unknown destination names fail the durable outbox attempt instead of silently dropping or guessing a route. Recipient and operation policy remains application-owned; `assertAuthorizedRecipient` represents a lookup against trusted workflow or tenant data, not another check of the recipient's string shape. ## 3. Let LangGraph run the workflow and choose WhatsApp LangGraph passes `config.writer` to the node. The graph declares data; the destination callback performs the external side effect only after the run settles. ```ts import { Annotation, END, interrupt, START, StateGraph, type LangGraphRunnableConfig, } from "@langchain/langgraph"; import { declareChannelDestinationDelivery } from "@skein-js/channels"; const State = Annotation.Root({ source: Annotation<"email">, refundId: Annotation, customerEmail: Annotation, orderId: Annotation, amountKes: Annotation, reason: Annotation, financeWhatsapp: Annotation, financeDecision: Annotation<"approve" | "reject" | undefined>, }); function notifyFinance(state: typeof State.State, config: LangGraphRunnableConfig) { declareChannelDestinationDelivery(config.writer, { destination: "whatsapp", target: { to: state.financeWhatsapp }, payload: { body: `Approve KES ${state.amountKes.toLocaleString()} refund for ${state.orderId}? ` + `Reason: ${state.reason}. Reply APPROVE or REJECT.`, }, }); return {}; } function awaitFinance(state: typeof State.State) { const decision = interrupt({ kind: "refund-approval", refundId: state.refundId, assignedTo: state.financeWhatsapp, }); return { financeDecision: decision as "approve" | "reject" }; } function emailCustomer(state: typeof State.State, config: LangGraphRunnableConfig) { const approved = state.financeDecision === "approve"; declareChannelDestinationDelivery(config.writer, { destination: "email", target: { to: state.customerEmail }, payload: { subject: approved ? `Refund approved for ${state.orderId}` : `Refund update for ${state.orderId}`, body: approved ? `Finance approved your KES ${state.amountKes.toLocaleString()} refund.` : "Finance could not approve this refund. Our support team will contact you.", }, }); return {}; } export const graph = new StateGraph(State) .addNode("notify-finance", notifyFinance) .addNode("await-finance", awaitFinance) .addNode("email-customer", emailCustomer) .addEdge(START, "notify-finance") .addEdge("notify-finance", "await-finance") .addEdge("await-finance", "email-customer") .addEdge("email-customer", END) .compile(); ``` On the first run, `notify-finance` supplies the durable WhatsApp delivery and `await-finance` parks the thread. The WhatsApp approval webhook needs its own verified source channel, must address the same trusted workflow thread, and resumes the interrupt: ```ts import { composeRoutedChannel, type Channel } from "@skein-js/channels"; import { z } from "zod"; import { destinations } from "./destinations.js"; import { workflowThreadId } from "./workflow-thread-id.js"; const financeReplySchema = z.object({ messageId: z.string().min(1), refundId: z.string().min(1), interruptId: z.string().min(1), from: z.string().min(1), decision: z.enum(["approve", "reject"]), }); const whatsappSource = { name: "whatsapp-source", verify(request) { if (!verifyWhatsAppWebhook(request)) return false; const message = financeReplySchema.parse(request.json()); return { identity: `channel:whatsapp:${message.from}` }; }, parseEvent(request) { const message = financeReplySchema.parse(request.json()); return { kind: "event" as const, event: { threadKey: message.refundId, threadId: workflowThreadId(message.refundId), idempotencyKey: message.messageId, input: { source: "whatsapp", ...message }, resumeWith: { [message.interruptId]: message.decision }, }, }; }, } satisfies Pick; export const channel = composeRoutedChannel(whatsappSource, destinations); ``` `workflowThreadId` must derive a tenant-scoped ID from trusted workflow data, and both sources must use the same function. Derive the Finance principal from the verified WhatsApp sender and validate that the principal is assigned to this approval before accepting the decision. The runnable example performs both checks and keeps the actor, provider event ID, and timestamp in graph state as an audit trail. Only `declareChannelDestinationDelivery` triggers a routed destination. Ordinary `replyWith` output and inferred AI replies are ignored here, preventing an accidental chat response from becoming an external action. `target` and `payload` must be JSON-persistable; the destination validates their provider-specific shape. ## 4. Configure the workflow's source routes ```jsonc { "graphs": { "relay": "./src/relay-graph.ts:graph" }, "skein": { "channels": { "email": { "path": "./src/email-channel.ts:channel", "assistant": "relay", "public_url": "https://api.example.com", }, "whatsapp": { "path": "./src/whatsapp-channel.ts:channel", "assistant": "relay", "public_url": "https://api.example.com", }, }, }, } ``` Configured route keys and explicit source names must be unique. Skein rejects those collisions at boot so a delivery alias cannot resolve to the wrong channel. That does not namespace an explicit `threadId`: this recipe intentionally shares one between email and WhatsApp. Build explicit IDs from trusted, tenant-scoped workflow identifiers; when `threadId` is omitted, Skein safely namespaces the derived ID by channel name instead. The complete workflow is: ```text Customer email → LangGraph → Finance WhatsApp → interrupt/resume → LangGraph → customer email ``` For conditional email/WhatsApp routing plus parallel LangGraph `interrupt()` approvals, run [`examples/decoupled-delivery`](https://github.com/skein-js/skein-js/tree/main/examples/decoupled-delivery). --- # Memory Remembering things across threads and sessions. ## Long-term memory (`getStore()`) The store is injected into every run as a LangGraph `BaseStore`, so a node reads and writes it the native way — with memory for the default `skein dev`, SQLite for local apps, or Postgres for a shared server. The graph code stays the same. ```ts import { getStore } from "@langchain/langgraph"; async function remember(state, config) { const store = getStore(); const userId = config.configurable.langgraph_auth_user_id ?? "anon"; await store.put(["memories", userId], "prefs", { units: "metric" }); const hits = await store.search(["memories", userId], { query: "units" }); return { known: hits.map((h) => h.value) }; } ``` The same items are reachable over the `/store/items` HTTP endpoints. Working versions: [`chat-app`](https://github.com/skein-js/skein-js/tree/main/examples/chat-app) (recalls a user across sessions) and [`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) (reads your conventions back into the prompt), and [`deep-agent`](https://github.com/skein-js/skein-js/tree/main/examples/deep-agent) (a Deep Agent's `/memories/` files, one namespace per user). > [!WARNING] > **Read this before writing a dedup rule** > > Memory, SQLite, and Postgres **without** `store.index` return `score: 1` for every text hit — so > the obvious "score >= 0.9 means duplicate" rule classifies everything as a duplicate and the agent > **silently stops recording memories**. It does not error. [memory.md](../memory.md) covers this and the > other shapes that bite. ## Rank by meaning, not substring On Postgres, configure an embedder and `store.search({ query })` uses pgvector. In-memory falls back to a naive scan, so dev behaviour matches. ```jsonc // langgraph.json { "store": { "index": { "embed": "openai:text-embedding-3-small", "dims": 1536, "fields": ["$"] } }, } ``` `embed` takes a `provider:model` string or a function path; `dims` is required with it. ## Expire what you don't need `store.ttl` (minutes) expires items on a background sweep, with `refresh_on_read` to keep active ones alive. Threads have their own `checkpointer.ttl`. Both in [storage.md](../storage.md#store-item-ttl). ## Bring your own store Long-term memory is the one repo you can swap without implementing the other five — point `store.adapter` at a LangGraph `BaseStore` (including `PostgresStore`, which brings hybrid text+vector search skein's own driver lacks) or a skein `StoreRepo`. Details: [storage.md](../storage.md#bringing-your-own-store-storeadapter). --- # Deep Agents > [!WARNING] > **Experimental.** Deep Agents support is tested end to end against `deepagents` 1.15, but Deep > Agents itself moves quickly and this integration may change in a minor release. See > [what is covered](#what-is-covered) before relying on it in production. Serving a LangChain [Deep Agent](https://docs.langchain.com/oss/javascript/deepagents/overview) — planning, files, subagents, human approval — with skein. `createDeepAgent()` returns a compiled LangGraph graph, so there is nothing to adapt: export it, name it in `langgraph.json`, and skein supplies the checkpointer, the store, streaming and interrupt/resume around it. There is no `@skein-js/deepagents` package; depend on `deepagents` directly. ```ts // src/agent.ts import { CompositeBackend, createDeepAgent, StateBackend, StoreBackend, type StoreBackendContext, } from "deepagents"; function memoryNamespace({ config }: StoreBackendContext): string[] { const userId = config?.configurable?.langgraph_auth_user_id; if (typeof userId !== "string" || userId.length === 0) { throw new Error("Durable memory requires an authenticated user."); } const component = userId.replace( /[^A-Za-z0-9\-_.@+:]/g, (character) => `~${character.codePointAt(0)!.toString(16)}~`, ); return ["memories", component]; } export const graph = createDeepAgent({ model: "google-genai:gemini-2.5-flash", systemPrompt: "You are a careful research assistant.", backend: new CompositeBackend(new StateBackend(), { "/memories/": new StoreBackend({ namespace: memoryNamespace, }), }), subagents: [ { name: "researcher", description: "Answers one question.", systemPrompt: "Be brief." }, ], }); ``` ```json { "graphs": { "agent": "./src/agent.ts:graph" }, "auth": { "path": "./src/auth.ts:auth" } } ``` `skein dev` serves it, with the same `langgraph.json` `langgraph dev` reads. A working version is [`examples/deep-agent`](https://github.com/skein-js/skein-js/tree/main/examples/deep-agent): a research briefing assistant that plans, delegates research to a subagent, drafts briefings, remembers each user's preferences, asks before emailing, and runs deep dives in the background. It runs offline with a scripted model, so it needs no API key. ## What is covered Tested end to end in the example, through the official `@langchain/langgraph-sdk` client: - a `createDeepAgent()` export loaded from an unchanged `langgraph.json`, including as a graph factory; - planning with `write_todos`, and files in thread state that persist across a thread's turns; - `StoreBackend` files shared across a user's threads and isolated between users; - a synchronous subagent streamed under its own namespace with `stream_subgraphs`; - approve / edit / reject on an `interruptOn` tool; - an async subagent started, checked, updated and cancelled over HTTP on the same server. Not covered, or deliberately out of scope: - **The URL-less async transport.** LangGraph Platform runs co-deployed async subagents in process; skein needs an explicit `url`. - **Subgraph streaming on `/invoke`.** Use the run endpoints. - **Python Deep Agents.** skein loads TypeScript graphs; a Python agent can only be reached as a separate Agent Protocol service. - **Version range.** Tested with `deepagents` 1.15 and `@langchain/langgraph` 1.4.10 or later. Pin `deepagents` in your application. ## Files: thread state by default, the store for what outlives a thread - **`StateBackend`** keeps files in the thread's state. They survive every turn of that thread and vanish with it. Deep Agents also writes its own scratch (offloaded tool results, history) to the default backend, so state is the right default. - **`StoreBackend`** keeps files in skein's store — memory under `skein dev`, SQLite or Postgres in production — so they are shared across threads. Mount it on a prefix with `CompositeBackend`. **Give `StoreBackend` an explicit `namespace` on a shared server.** Its default uses assistant identity when available and otherwise a shared `filesystem` namespace; neither isolates users. Build it from `langgraph_auth_user_id`: skein stamps that from your [`auth`](./authentication.md) handler and strips it from anything a client sends, so it cannot be spoofed. With no `auth` configured it is absent — fail rather than pool everyone together. Namespace components accept only letters, digits and `-_.@+:~`, so encode identities that may contain anything else. [Memory](./memory.md) covers the store itself. Secure the HTTP store routes separately: graph backend namespacing does not authorize `/store/items`. The example denies HTTP store access with `.on("store", () => false)` while retaining the graph's injected store. Applications needing that HTTP API must scope every store action through `auth`. Legacy `assistant_id` and `graph_id` configurable values remain application-owned; do not use them as authenticated identity. ## Streaming subagents A synchronous subagent runs inside the `task` tool. Ask for subgraph frames and its progress streams alongside the main agent's, under the tool call's namespace: ```ts for await (const chunk of client.runs.stream(threadId, "agent", { input, streamMode: ["updates", "messages-tuple"], streamSubgraphs: true, })) { chunk.event; // "updates" from the main agent, "updates|tools:" from the subagent } ``` The SDK's UI helpers route on these names. See [subgraphs and subagents](../streaming.md#subgraphs-and-subagents). ## Planning `write_todos` comes from LangChain's todo-list middleware. Deep Agents adds it only for some model profiles, so pass it yourself when you rely on planning: ```ts import { todoListMiddleware } from "langchain"; createDeepAgent({ /* … */ middleware: [todoListMiddleware()] }); ``` ## Human approval `interruptOn` pauses before a tool runs; the run ends `interrupted`. Resume it with a decision per pending action: ```ts createDeepAgent({ /* … */ interruptOn: { send_briefing: { allowedDecisions: ["approve", "edit", "reject"] } }, }); await client.runs.wait(threadId, "agent", { command: { resume: { decisions: [{ type: "approve" }] } }, // or { type: "edit", editedAction: { name: "send_briefing", args: { … } } } // or { type: "reject", message: "not this week" } }); ``` See [human-in-the-loop](../human-in-the-loop.md). ## Async subagents An async subagent is another graph, started and polled over the Agent Protocol. Give it an explicit `url` and credentials scoped to the originating caller. Build the main agent as a graph factory so upstream's async client is constructed with that caller's credentials. For the example's demo auth: ```ts import type { RunnableConfig } from "@langchain/core/runnables"; import { createDeepAgent } from "deepagents"; export function graph(config: RunnableConfig) { const userId = config.configurable?.langgraph_auth_user_id; // An empty identity is rejected by the demo auth handler, including on an unbound factory. const headers = { "x-user": typeof userId === "string" ? userId : "" }; return createDeepAgent({ subagents: [ { name: "deep-researcher", description: "Runs a longer research investigation in the background.", graphId: "researcher", url: process.env.SKEIN_URL ?? "http://127.0.0.1:2024", headers, }, ], }); } ``` In production, mint or forward a caller-scoped credential your auth handler verifies. A shared service credential would give user-supplied `asyncTasks` records authority over other callers' child runs. Keep receiving thread ownership checks in place; task identifiers are not authorization. Every call is an ordinary route — thread create, run create, run get, thread state, a new run with `multitask_strategy: "interrupt"` to update, and cancel — so it works against any skein deployment. The URL-less, in-process transport LangGraph Platform offers is not recreated. Each started task is a background run, so give the server enough [run concurrency](../runs-and-redis.md#run-concurrency) for the main agent and its children at once. ## Shell and code execution Use a remote sandbox provider for anything that executes commands. **Never use `LocalShellBackend` or `FilesystemBackend` on a shared server**: both act directly on the host skein runs on, with that process's permissions, for whoever can reach the agent. State and store backends need no sandbox. ## Dependency security This workspace applies a local patch for the unpatched `braces` nesting advisory in Deep Agents' glob dependency. Applications copying this example must audit their own lockfiles and apply the patch in their own package-manager configuration; workspace patches do not propagate through a published package or an exported artifact. See the [dependency audit](../audits/dependency-advisories-2026-10-20.md) for the patch, compatibility details, and current scan results. --- # Authenticating requests Keep the login flow you already have. Skein only needs the server-side step that turns an incoming request into a stable principal. Its `auth.path` uses LangGraph's `Auth` class, so the adapter is one function: ```text cookie or bearer token → your provider's server SDK → { identity, permissions } ``` The examples below were checked against the providers' official server documentation on 2026-09-17 (Better Auth 1.7, Auth.js `next-auth` 5.0 beta 32, `@clerk/backend` 3.16, `@supabase/supabase-js` 2.116, `firebase-admin` 14.3, and `jose` 6.2). Provider APIs move independently of Skein, so follow the linked provider documentation if your installed major differs. | If your app already uses… | Start here | | ----------------------------------------------- | ------------------------------------------------------------------ | | Better Auth sessions | [Better Auth](#better-auth) — pass Skein's request headers through | | Auth.js / NextAuth sessions | [Auth.js](#authjs--nextauth) — wrap your session reader | | Clerk users and organizations | [Clerk](#clerk) — authenticate the whole request | | Supabase Auth | [Supabase](#supabase-auth) — verify its access token | | Firebase Auth | [Firebase](#firebase-auth) — verify its ID token | | Auth0, Okta, Cognito, WorkOS, or another issuer | [OIDC/JWT](#auth0-okta-cognito-workos-or-another-oidc-issuer) | Nothing here replaces your sign-in page, callbacks, or session storage. Those stay with the provider you already chose; this page is only the bridge at the API boundary. ## Wire the provider into Skein Choose one `authenticate-request.ts` implementation below, then apply your authorization policy in `auth.ts`. The Better Auth section shows a co-located version instead, because its adapter must reuse the exact Better Auth instance your application already exports. ```ts // auth.ts import { Auth } from "@langchain/langgraph-sdk/auth"; import { authenticateRequest } from "./authenticate-request.js"; const tenantLabel = (identity: string) => encodeURIComponent(identity).replace(/\./g, "%2E").replace(/\*/g, "%2A"); export const auth = new Auth() .authenticate(authenticateRequest) // A filter hides other owners' rows on reads and stamps ownership onto writes. Runs authorize // through their thread, and crons fall back to this handler when no crons handler is registered. .on("threads", ({ user }) => ({ owner: user.identity })) // Store items have no metadata to filter. Root their namespace instead. .on("store", ({ user, value }) => { value.namespace = [tenantLabel(user.identity), ...(value.namespace ?? []).slice(1)]; }); ``` Point `langgraph.json` at that export: ```jsonc { "auth": { "path": "./src/auth.ts:auth", "disable_studio_auth": true }, } ``` The CLI requires credentials for Studio even when `disable_studio_auth` is `false`: a caller can forge Studio's identifying header. For local development, `skein dev --dev-ui-access` or `auth.dev_ui_access: true` in `langgraph.json` prints a temporary API key accepted by both Studio and the console on a loopback server. Authentication answers _who is calling_; the `.on(...)` handlers answer _what they may access_. Keep organization membership, roles, and other application policy in those handlers rather than in Skein config. Bearer-token providers can share this small helper: ```ts // auth-helpers.ts import { HTTPException } from "@langchain/langgraph-sdk/auth"; export function requireBearerToken(request: Request): string { const match = request.headers.get("authorization")?.match(/^Bearer\s+(.+)$/i); if (!match?.[1]) throw new HTTPException(401, { message: "Missing bearer token" }); return match[1]; } ``` ## Better Auth Better Auth's server API accepts the same `Headers` object Skein gives the authentication callback. This works with its normal session cookie; it also accepts `Authorization: Bearer ...` when you have enabled Better Auth's [Bearer plugin](https://better-auth.com/docs/plugins/bearer). ```ts // src/auth.ts — add `skeinAuth` beside the Better Auth instance your app already uses import { Auth as LangGraphAuth, HTTPException } from "@langchain/langgraph-sdk/auth"; import { betterAuth } from "better-auth"; export const auth = betterAuth({ // Keep your existing database, providers, plugins, and session options here. }); const tenantLabel = (identity: string) => encodeURIComponent(identity).replace(/\./g, "%2E").replace(/\*/g, "%2A"); export const skeinAuth = new LangGraphAuth() .authenticate(async (request) => { const session = await auth.api.getSession({ headers: request.headers }); if (!session) throw new HTTPException(401, { message: "Unauthorized" }); return { identity: session.user.id, display_name: session.user.name, email: session.user.email, permissions: [], }; }) .on("threads", ({ user }) => ({ owner: user.identity })) .on("store", ({ user, value }) => { value.namespace = [tenantLabel(user.identity), ...(value.namespace ?? []).slice(1)]; }); ``` This reuses your existing Better Auth instance and session store; do not create a second auth database for Skein. Let Better Auth's own client and endpoints continue to handle login, logout, and cookie refresh—Skein only reads the session presented on each request. See Better Auth's [server-side session API](https://better-auth.com/docs/basic-usage#get-session). For this variant, point `langgraph.json` at `"./src/auth.ts:skeinAuth"` so it does not confuse the Better Auth instance with LangGraph's `Auth` instance. ## Clerk Use Clerk's request-level verifier rather than decoding the session JWT yourself. Set `authorizedParties` to the origins that are allowed to send Clerk credentials to this server. ```ts // authenticate-request.ts import { createClerkClient } from "@clerk/backend"; import { HTTPException } from "@langchain/langgraph-sdk/auth"; const clerk = createClerkClient({ secretKey: process.env.CLERK_SECRET_KEY, publishableKey: process.env.CLERK_PUBLISHABLE_KEY, }); export async function authenticateRequest(request: Request) { const requestState = await clerk.authenticateRequest(request, { authorizedParties: [process.env.APP_ORIGIN!], }); if (!requestState.isAuthenticated) { throw new HTTPException(401, { message: "Unauthorized" }); } const principal = requestState.toAuth(); if (!principal.userId) throw new HTTPException(401, { message: "Unauthorized" }); return { identity: principal.userId, org_id: principal.orgId, permissions: principal.orgPermissions ?? [], }; } ``` Clerk documents the full [`authenticateRequest()` contract](https://clerk.com/docs/reference/backend/authenticate-request), including networkless verification with `CLERK_JWT_KEY` and separate machine-token modes. ## Supabase Auth For an API server, accept the user's Supabase access token as a bearer token and verify its claims. `getClaims(token)` uses the project's cached JWKS when possible; unlike `getSession()`, it does not trust client-side session storage. ```ts // authenticate-request.ts import { HTTPException } from "@langchain/langgraph-sdk/auth"; import { createClient } from "@supabase/supabase-js"; import { requireBearerToken } from "./auth-helpers.js"; const supabase = createClient(process.env.SUPABASE_URL!, process.env.SUPABASE_PUBLISHABLE_KEY!); export async function authenticateRequest(request: Request) { const { data, error } = await supabase.auth.getClaims(requireBearerToken(request)); if (error || !data || typeof data.claims.sub !== "string") { throw new HTTPException(401, { message: "Unauthorized" }); } const scope = data.claims.scope; return { identity: data.claims.sub, permissions: typeof scope === "string" ? scope.split(" ").filter(Boolean) : [], }; } ``` See Supabase's [`getClaims()` reference](https://supabase.com/docs/reference/javascript/auth-getclaims). If your project still uses an HS256 signing secret, Supabase may call its Auth server to verify each token; do not decode the JWT without verifying it. ## Firebase Auth The browser obtains a Firebase **ID token** after sign-in and sends it as a bearer token. Verify that ID token with the Admin SDK; do not send or accept a Firebase custom token here. ```ts // authenticate-request.ts import { applicationDefault, getApps, initializeApp } from "firebase-admin/app"; import { getAuth } from "firebase-admin/auth"; import { HTTPException } from "@langchain/langgraph-sdk/auth"; import { requireBearerToken } from "./auth-helpers.js"; const firebaseApp = getApps()[0] ?? initializeApp({ credential: applicationDefault() }); const firebaseAuth = getAuth(firebaseApp); export async function authenticateRequest(request: Request) { try { const token = await firebaseAuth.verifyIdToken(requireBearerToken(request)); return { identity: token.uid, permissions: [] }; } catch { throw new HTTPException(401, { message: "Unauthorized" }); } } ``` Firebase's [ID-token verification guide](https://firebase.google.com/docs/auth/admin/verify-id-tokens) explains service-account setup and the optional revocation check. Map only custom claims you created for authorization; do not turn every JWT claim into a permission. ## Auth0, Okta, Cognito, WorkOS, or another OIDC issuer For a standards-based issuer, verify the access-token signature, issuer, audience, and expiry from its JWKS. The `jose` package works across Node, Bun, and Deno: ```ts // authenticate-request.ts import { HTTPException } from "@langchain/langgraph-sdk/auth"; import { createRemoteJWKSet, errors, jwtVerify } from "jose"; import { requireBearerToken } from "./auth-helpers.js"; const issuer = process.env.OIDC_ISSUER!; // exact `iss` const audience = process.env.OIDC_AUDIENCE!; // this Skein API's identifier const issuerBase = issuer.endsWith("/") ? issuer : `${issuer}/`; const jwks = createRemoteJWKSet(new URL(".well-known/jwks.json", issuerBase)); export async function authenticateRequest(request: Request) { try { const { payload } = await jwtVerify(requireBearerToken(request), jwks, { issuer, audience, }); if (!payload.sub) throw new HTTPException(401, { message: "Unauthorized" }); return { identity: payload.sub, permissions: typeof payload.scope === "string" ? payload.scope.split(" ").filter(Boolean) : [], }; } catch (error) { if (error instanceof HTTPException) throw error; if (error instanceof errors.JOSEError) { throw new HTTPException(401, { message: "Unauthorized" }); } throw error; } } ``` Use an **access token whose audience is this API**, not an ID token meant for a browser client. Auth0, for example, makes that distinction explicit in its [token guide](https://auth0.com/docs/secure/tokens). Some issuers publish a discovery document at `/.well-known/openid-configuration`; use its `jwks_uri` when it differs from the conventional path above. ## Authorization: decide what callers can access Once a provider has proved who the caller is, stop thinking about Clerk, Firebase, or JWTs. Every provider now reaches the same LangGraph authorization handlers with a normalized `user`: - `user.identity` is the stable user ID. - `user.permissions` is the trusted permission list you returned during authentication. - Extra verified fields such as Clerk's `org_id` remain available on `user`. An authorization handler has three useful answers: | Return value | Meaning | | ---------------------------- | ----------------------------------------------------------------------- | | `false` | Deny with `403` | | `true`, `null`, or no return | Allow without an ownership filter | | `{ owner: user.identity }` | Allow and scope the resource; the filter also stamps newly created rows | | Rewrite `value.namespace` | Scope long-term store access | Handlers match from most specific to broadest: `threads:delete` → `threads` → `*:delete` → `*`. That lets one sensitive action be stricter without repeating the policy for every route. ### A practical per-user policy This is a good default for an app where every person owns their own conversations: ```ts export const auth = new Auth() .authenticate(authenticateRequest) .on("threads:delete", ({ user }) => user.permissions.includes("threads:delete") ? { owner: user.identity } : false, ) .on("threads", ({ user }) => user.permissions.includes("skein:admin") ? true : { owner: user.identity }, ) .on("assistants", ({ user }) => user.permissions.includes("assistants:read")) .on("store", ({ user, value }) => { value.namespace = [tenantLabel(user.identity), ...(value.namespace ?? []).slice(1)]; }); ``` Here an administrator can see all threads, ordinary users see only their own, deletion needs an additional permission, and assistant discovery is explicitly gated. Returning an ownership filter for `threads:delete` is important: returning only `true` would let any caller with that permission delete any user's thread. Thread policy automatically covers runs because runs belong to a thread. Crons have their own resource, but fall back to the `threads` handler when you do not register a `crons` handler. Add an explicit handler when scheduling is more privileged than chatting: ```ts auth.on("crons", ({ user }) => user.permissions.includes("crons:manage") ? { owner: user.identity } : false, ); ``` ### Share data inside an organization If threads belong to a Clerk organization, Auth0 organization, or your own workspace rather than one person, authenticate the organization ID as a trusted field and filter on it consistently: ```ts const organizationId = (user: { org_id?: unknown }) => typeof user.org_id === "string" ? user.org_id : undefined; auth .on("threads", ({ user }) => { const orgId = organizationId(user); return orgId ? { org_id: orgId } : false; }) .on("store", ({ user, value }) => { const orgId = organizationId(user); if (!orgId) throw new HTTPException(403, { message: "Choose an organization" }); value.namespace = [tenantLabel(orgId), ...(value.namespace ?? []).slice(1)]; }); ``` Do not accept `org_id`, `owner`, or permissions from request metadata or graph input. They must come from provider-verified claims. Decide explicitly what a user with no active organization should do; the example denies access instead of silently falling back to a personal tenant. ### Know which resources can be filtered | Resource | What authorization can do | | ------------ | --------------------------------------------------------------------------------------- | | `threads` | Deny or return metadata filters; the same policy protects their runs | | `crons` | Deny or return metadata filters; falls back to `threads` when no cron handler exists | | `assistants` | Gate access only; graph-backed assistants are shared, so ownership filters do not apply | | `store` | Rewrite `value.namespace`; returning a metadata filter does not isolate store items | The last row is the easy one to miss: authentication alone does **not** make long-term memory multi-tenant. Keep the namespace rewrite even if every other resource is owner-filtered. Calls to `getStore()` from inside a graph do not pass through HTTP authorization, so build their namespace from `config.configurable.langgraph_auth_user_id` (or another server-injected verified field), never from model output. ## Auth.js / NextAuth Auth.js already knows how to verify its own session. You only need a small wrapper that accepts two functions: - `readVerifiedSession(request)` returns your provider's verified session, or `null`. - `toPrincipal(session)` returns the user shape Skein needs. `identity` must be a stable user ID; everything else is application-defined context for authorization handlers. The wrapper returns exactly the function accepted by `.authenticate(...)`: `(request: Request) => Promise`. ```ts import { HTTPException } from "@langchain/langgraph-sdk/auth"; type SkeinPrincipal = { identity: string; permissions: string[]; [attribute: string]: unknown; }; type ReadVerifiedSession = (request: Request) => Promise; type ToPrincipal = (session: TSession) => SkeinPrincipal; function createSessionAuthenticator( readVerifiedSession: ReadVerifiedSession, toPrincipal: ToPrincipal, ): (request: Request) => Promise { return async (request) => { const session = await readVerifiedSession(request); if (!session) throw new HTTPException(401, { message: "Unauthorized" }); const principal = toPrincipal(session); if (!principal.identity) throw new HTTPException(401, { message: "Unauthorized" }); return principal; }; } ``` For example, in the same module where a Next.js app already configures Auth.js, use the exported `auth()` session reader and map your session fields: ```ts import { Auth as LangGraphAuth } from "@langchain/langgraph-sdk/auth"; import NextAuth, { type DefaultSession } from "next-auth"; import GitHub from "next-auth/providers/github"; declare module "next-auth" { interface Session { user: { id: string; permissions?: string[]; } & DefaultSession["user"]; } } const { auth: readAuthJsSession, handlers } = NextAuth({ providers: [GitHub], callbacks: { session({ session, token }) { // This example uses Auth.js's JWT session strategy. For database sessions, use `user.id`. if (!token.sub) throw new Error("Auth.js token is missing a user ID"); session.user.id = token.sub; return session; }, }, }); export { handlers }; export const skeinAuth = new LangGraphAuth() .authenticate( createSessionAuthenticator( async () => readAuthJsSession(), (session) => ({ identity: session.user.id, display_name: session.user.name, email: session.user.email, permissions: session.user.permissions ?? [], }), ), ) .on("threads", ({ user }) => ({ owner: user.identity })); ``` This is intentionally code you own: change only the session reader and field mapping when your framework or session shape differs. Auth.js documents how to expose a stable user ID for [JWT and database sessions](https://authjs.dev/guides/extending-the-session) and how to keep custom session fields type-safe with [module augmentation](https://authjs.dev/getting-started/typescript). If Skein runs as a separate server, the same wrapper still applies, but `readVerifiedSession(request)` should call a trusted session-validation endpoint. Alternatively, exchange the application session for a short-lived signed access token and use the [OIDC/JWT recipe](#auth0-okta-cognito-workos-or-another-oidc-issuer). Do not feed an Auth.js database-session cookie to a JWT decoder: that cookie is a lookup key, not a JWT. ## Browser cookies and CORS Same-origin cookies need no CORS configuration. Across origins, all three pieces must agree: 1. The browser request includes credentials. 2. The auth provider issued a cookie valid for the Skein origin/domain and appropriate `SameSite` and `Secure` attributes. 3. Skein allows the exact frontend origin and credentials—never `*` with credentials: ```jsonc { "http": { "cors": { "allow_origins": ["https://app.example.com"], "allow_credentials": true, }, }, } ``` Bearer tokens do not require credentialed cookies, but the browser still needs the `authorization` header allowed by its CORS preflight. Prefer same-origin mounting when your framework already hosts the frontend. ## What Skein does after authentication The returned object becomes `user` in every `.on(...)` callback. Skein also stamps it into graph run config as `langgraph_auth_user`, `langgraph_auth_user_id`, and `langgraph_auth_permissions`; clients cannot spoof those keys. Thread filters apply to runs as well. Long-term store access is separate and must be rooted by namespace, as in the shared policy above. For the full request lifecycle, route-to-permission map, and store-scoping traps, see [Authentication + authorization](../agent-protocol.md#authentication--authorization). --- # Production Locking it down, being told when work finishes, and running it durably. ## Custom auth The server is open by default. skein implements [LangGraph's custom-auth model](https://docs.langchain.com/langsmith/custom-auth), so an existing `Auth` file is drop-in. Using Better Auth, Clerk, Supabase, Firebase, Auth0, or another OIDC provider? Start with the [authentication provider recipes](./authentication.md). ```ts // auth.ts import { Auth, HTTPException } from "@langchain/langgraph-sdk/auth"; export const auth = new Auth() .authenticate(async (request) => { const token = request.headers.get("authorization")?.replace(/^Bearer /, ""); const user = token ? await verify(token) : undefined; if (!user) throw new HTTPException(401, { message: "Unauthorized" }); return { identity: user.id, permissions: user.scopes }; }) .on("threads", ({ user }) => ({ owner: user.identity })); // scopes threads and their runs ``` Wire it with `"auth": { "path": "./auth.ts:auth" }` in `langgraph.json`, or in code as `embedInMemoryGraphs(graphs, { auth })` — note the second argument **is** the overrides bag here, while `embedPostgresGraphs` nests them under `overrides`. A returned filter both hides other owners' rows and stamps ownership onto new ones. **Inside a graph**, the principal arrives on the run config as `config.configurable.langgraph_auth_user`, `…_user_id` and `…_permissions` — server-owned and unspoofable, present only when auth is configured. > [!WARNING] > **The store is not scoped by an ownership filter** > > A store item carries no metadata to filter on, so **an authenticated caller can read every tenant's > items** unless an `@auth.on.store` handler narrows it. Scope it by rewriting `value.namespace` — the > pattern and its traps are in [agent-protocol.md](../agent-protocol.md#scoping-the-store). Full route → permission map: [agent-protocol.md](../agent-protocol.md#authentication--authorization). ## Get notified when a run finishes Pass a `webhook` URL on run creation; skein POSTs the settled run to it. ```ts await client.runs.create(threadId, "agent", { input, webhook: "https://example.com/hooks/run" }); ``` **The callback is recorded in the same transaction as the run's terminal status**, so a crash between "the run finished" and "someone was told" cannot lose it. skein attempts the first delivery inline, so a healthy receiver hears within milliseconds; a failure is recorded and retried — for about 34 minutes by default, which rides out a rolling deploy. **It is at-least-once, not exactly-once.** A retry can duplicate a callback your receiver already processed — after a network timeout that actually succeeded, say. Every attempt carries the same `X-Skein-Delivery-Id`; dedupe on it. The API remains the source of truth for run state. Set `SKEIN_WEBHOOK_SECRET` and every callback is signed, so a receiver can tell a real one from anybody who guessed the URL: ```ts import { verifySkeinSignature } from "@skein-js/agent-protocol"; const result = verifySkeinSignature({ header: req.headers["x-skein-signature"], body: rawBody, // the RAW bytes, not JSON.stringify(req.body) secrets: [process.env.SKEIN_WEBHOOK_SECRET!], }); if (!result.ok) return res.status(401).end(); ``` Working receiver: `src/webhook-receiver.ts` in [`examples/express-basic`](https://github.com/skein-js/skein-js/tree/main/examples/express-basic) (`pnpm webhook-receiver`) — accepts a genuine callback, refuses a forged and a replayed one. **Persist the delivery outbox** with Postgres or SQLite. The delivery worker polls pending records after restart; Redis supplies scheduled delivery when configured, but is not required for recovery. Details — the retry tiers, signature verification and key rotation, listing and replaying failed deliveries, and what gets stored and for how long: [webhooks.md](../webhooks.md). ## Go durable and scale out Back skein with Postgres (state + checkpoints) and Redis (queue + cross-instance streaming). Your server code doesn't change — only how `deps` is built. ```ts import { buildRuntime } from "@skein-js/runtime"; const rt = await buildRuntime({ configPath: "./langgraph.json", store: "postgres", queue: "redis", }); // pass { deps: rt.deps, channels: rt.channels } to an adapter; call rt.dispose() on shutdown ``` Or use the CLI: `skein build` produces a deployable image, `skein up` brings up app + Postgres + Redis via Compose, and `skein dev --store postgres --queue redis` runs the durable stack locally. Redis is optional for one instance and **required for more than one**. ## Deploy it For a local app or one Node owner, SQLite is another durable option. It needs no Postgres/Redis services and uses the same protocol; the host owns lifetime and recovery decisions. Follow [local deployment](../local-deployment.md) for CLI selection or `embedSqliteGraphs`. That image runs anywhere you can run a container. [deploy.md](../deploy.md) covers what every platform needs — Postgres, Redis, the port, the `/ok` probe, pool sizing, SIGTERM draining, SSE through proxies — with guides for [Cloud Run](../deploy-cloud-run.md), [Railway](../deploy-railway.md), [Fly.io](../deploy-fly.md), [Render](../deploy-render.md), [AWS](../deploy-aws.md), [Kubernetes](../deploy-kubernetes.md), [a VPS](../deploy-vps.md), and [what doesn't work on serverless](../deploy-serverless.md). ## Watch it run The [console](../console.md) at `/console` shows threads, live run tails, interrupt approvals, time travel, the store browser and crons — served by your own server, no account and no tunnel. It is **off by default** in production; opt in with `{"http": {"console": true}}`. For tracing and metrics, [observability.md](../observability.md) covers the `TelemetrySink` seam and the LangSmith, PostHog and OpenTelemetry adapters. --- # Building your own adapter skein-js ships adapters for [Express](https://github.com/skein-js/skein-js/tree/main/packages/server-express), [Fastify](https://github.com/skein-js/skein-js/tree/main/packages/server-fastify), [NestJS](https://github.com/skein-js/skein-js/tree/main/packages/server-nestjs), and [Next.js](https://github.com/skein-js/skein-js/tree/main/packages/server-nextjs) (App + Pages Router). If your stack isn't one of those — a raw Node `http` server, [Hono](https://hono.dev), Koa, an existing app on some other framework — you can write your own adapter in a few dozen lines. This guide shows how; the four shipped adapters are all built exactly this way. ## Why this is easy All of skein-js's protocol logic lives in [`@skein-js/agent-protocol`](https://github.com/skein-js/skein-js/tree/main/packages/agent-protocol) behind a **transport-neutral handler table**. An adapter adds _no protocol logic_ — it only does shape translation: turn your framework's request into a normalized `ProtocolRequest`, call the right handler, and write the returned `ProtocolResponse` back out. The shipped [Express adapter](https://github.com/skein-js/skein-js/tree/main/packages/server-express) is exactly this and nothing more; yours will mirror it. ```text your framework request ──▶ ProtocolRequest ──▶ handler ──▶ ProtocolResponse ──▶ your framework response (Step 3) (Step 2) (Step 4) ``` ## The contract Three types from `@skein-js/agent-protocol` are all you touch: ```ts interface ProtocolRequest { method: string; // "POST" url: string; // absolute URL (path + query) — an auth handler may read it params: Record; // path params, e.g. { thread_id } query: Record; body: unknown; // parsed JSON body headers: Record; // lowercased names } type ProtocolResponse = | { kind: "json"; status: number; body: unknown; headers?: Record } | { kind: "empty"; status: number; headers?: Record } | { kind: "sse"; status: number; events: AsyncIterable; headers?: Record; }; type ProtocolHandler = (req: ProtocolRequest) => Promise; ``` `ProtocolHandlers` is a table of named handlers (`createThread`, `createStreamRun`, `joinRunStream`, `putStoreItem`, …). Each validates the request (with Zod), calls the typed service, and returns a `ProtocolResponse`. You dispatch to them by name. For a worked example, `@skein-js/fetch` is the shortest one to read: a single file over WHATWG `Request`/`Response`, so it shows the whole contract — request mapping, the headers channel, a pull-driven SSE body, and a bounded request read — without a framework's conventions in the way. ## Step 1 — assemble a runtime Build a `ProtocolRuntime` from a `ProtocolDeps` (the injected storage/queue/graph bundle). The easiest way to get production `deps` from a `langgraph.json` is [`@skein-js/runtime`](https://github.com/skein-js/skein-js/tree/main/packages/runtime)'s `buildRuntime`: ```ts import { buildRuntime } from "@skein-js/runtime"; import { resolveProtocolRuntime } from "@skein-js/server-kit"; const assembled = await buildRuntime({ configPath: "./langgraph.json", store: "memory", // or "postgres" queue: "memory", // or "redis" }); const resolved = await resolveProtocolRuntime({ deps: assembled.deps, channels: assembled.channels, }); // resolved.runtime is running; resolved.routes includes optional channel routes. ``` > You can also construct `deps` by hand (your own `SkeinStore`, `RunQueue`, `RunEventBus`, and a > `GraphResolver`) — see [`@skein-js/core`](https://github.com/skein-js/skein-js/tree/main/packages/core) for the interfaces and > [storage.md](./storage.md) / [runs-and-redis.md](./runs-and-redis.md) for the drivers. ## Step 2 — the route table The paths mirror the `@langchain/langgraph-sdk` client exactly (that's the conformance oracle — don't invent your own spelling). Bind each `method + path` to a handler name. The canonical table is exported from `@skein-js/agent-protocol` as `skeinRoutes` (re-exported from `@skein-js/express` too, for back-compat). Mount `resolved.routes` when using `resolveProtocolRuntime`: it starts with that canonical table and appends configured channel routes without changing the core protocol surface. ```ts import { skeinRoutes, copyThreadIdIntoBody, matchSkeinRoute } from "@skein-js/agent-protocol"; // skeinRoutes: { method, path, handler, foldThreadIdIntoBody? }[] // resolved.routes: the same table plus deployment-specific channel bindings // e.g. { method: "post", path: "/threads/:thread_id/runs/stream", // handler: "createStreamRun", foldThreadIdIntoBody: true } // // copyThreadIdIntoBody(request) — the body-fold rule for foldThreadIdIntoBody routes // matchSkeinRoute(method, pathname) — match a catch-all path → { binding, params } // (handy for adapters that dispatch from one route, like the NestJS + Next.js adapters) ``` Three things to get right: - **Order most-specific first** within each method so literals win over params (e.g. `/threads/search` before `/threads/:thread_id`). - **`foldThreadIdIntoBody`** — the SDK addresses a thread-scoped run by its path (`POST /threads/{id}/runs/stream`) but the stateless run handlers read `thread_id` from the body. For those routes, copy the path `thread_id` into the body before dispatch (see the worked example). - **Strip your mount prefix before matching.** `skeinRoutes` paths are anchored at the protocol root, so if your adapter mounts a **catch-all** and matches by hand, the framework hands it the full external path (`/api/threads`) which will never match `^/threads$`. Use `stripBasePath` from `@skein-js/server-kit`, and treat its `null` as "not ours" — pass the request through untouched so the host app's own routes still resolve: ```ts import { stripBasePath } from "@skein-js/server-kit"; const pathname = stripBasePath(url.pathname, mountPrefix); if (pathname === null) return next(); // not under our mount — the host app's problem const match = matchSkeinRoute(method, pathname); ``` Adapters that mount each route **explicitly** (Express's `Router`, Fastify's plugin `prefix`) get this from their router for free and can skip it. Where `mountPrefix` comes from is framework-specific: the NestJS adapter reads Nest's own `app.setGlobalPrefix(...)` via `ApplicationConfig`, while the Next.js adapters take an explicit `basePath` option. ## Step 3 — map your request onto `ProtocolRequest` Pure shape translation. Header names must be **lowercased** (handlers look up `last-event-id`), and array-valued headers flattened to a single value: ```ts function toProtocolRequest(req /* your framework request */, params): ProtocolRequest { return { method: req.method, url: absoluteUrl(req), // e.g. `http://${host}${originalUrl}` — must include the query string params, // from your router match, e.g. { thread_id } query: parsedQuery(req), body: parsedJsonBody(req), headers: lowercasedSingleValueHeaders(req), }; } ``` If you're on Express specifically, `@skein-js/express` exports `toProtocolRequest` so you don't have to write this. ## Step 4 — serialize the `ProtocolResponse` Switch on `response.kind`: - **`json`** — **serialize with `serializeWireJson` from `@skein-js/core`, not your framework's `res.json`.** Bodies may contain LangChain messages (thread state, history, `runs.wait` values) that must be flattened to the wire shape clients expect. Send with `Content-Type: application/json`. - **`headers`** — present on every kind, and **optional to you but not invisible to clients**: this is how the engine returns response metadata it cannot put in the body, such as `x-pagination-total` on assistant search. Set them all before writing the body. Dropping them costs no test failure and no error — the client just silently loses the metadata — so it is worth wiring in from the start. - **`empty`** — just write the status and end. - **`sse`** — set the SSE headers (`SSE_HEADERS`), flush them, then write each string from `response.events` as-is. **Do not re-encode** — the core already produced complete frames (each ends in `\n\n`). When the client disconnects, call the iterator's `return()` so the run's frame subscription is torn down. The [complete adapter](#a-complete-minimal-adapter) below shows this switch assembled. ## Step 5 — handle errors Handlers throw `SkeinHttpError` for client-visible faults (it carries the intended `status`, `message`, and optional `code`/`details`). Anything else is an unexpected `500`. Once SSE headers are flushed you can no longer set a status — just end the stream. `SkeinHttpError` carries `code` and `details` alongside `status`/`message`; forward them when present. Log anything that is _not_ a `SkeinHttpError` — that is a bug, not a client fault. ## Step 6 — worker lifecycle & CORS - **Worker** — `runtime.worker.start()` drains the run queue (background runs). Call `runtime.worker.stop()` on shutdown so in-flight runs drain cleanly. If you wire `createProtocolRuntime` by hand rather than using the shortcut below, pass `{ worker: { maxConcurrency: resolveRunConcurrency(options.worker?.maxConcurrency) } }` — otherwise your adapter silently ignores `--concurrency` and `SKEIN_RUN_CONCURRENCY`, which is exactly the bug `resolveProtocolRuntime` centralizes away. See [run concurrency](./runs-and-redis.md#run-concurrency). - **CORS** — browser clients (Agent Chat UI, React `useStream`) run on a different origin than your server, so you must send `Access-Control-Allow-*` headers (and answer preflight `OPTIONS`) on every route, including the SSE streams. [`@skein-js/server-kit`](https://github.com/skein-js/skein-js/tree/main/packages/server-kit) exports `corsFromHttpConfig` / `toCorsOptions` (the shared, framework-agnostic home; also re-exported from `@skein-js/express`) to derive `cors`-style options from the `langgraph.json` `http.cors` block; on another framework, apply the equivalent middleware. - **Logging** — `resolveProtocolRuntime(options, frameworkLogger?)` takes an optional second argument: your framework's own logger, used only when the caller supplied neither `options.logger` nor `deps.logger`. Pass one **only if your framework owns a logger the host has already configured** (NestJS's `Logger`, `fastify.log`) — then defaulting it on borrows the host's decision, silence included. If it doesn't, pass nothing; a library should not decide on its host's behalf to start writing to stdout. Either way, read the resolved logger back off the result and use _that_ for your transport-fault logging, so the engine and the transport can't disagree about where output goes: ```ts const { runtime, cors, logger } = await resolveProtocolRuntime(options, myFrameworkLogger); // …later, in your error path: sendError(error, res, logger); // the try/catch shape shown in the complete adapter below ``` > **Shortcut:** [`@skein-js/server-kit`](https://github.com/skein-js/skein-js/tree/main/packages/server-kit)'s `resolveProtocolRuntime(options)` > does Steps 1 + the worker lifecycle in one call — resolve `{ config } | { deps }` into a running > runtime (assistants seeded, worker started) plus any CORS from the config and the resolved logger. > It's what the Express, Fastify, NestJS, and Next.js adapters all use. ## A complete minimal adapter A dependency-free adapter over Node's built-in `http` server — no Express, no framework: ```ts import { createServer } from "node:http"; import { SSE_HEADERS } from "@skein-js/agent-protocol"; import { isSkeinHttpError, serializeWireJson } from "@skein-js/core"; import { buildRuntime } from "@skein-js/runtime"; import { resolveProtocolRuntime } from "@skein-js/server-kit"; const assembled = await buildRuntime({ configPath: "./langgraph.json", store: "memory", queue: "memory", }); const { runtime, routes: routeBindings } = await resolveProtocolRuntime({ deps: assembled.deps, channels: assembled.channels, }); // Compile the resolved protocol + channel route patterns to matchers once. const routes = routeBindings.map((r) => ({ ...r, regex: new RegExp("^" + r.path.replace(/:(\w+)/g, "(?<$1>[^/]+)") + "$"), })); const server = createServer(async (req, res) => { try { const url = new URL(req.url ?? "/", `http://${req.headers.host ?? "localhost"}`); const match = routes.find( (r) => r.method === req.method?.toLowerCase() && r.regex.test(url.pathname), ); if (!match) { res.writeHead(404).end(); return; } const params = match.regex.exec(url.pathname)?.groups ?? {}; const body = await readJson(req); // parse the JSON body (omitted for brevity) let protocolRequest = { method: req.method, url: url.href, params, query: Object.fromEntries(url.searchParams), body, headers: req.headers, // already lowercased by Node }; // Thread-scoped run routes: fold the path thread_id into the body. if (match.foldThreadIdIntoBody && params.thread_id) { protocolRequest = { ...protocolRequest, body: { ...(body ?? {}), thread_id: params.thread_id }, }; } const response = await runtime.handlers[match.handler](protocolRequest); if (response.kind === "json") { res.writeHead(response.status, { "content-type": "application/json" }); res.end(serializeWireJson(response.body)); } else if (response.kind === "empty") { res.writeHead(response.status).end(); } else { res.writeHead(response.status, SSE_HEADERS); const it = response.events[Symbol.asyncIterator](); res.on("close", () => void it.return?.(undefined)); for (let n = await it.next(); !n.done; n = await it.next()) res.write(n.value); if (!res.writableEnded) res.end(); } } catch (error) { if (res.headersSent) { if (!res.writableEnded) res.end(); } else if (isSkeinHttpError(error)) { res.writeHead(error.status, { "content-type": "application/json" }); res.end(JSON.stringify({ status: error.status, message: error.message })); } else { res.writeHead(500, { "content-type": "application/json" }); res.end(JSON.stringify({ status: 500, message: "Internal Server Error" })); } } }); server.listen(2024); // On shutdown: await runtime.worker.stop(); server.close(); await assembled.dispose(); ``` Point the official SDK at `http://localhost:2024` and it just works — because the wire format is produced by the same handler table the Express adapter uses. ## Checklist - [ ] Assembled a `ProtocolRuntime`; called `registerGraphAssistants()` and `worker.start()`. - [ ] Bound every route in `resolved.routes`, most-specific-first, with `foldThreadIdIntoBody` honored. - [ ] `ProtocolRequest` has lowercased single-value headers and an absolute `url` (with query). - [ ] JSON responses serialized with `serializeWireJson` (not a plain `JSON.stringify`/`res.json`). - [ ] `response.headers` forwarded on all three kinds (silently lost otherwise — see step 4). - [ ] SSE responses stream frames unmodified, set `SSE_HEADERS`, and tear down on client close. - [ ] `SkeinHttpError` mapped to its status; everything else → `500`; no status changes mid-stream. - [ ] CORS applied for browser clients; `worker.stop()` on shutdown. - [ ] Verified with the real `@langchain/langgraph-sdk` client (see [testing.md](https://github.com/skein-js/skein-js/blob/main/docs/testing.md)). Reference implementation: [`@skein-js/express`](https://github.com/skein-js/skein-js/tree/main/packages/server-express) — [`routes.ts`](https://github.com/skein-js/skein-js/blob/main/packages/server-express/src/routes.ts), [`to-protocol-request.ts`](https://github.com/skein-js/skein-js/blob/main/packages/server-express/src/to-protocol-request.ts), [`send-protocol-response.ts`](https://github.com/skein-js/skein-js/blob/main/packages/server-express/src/send-protocol-response.ts), [`error-response.ts`](https://github.com/skein-js/skein-js/blob/main/packages/server-express/src/error-response.ts). Built an adapter for a framework we don't ship? We'd love a PR — see [CONTRIBUTING.md](https://github.com/skein-js/skein-js/blob/main/CONTRIBUTING.md). --- # Serving the Agent Protocol with your own agent Use `@skein-js/agent-protocol` to expose your own agent implementation through the Agent Protocol. It installs without a graph runtime, so your application can supply an AI SDK loop, a workflow, or another execution engine. Implement the capabilities your clients need and verify them with a [server smoke test](./testing-your-project.md#check-the-running-server) adapted to your agent. This guide covers the agent integration contract. To connect another HTTP framework, follow [Building your own adapter](./building-an-adapter.md). Thread-event frontend bindings need additional capabilities; check the table below before choosing a client. ## What you implement Implement **`AgentGraph`** with the two required methods. Add optional methods for the endpoints you want to support; missing methods have the explicit behavior listed below. ```ts interface AgentGraph { // REQUIRED — run the agent, yielding one chunk per step. stream( input: unknown, options?: unknown, ): Promise> | AsyncIterable; // REQUIRED — the thread's authoritative state after a run. getState(config: { configurable?: Record }): Promise; // Optional. Each maps to endpoints; see the capability table below. getStateHistory?(config, options?): AsyncIterable; updateState?(config, values, asNode?): Promise<{ configurable?: Record }>; bulkUpdateState?(config, supersteps): Promise<{ configurable?: Record }>; streamEvents?(input, options?): AsyncIterable; invoke?(input, options?): Promise; getGraphAsync?(options?): Promise<{ toJSON(): unknown }>; getSubgraphsAsync?(namespace?, recurse?): AsyncIterable<[string, unknown]>; } ``` A LangGraph `CompiledGraph` satisfies this by construction — that is what keeps the type additive. ### Streaming: the one trick worth knowing `stream` yields whatever you like, but a **`[mode, data]` tuple** is unwrapped into that stream mode. `StreamMode` is a plain string union from `@skein-js/core`; nothing LangGraph participates. So a plain async generator produces correct SSE: ```ts async *stream(input, options) { const threadId = String(options?.configurable?.thread_id ?? ""); for await (const step of myAgent(input)) { yield ["values", step]; // -> event: values } } ``` If your agent runs nested agents, a client can ask for their frames with `stream_subgraphs`. Your `stream` then receives `options.subgraphs === true`, and may yield a **`[namespace, mode, data]` triple** — `namespace` a string array, one segment per nesting level, empty for your own frames: ```ts function* streamNestedStep(step: unknown) { yield [["researcher:1"], "updates", step]; // -> event: updates|researcher:1 } ``` Ignore the option and keep yielding pairs if you have nothing nested; see [subgraphs and subagents](./streaming.md#subgraphs-and-subagents). Anything that is _not_ such a tuple is published as an `updates` payload. ### State: seven fields `getState` returns an `AgentStateSnapshot`. skein projects it onto the wire `ThreadState`, and reads exactly these: | Field | Meaning | | -------------- | ---------------------------------------------------------------------------------------------- | | `values` | current state — what `GET /threads/{id}` mirrors | | `next` | nodes still to run. **Non-empty ⇒ the run is reported `interrupted`** | | `tasks` | `{ id, name, error?, interrupts, result? }` — carries pending interrupts and per-node failures | | `config` | its `configurable` carries the checkpoint coordinates | | `parentConfig` | the parent snapshot's config, if any | | `metadata` | passed through untouched | | `createdAt` | ISO timestamp | Only `values`, `next` and `tasks` are required. `{ values: state, next: [], tasks: [] }` is a valid snapshot for an agent that does not pause. ## Capabilities: what an absent method does An optional method you do not implement is a **handled 422**, not a crash: ```json { "status": 422, "message": "This agent does not implement \"updateState\", which this endpoint requires.", "code": "agent_capability_missing", "details": { "capability": "updateState" } } ``` | Method | Endpoints it serves | | ------------------- | ---------------------------------------- | | `getStateHistory` | `GET`/`POST /threads/{id}/history` | | `updateState` | `POST /threads/{id}/state` (time travel) | | `bulkUpdateState` | `POST /threads` carrying `supersteps` | | `streamEvents` | the `events` stream mode | | `invoke` | `POST /invoke/{graph_id}` | | `getGraphAsync` | `GET /assistants/{id}/graph` | | `getSubgraphsAsync` | `GET /assistants/{id}/subgraphs` | **Why 422 and not 501.** `@langchain/langgraph-sdk`'s `AsyncCaller` retries any status outside `STATUS_NO_RETRY = [400,401,402,403,404,405,406,407,408,409,422]`. A 501 would cost the official client five requests and exponential backoff to learn a fact that cannot change on retry. 422 is in that list, and "the request cannot be processed as sent" is honest: the route exists, the body is fine, this agent cannot serve it. `GET /threads/{id}/state` is deliberately **not** in that table: it falls back to `getState` when `getStateHistory` is absent, so the required tier alone serves it. ## A complete server No `langgraph.json`, no CLI, no `@langchain/*` in your file: ```ts import type { AgentGraph, AgentStateSnapshot } from "@skein-js/agent-protocol"; import { createExpressServer } from "@skein-js/express"; import { embedInMemoryGraphs } from "@skein-js/server-kit"; const stateByThread = new Map(); const threadId = (c?: { configurable?: Record }) => String(c?.configurable?.["thread_id"] ?? ""); const agent: AgentGraph = { stream(input, options) { const id = threadId(options as { configurable?: Record }); return (async function* () { const reply = { messages: [{ role: "ai", content: "hello" }] }; stateByThread.set(id, reply); yield ["values", reply]; })(); }, async getState(config): Promise { return { values: stateByThread.get(threadId(config)) ?? {}, next: [], tasks: [] }; }, }; // Pass a `GraphResolver` — not a graph map. A map's values are LangGraph compiled graphs by type. const deps = embedInMemoryGraphs({ ids: ["chat"], load: async () => agent, schemas: async (id) => ({ [id]: { graph_id: id } }) as never, }); await (await createExpressServer({ deps })).listen(2024); ``` That serves `POST /threads`, `GET /threads/{id}`, `POST /runs/wait`, `POST /runs/stream` (SSE with replay), `GET /threads/{id}/state`, `GET /info`, plus auth, `Idempotency-Key`, crons, the store, and multitask strategies — none of which you implement. `storeBridge`, `ephemeralCheckpointer` and `cloneCheckpoint` on `ProtocolDeps` are all **optional** and runtime-specific; omit them. `checkpointer` is a structural `ThreadCheckpointer` (five methods: `getTuple`, `list`, `put`, `putWrites`, `deleteThread`) — supply a stub if your agent keeps its own state, or a real one if you want thread copy / prune / rollback. ## Interrupts and resume `POST /runs` with a `command` body resumes an interrupted run. The engine does **not** construct a runtime's command type — that would be a runtime import. It hands your agent a branded envelope: ```ts import { isAgentCommand, agentCommandPayload } from "@skein-js/agent-protocol"; stream(input, options) { if (isAgentCommand(input)) { const { resume, update, goto } = agentCommandPayload(input); // …resume your agent with `resume` } } ``` Use `agentCommandPayload` rather than reading fields off the envelope: the wire schema passes unknown fields through, and the payload is the complete, unbranded object. To report a pause, return a snapshot with non-empty `next` (or tasks carrying `interrupts`). ## What you cannot change - **The wire types.** Use the SDK-shaped protocol types and implement the capabilities your client needs. Check the [verified client combinations](./langgraph-compatibility.md) rather than assuming type compatibility establishes every client behavior. - **The run lifecycle** — statuses, multitask strategies, the SSE frame envelope. - **The auth model** — see [the Agent Protocol reference](./agent-protocol.md). Auth is the deployment's concern, not the agent's; a runner never sees it. ## Known limits - **`events` mode is effectively LangChain-only.** Its demux keys on `on_chain_stream`, root `run_id`, and `langsmith:hidden` tags. Your agent may yield `mode: "events"` chunks, but their payload is your contract with your own clients — `useStream` will not render them as token events. - **`useStream` hydration** calls `GET /threads/{id}/state`. Implement `getState` well (it is required anyway) or hydration is empty. - **Time travel** needs `updateState` + `getStateHistory`. Without them, `checkpoint_id` on a run is ignored rather than honoured. ## The reference implementation [`@skein-js/langgraph`](https://github.com/skein-js/skein-js/tree/main/packages/langgraph) is one implementation of this seam, and it imports only `@skein-js/agent-protocol`'s public entry point — no privileged access. If it ever needs an internal, the internal is missing from the public API. Reading it is the fastest way to see what a complete binding looks like. --- # Deploy anywhere ## Choose a deployment | Option | Use it for | Storage and host | | --------------------- | ----------------------------------------------------- | -------------------------------------------------------------------------- | | **Shared server** | Web/API clients and workers spanning instances | Postgres for resources/checkpoints; Redis for a shared queue and streaming | | **Local application** | Desktop apps, CLI tools, daemons, or one Node process | One exclusively owned SQLite file through `@skein-js/local` | For the SQLite option, follow [local deployment](./local-deployment.md). It supports Node 22/24/26; Electron and Tauri hosts package the same engine and use the existing protocol. Models and tools remain application choices. The default `skein dev` continues to use memory drivers. The rest of this page covers the **server image and shared-service deployment**. **Deploy your LangGraph.js graphs anywhere you can run a container — or just Node.** skein has no `skein deploy` and no control plane — that's the point ([roadmap.md](./roadmap.md) lists it as an explicit non-goal). What it ships instead is an ordinary OCI image: `skein build` bundles your TypeScript graphs to plain JS and produces a Docker image that needs a Postgres, a Redis, and two environment variables. Nothing about it is specific to any host. Production artifacts can run on Node, Bun, or Deno. Node 24 LTS is the default and uses Express. Bun and Deno use the Web-standard `@skein-js/fetch` adapter and their native HTTP servers. Select one in `langgraph.json` or at build time: Use Node for the default production path. Bun and Deno are preview targets; verify streaming, background execution, persistence, and shutdown on your chosen runtime before deployment. ```json { "skein": { "runtime": { "name": "deno", "version": "2.9.4" } } } ``` ```bash skein build --runtime bun --runtime-version 1.3.14 -t my-agent ``` CLI flags override config. The image pins the official runtime image, imports every graph under that runtime during the build, starts the runtime directly as PID 1, and uses a non-root user. Deno gets explicit network, environment, artifact-read, system, and native-library permissions. Skein cannot make an arbitrary Node-native graph dependency portable; the compatibility probe fails that image build so the dependency can be replaced or isolated before deployment. The server image contract below applies across the listed hosting platforms. The per-platform guides are the dashboard and CLI steps on top of it. ## Pick a platform **Background runs is the deciding column.** A platform that throttles or suspends your container between requests will stall a queued run until the next request wakes it — which is fine for a request/response graph and wrong for anything scheduled or fired-and-forgotten. :::tabs == Cloud Run - **Deploy from** — push image to Artifact Registry - **Postgres + Redis** — Cloud SQL + Memorystore, or any hosted - **Background runs** — ⚠️ needs `--no-cpu-throttling` + `--min-instances=1` - **Scales to zero** — yes — queued runs stall until a request - **Stop-signal window** — 10s default, configurable → [Google Cloud Run guide](./deploy-cloud-run.md) == Railway - **Deploy from** — Dockerfile in repo - **Postgres + Redis** — Railway plugins - **Background runs** — ✅ default - **Scales to zero** — no - **Stop-signal window** — ~30s → [Railway guide](./deploy-railway.md) == Fly.io - **Deploy from** — Dockerfile in repo - **Postgres + Redis** — Fly Postgres / Upstash Redis - **Background runs** — ⚠️ needs `min_machines_running = 1` - **Scales to zero** — yes, if auto-stop is on - **Stop-signal window** — `kill_timeout` (5s default) → [Fly.io guide](./deploy-fly.md) == Render - **Deploy from** — Dockerfile or image - **Postgres + Redis** — Render Postgres + Key Value - **Background runs** — ⚠️ paid instance (free ones spin down) - **Scales to zero** — free tier only - **Stop-signal window** — ~30s → [Render guide](./deploy-render.md) == AWS App Runner - **Deploy from** — push image to ECR - **Postgres + Redis** — RDS + ElastiCache - **Background runs** — ⚠️ CPU throttled between requests - **Scales to zero** — no (min 1 instance) - **Stop-signal window** — ~30s → [AWS App Runner guide](./deploy-aws.md#app-runner) == AWS Fargate - **Deploy from** — push image to ECR - **Postgres + Redis** — RDS + ElastiCache - **Background runs** — ✅ default - **Scales to zero** — no - **Stop-signal window** — `stopTimeout` (30s default) → [AWS ECS Fargate guide](./deploy-aws.md#ecs-fargate) == Kubernetes - **Deploy from** — push image to any registry - **Postgres + Redis** — whatever you run - **Background runs** — ✅ default - **Scales to zero** — no (unless KEDA/Knative) - **Stop-signal window** — `terminationGracePeriodSeconds` 30s → [Kubernetes guide](./deploy-kubernetes.md) == VPS - **Deploy from** — build on the box or pull - **Postgres + Redis** — containers or managed - **Background runs** — ✅ default - **Scales to zero** — no - **Stop-signal window** — `docker stop -t` (10s default) → [VPS / plain Docker guide](./deploy-vps.md) == Serverless - **Deploy from** — n/a — not a container - **Postgres + Redis** — any hosted - **Background runs** — ❌ not supported - **Scales to zero** — yes - **Stop-signal window** — none → [Vercel & serverless guide](./deploy-serverless.md) ::: > **How these were verified.** The image contract itself — port binding, the `/ok` probe, `SIGTERM` > draining in-flight runs to a terminal status, Postgres migrations on boot — was exercised > end-to-end against real Postgres and Redis containers, and the behaviors described below are what > was actually observed. The individual platform guides apply that same contract using each > platform's own documentation; they have not each been deployed for real. If a step is wrong on your > platform, please [open an issue](https://github.com/skein-js/skein-js/issues) — corrections to > these are very welcome. ## What the image already does for you `skein build` produces the image; `skein dockerfile` prints the same Dockerfile if you'd rather commit it and let your platform build it. Either way: - **Binds the port the platform gives it.** The CMD passes no `--port`, so the server binds `$PORT` when one is injected (Railway, Render, Cloud Run and AWS App Runner all do). When nothing is injected it falls back to **8123** — the same port the image `EXPOSE`s and health-checks — so a bare `docker run -p 8123:8123` works, as do platforms that route to a port you declare without setting `PORT` for you (Fly.io, ECS, Kubernetes). - **Handles `SIGTERM` properly.** The selected runtime is PID 1 (the CMD invokes it directly, not through `npx` — under `npx`, PID 1 is npm, which exits on `SIGTERM` without waiting for the server). On signal, skein stops accepting queued runs, gives in-flight runs [a grace window](#graceful-shutdown) to finish, aborts whatever is left so it lands in a **terminal** status rather than stranded as `running`, then closes the pools. - **Serves a health probe** at `GET /ok` → `200 {"ok":true}`, and declares a Docker `HEALTHCHECK` against it with a 20s start period. - **Runs unprivileged** as the runtime image's `node`, `bun`, or `deno` user. - **Reads config from the environment only** — `POSTGRES_URI` and `REDIS_URI`. The generated `.dockerignore` excludes `.env*` and `.npmrc*`, so secrets are never baked into a layer. - **Installs pinned production dependencies only.** No vite/tsx, no devDependencies, no runtime TypeScript transform: `skein build` resolved your tsconfig `paths` and workspace aliases once, on the host. Source maps stay on, so stack traces still point at your TypeScript. The pinned set is **derived from the bundle**: every published package your graphs still import after bundling is recorded at the exact version installed on the build host, and `skein build` fails on the host if the two ever disagree. Packages you load **by name at runtime** are the exception a bundler cannot see — declare those under `dependencies` in `langgraph.json` ([what `skein build` bundles](./bundling.md#what-skein-build-inlines-vs-externalizes)). - **Caches dependency installs** via a BuildKit cache mount, and accepts an optional `id=npmrc` build secret for private scoped packages — `skein build --npmrc `, or `docker build --secret id=npmrc,src=$HOME/.npmrc` for the standalone Dockerfile. See [langgraph-cli-compat.md](./langgraph-cli-compat.md). **Migrations run automatically on boot.** There is no `skein migrate` step. On startup skein applies its schema (tracked in a `skein_migrations` table), sets up LangGraph's checkpoint tables, registers one assistant per declared graph, and — because `skein start` warms graphs — imports every graph module. All of that finishes _before_ the server starts listening. Migrations take their own advisory lock, so several instances booting at once during a rolling deploy is safe — the ones that don't win the lock **wait**, then find nothing to apply. That covers **both** schemas, on two separate lock keys: skein's own, and LangGraph's checkpoint tables. The second one is skein's lock around someone else's migration — `PostgresSaver.setup()` in `@langchain/langgraph-checkpoint-postgres` reads its version ledger and then creates types and inserts rows with no exclusion of its own, so concurrent boots against a database with pending migrations collide on `checkpoint_migrations_pkey` or `pg_type`. Separate keys because the two schemas are independent: sharing one would make every boot wait on a migration it does not depend on. It only bites when migrations are genuinely pending — a first deploy, or an upgrade that bumps that package's schema — which is exactly when replicas start together. > **Building on Apple Silicon?** `skein build` doesn't pass `--platform`, so you'll get an arm64 image > that most hosts reject. Export `DOCKER_DEFAULT_PLATFORM=linux/amd64` before building. ## Without Docker A container is the default, not a requirement. `skein build --artifact-only` stops before invoking Docker and writes `.skein/build` — your graphs bundled to plain JS, a rewritten `langgraph.json`, the baked `schemas.json`, and a `package.json` whose dependencies are **exact-pinned**, so the install is deterministic without a lockfile. Ship that directory to a machine with Node and run it: ```bash skein build --artifact-only # on any machine with Node; no Docker daemon needed rsync -a .skein/build/ server:/srv/skein/ # on the server cd /srv/skein && npm install --omit=dev export POSTGRES_URI=postgres://… REDIS_URI=redis://… npx skein start # or: node node_modules/skein-js/dist/index.js start ``` The bare command above runs inside the copied artifact directory. To serve a local build from the project root, select the artifact's config explicitly: ```bash skein build --artifact-only skein start -c .skein/build/langgraph.json ``` `skein start` defaults to `langgraph.json` in the working directory. When that config has no adjacent schemas and a build exists in its directory's `.skein/build`, the error suggests a `-c` argument to use in your original start command. Keep its other options, including storage drivers and any custom `--sqlite-path`. Artifact selection stays explicit. `skein start` also reads a conventional `.env` from the directory you run it in, as well as one beside the config it loads — so running your own build from the project root (`skein start -c .skein/build/langgraph.json`) picks up that project's `.env` without exporting anything. An artifact never carries a `.env` of its own: it is the Docker build context, and `skein build` drops a file `env` from the config for that reason. The ambient environment still wins over both, which is what the `export` above relies on. `skein start` serves the artifact and is the same entrypoint the image uses — the container's `CMD` is literally `node /app/node_modules/skein-js/dist/index.js start --store postgres --queue redis --host 0.0.0.0`. `skein-js` is pinned into the artifact's own dependencies, so the install above puts the CLI on the box with it. Four differences from the containerised path: - **It binds `127.0.0.1` by default**, where the image passes `--host 0.0.0.0`. That default is right behind a reverse proxy on the same machine and wrong if something else must reach it directly — the image overrides it because a container is already isolated. - **`--store postgres --queue redis` are the defaults**, so a bare `skein start` reaches for both and fails with an actionable error if `POSTGRES_URI` / `REDIS_URI` are missing. There is no in-memory production mode. - **The port is 8123**, not `dev`'s 2024, and `PORT` from the environment wins over the default. - **Bun and Deno need `--runtime`.** The artifact records which runtime it was built for and refuses to start under a different one. Node needs no flag. Everything else on this page still applies — migrations run on boot, `/ok` is the probe, `SIGTERM` drains in-flight runs. What you give up is the pinned base image and the process supervision the platform would have done, which on a VM is a [systemd unit](./deploy-vps.md#run-it-without-docker). ## What every deployment needs ### 1. A Postgres Set `POSTGRES_URI`. It holds protocol resources (assistants, threads, runs, store items) and LangGraph checkpoints. The base schema needs **no extensions**. pgvector is needed **only if you set `store.index` in `langgraph.json`** for semantic search. skein runs `CREATE EXTENSION IF NOT EXISTS vector` on boot, which can only enable an extension the server already has — it cannot install one. If it's missing, boot fails with an error telling you so. | Provider | pgvector available | | ------------------- | ---------------------------------------------------------- | | Cloud SQL (PG 13+) | ✅ (`vector` is a supported extension) | | AWS RDS (PG 15+) | ✅ | | Neon | ✅ | | Supabase | ✅ (enabled by default) | | Render Postgres | ✅ | | Railway | ⚠️ use the **pgvector template**, not the default Postgres | | `postgres:16` image | ❌ — use `pgvector/pgvector:pg16` | ### 2. A Redis Set `REDIS_URI`. skein uses it for the run queue (BullMQ) and for cross-instance stream pub/sub with replay. Configure the instance with `maxmemory-policy noeviction` — BullMQ's job data must not be evicted. **The image requires it, and so does the entrypoint.** Its CMD runs `skein start --store postgres --queue redis` — but those are also `skein start`'s own defaults now, and it _rejects_ `--store memory` / `--queue memory` outright. Overriding the CMD, or running the binary by hand, can no longer produce a production server whose queue is process-local and whose state disappears on restart. The redis queue driver fails the boot if `REDIS_URI` is unset. (Redis is only _optional_ on the in-code embedding path — `embedPostgresGraphs` falls back to an in-memory queue and bus when no Redis URL is given, which keeps state durable but limits you to a single instance. See [embedding.md](./embedding.md#going-to-production).) ### 3. The port Nothing to do if your platform injects `PORT`. If it asks you which port the container listens on, answer **8123**. ### 4. A health probe Point it at **`/ok`**. It's a dependency-free liveness check that deliberately does _not_ touch Postgres or Redis, so a transient database blip can't flap an otherwise healthy instance. It also works as a **startup/readiness probe**: migrations, assistant registration and graph warming all complete before the server binds, so a responding `/ok` genuinely means "fully booted". Budget your startup probe accordingly — the image's own estimate is 20 seconds. ### 5. Auth — read this before you expose it > ⚠️ **skein's auth is off by default, and this is the production path.** With no `auth.path` > configured in `langgraph.json`, every protocol endpoint is open: anyone who can reach the URL can > create threads, run your graphs, and spend your model-provider tokens. Either configure auth (see > [agent-protocol.md](./agent-protocol.md) for the route→permission map) or keep the service private — > behind your platform's authenticated ingress, a VPC, or an authenticating proxy. > > `/ok` is registered ahead of the auth engine, so health probes keep working either way. ## Sizing & tuning ### Connection budget skein opens **three** Postgres pools per instance: one for protocol resources, one for LangGraph's `PostgresSaver`, and one for the per-thread execution claim (see [Scaling past one instance](#scaling-past-one-instance)). All three are capped by `PG_POOL_MAX`, so plan for: ```text max connections ≈ 3 × PG_POOL_MAX × instances ``` Check that against your database's limit — this is the most common way to exhaust a small managed Postgres once autoscaling kicks in. `PG_POOL_MAX=5` is a sane starting point. The execution-claim pool behaves differently from the other two, and it is worth knowing how: a connection is held for the **whole duration** of an executing run, not for the length of a query. So its working size is your run concurrency, not your request rate — and it is sized from `SKEIN_RUN_CONCURRENCY` plus headroom for the inline run modes (`/runs/wait`, `/runs/stream`, `/threads/{id}/stream`, `/threads/{id}/commands`), which execute without consuming a worker slot and are therefore bounded by request arrival rather than by run concurrency. Past that headroom an inline run waits for a free claim connection and then **fails** rather than executing unguarded — the safe direction, since executing without the claim is what interleaves two runs' checkpoint writes. Keep `PG_POOL_MAX` at or above run concurrency (`skein start` warns at boot when it is not), and raise it if you serve heavy concurrent streaming alongside a saturated background worker. ### When the database stops answering `pg` waits for a pool connection **forever** by default, which turns an unreachable database into a **hang** rather than an error: no status code, no log line, just a socket the client eventually abandons. skein applies a 30s `PG_CONNECTION_TIMEOUT_MS` so the fault surfaces. Thirty rather than something tighter, because `pg` uses that one timer for two different waits — the connection handshake **and** waiting for a free client when the pool is already at `PG_POOL_MAX`. A tight bound therefore fails two ordinary situations: a burst of slow-but-working queries against a small pool, and an autosuspended serverless Postgres (Neon, Supabase) waking up, which regularly takes longer than ten seconds and happens on the boot path. Set `PG_CONNECTION_TIMEOUT_MS=0` for `pg`'s original wait-forever behaviour. `PG_STATEMENT_TIMEOUT_MS` bounds a single statement server-side — the last line of defence against one pathological query pinning a pool connection. **On by default at 30s**; `0` disables it. It is per _statement_, not per request, so a legitimately long sequence of quick queries is unaffected; what it catches is one query that is stuck or scanning something it shouldn't. Suggested 15000 on a small instance, 60000 on a large one. It became a default only once the list/search paths were page-bounded and indexed — before that it would have turned slow-but-working queries into errors. If your deployment has a query that genuinely runs longer, raise it or set `0`; a cancelled statement surfaces as a `57014` error naming the statement, rather than as a hang. The shapes that can still take a while, all of them either bounded by you or inherently large: a deep `OFFSET`, an unindexed `values` filter, `POST /store/namespaces` (`SELECT DISTINCT` over the store), store search by text with no `store.index` configured (a whole-table read by design), and anything that walks a very large thread's whole checkpoint history — copying a thread, or a `multitask_strategy: "rollback"` on one. Rollback in particular reports a failure as a warning and continues, so on very large threads either raise the timeout or prefer another strategy. Schema DDL is exempt, and deliberately so: our migrations and the pgvector setup lift the timeout on their own connection, and the checkpointer's `setup()` runs on a separate untimed pool. Index builds are legitimately slow, a cancelled `CREATE INDEX CONCURRENTLY` leaves an _invalid_ index that the retry skips by name (recording the migration as applied while the index goes permanently unused), and a cancelled boot migration is a boot _loop_ rather than a slow boot. Those connections are destroyed rather than returned to the pool, so the lifted timeout can't leak into a later query that happens to reuse them. It is applied with a `SET` on each new connection rather than as a startup parameter, because PgBouncer and Supabase's pooler reject unrecognised startup parameters outright. Under **transaction** pooling a `SET` does not persist, so the timeout silently does not apply there — it is a backstop, not a guarantee. **Full sizing guidance, and every tuning knob in one table, is in [performance.md](./performance.md).** This section covers only what is specific to running the container. ### Heap size vs. the container's memory limit The image bakes **no** `--max-old-space-size`, because a Dockerfile cannot know the limit the container will be given. Instead `skein start` compares the two at boot and warns when V8's ceiling is above ~75% of the container's limit, naming the flag and a computed value. Node **is** cgroup-aware — since v12 it sizes the heap from the container's limit, not the host's — so the usual advice to set `--max-old-space-size` on every container is wrong here, and following it can make things worse. Measured on `node:22-slim`, V8's ceiling tracks about half the limit: 512Mi → 259MB, 1Gi → 524MB, 2Gi → 1048MB. Setting it to "75% of the limit" on a 512Mi container would _raise_ the heap from 259MB to 432MB and make an OOM kill more likely, not less. What the automatic sizing does not do is go below a floor of about **259MB**. So under roughly 345Mi — 256Mi and 128Mi are ordinary Cloud Run and Kubernetes settings — V8's ceiling meets or exceeds the whole container, it never feels the pressure that would trigger a full GC, and the kernel kills the process first. That appears as a restart with no stack, no log line, and nothing in your metrics. **Only act on this when the warning fires**, and then lower the heap rather than raising it. The warning names a computed value. Include the image's own flag when you set it — `NODE_OPTIONS` replaces the image's value rather than adding to it, so omitting `--enable-source-maps` silently costs you TypeScript stack traces: ``` NODE_OPTIONS="--enable-source-maps --max-old-space-size=153" ``` `skein start` also warns when run concurrency exceeds `PG_POOL_MAX`, for the same reason: runs then queue waiting for a connection rather than executing, which looks like flat throughput rather than like a pool limit. Budget **two** pools per instance (store + checkpointer). ### Heap pressure while running Separately from the boot check, skein samples heap usage every 30s and warns once when it passes 85% of the limit — then stays quiet until it drops back below 70%, so a sustained episode is one log line rather than one every 30 seconds. It runs for the life of the background worker and needs a `logger` to be configured; `SKEIN_HEAP_WARN_PERCENT=0` turns it off. The warning carries what makes it actionable, because the percentage alone does not: | What the line shows | What it means | | -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | | High `runs in flight`, at or near your concurrency | Too much work at once — lower `SKEIN_RUN_CONCURRENCY` or scale out. | | High `buffered frames` | A slow SSE consumer is holding a run's frames in memory — see `SKEIN_MEMORY_BUS_MAX_FRAMES_PER_RUN`, or move to Redis. | | Neither, and the heap stays high across episodes | A genuine leak. Capture a heap snapshot. | On Redis there is no `buffered frames` figure, because the frames are not in this process's heap — which is itself part of the answer. Two limits worth knowing. It watches the **JS heap only**: `used_heap_size` excludes external memory, so a process killed for RSS — large `Buffer`s, many sockets — never trips it. That is the right scope for what skein itself retains (buffered frames, mirrored graph state), but it does not make the boot check above redundant. And the re-arm band sits 15 points under whatever threshold you set, so `SKEIN_HEAP_WARN_PERCENT=60` re-arms at 45%; a value at or below 15 latches after its first warning for the life of the process, which is the safe direction rather than a flood. ### Run concurrency Each instance executes up to **10** queued runs at once. Set `SKEIN_RUN_CONCURRENCY` (or the LangGraph-compatible `N_JOBS_PER_WORKER`) to change it. Every in-flight run draws from both pools above, so `concurrency × instances` is the number to budget against the connection cap. Prefer more instances over higher concurrency when runs are CPU-bound. See [runs-and-redis.md](./runs-and-redis.md#dispatch-by-thread) for thread partitioning and custom-driver fallback. The environment variable is validated even when the flag is also passed, so a typo fails the boot loudly instead of silently reverting to the default. ### TLS to the database A URL with `?sslmode=require` and a real CA chain needs nothing extra — `pg` honors `sslmode`. For a database presenting a self-signed certificate, set `DATABASE_SSL_NO_VERIFY=true`. Over a private network (Railway's `*.railway.internal`, a VPC, a Unix socket) you need neither. If your provider offers a **pooled** connection endpoint (PgBouncer and friends), use the **direct** endpoint instead. Boot migrations take a session-level advisory lock, which transaction-mode pooling does not preserve. ### Graceful shutdown On `SIGTERM` skein stops pulling from the queue, waits `SKEIN_SHUTDOWN_GRACE_MS` (default **5000**) for in-flight runs to finish, then aborts the stragglers so they settle to a terminal status, closes the pools, and exits. If that whole sequence hasn't finished 3 seconds after the grace window, the process force-exits anyway — so the default worst case is ~8s, which fits inside the tightest common kill window. Raise the grace window where the platform allows a longer one, and keep it below what the platform actually grants — past that, you're just being SIGKILLed mid-drain: | Platform | Window between SIGTERM and SIGKILL | | ------------- | ----------------------------------------- | | Cloud Run | 10s default, configurable | | Railway | ~30s | | Fly.io | `kill_timeout` in `fly.toml` (5s default) | | Render | ~30s | | ECS | `stopTimeout` (30s default, 120s max) | | Kubernetes | `terminationGracePeriodSeconds` (30s) | | `docker stop` | `-t` (10s default) | Runs that get aborted are marked terminal, not lost: with Redis, BullMQ's stalled-job recovery re-delivers a job whose worker died, and the worker skips any run already in a terminal state. **Embedding skein in your own server instead of using `skein start`?** You get no signal handling — wire it yourself: `process.on("SIGTERM", …)` → `runtime.worker.stop()` → dispose. Drain first, dispose second; disposing while runs are still draining pulls the store out from under them. ### Cold starts Boot does real work: schema migrations, `PostgresSaver` setup, one get-or-create per declared graph, and eager-loading every graph module. Twenty seconds is a reasonable startup-probe budget. On platforms that scale to zero, this is paid on the first request after an idle period. ## Scaling past one instance With Postgres and Redis, replicas share state and streams — a client can join a run executing on another instance, rolling deploys are safe, **and the run semantics hold across instances**. No session affinity or single-instance restriction is needed for any of it: - **Cancellation** crosses instances. `POST …/runs/{id}/cancel` routed to instance B stops a run executing on instance A: the run row is settled immediately (which is what makes the cancel durable), and the _signal to stop now_ travels over a Redis pub/sub channel to whichever instance is executing it. Delivery is best-effort by design — a dropped message costs promptness, never correctness. - **One-active-run-per-thread** is decided by Postgres, not by a lock in one process: `multitask_strategy: "reject"` is an atomic check-and-insert, so two instances racing the same thread cannot both win. - **`multitask_strategy: "enqueue"`, `"interrupt"` and `"rollback"`** all hold. A run claims its thread with a Postgres **session advisory lock** held for the run's duration, so a queued run waits for the active one wherever that one is running. The displaced run's base checkpoint and the displacing run's rollback plan are persisted on the runs themselves, so any instance can apply them — and a run recovered after a crash still cleans up what it displaced. **Why a Postgres lock and not a Redis lease.** A run holds its claim for its whole execution, which can be minutes. A TTL lease has to be renewed for all of it, and a late renewal — a blocked event loop, a GC pause — expires the lease while the run is still writing, putting two instances on one thread's checkpoint history. A session lock has no TTL: Postgres holds it until the session releases it _or the connection dies_, so a crashed instance frees its threads at once and a slow one keeps them. This is the same split LangGraph Platform makes (Postgres for rows and exclusivity, Redis for ephemeral pub/sub). **Budget for it.** Each _concurrently-executing_ run holds one connection from a dedicated pool for the run's duration — the same trade LangGraph Platform makes. See [Connection budget](#connection-budget). The one remaining caveat is utilization, not correctness: a queued run waiting for a busy thread still occupies a worker slot, so a burst of `enqueue` runs on one thread can crowd out other threads' work. Tracked as per-thread partitioned dispatch on the [roadmap](./roadmap.md). See [runs-and-redis.md](./runs-and-redis.md). ## Streaming through proxies (SSE) skein sends `text/event-stream` with `cache-control: no-cache, no-transform` and flushes headers immediately. Streams are **back-pressured**: a client that reads slowly is paced rather than buffered in the server's memory, so a few hundred slow connections cost a bounded ~65 KB each instead of a full copy of each stream — see [streaming.md](./streaming.md#slow-clients-and-backpressure). Two things it does _not_ do, which matter in front of a proxy: - It sends **no `X-Accel-Buffering: no` header**. A buffering reverse proxy will hold the stream until the run finishes, which looks exactly like a hang. Turn buffering off: nginx `proxy_buffering off;`, ingress-nginx `nginx.ingress.kubernetes.io/proxy-buffering: "off"`, Caddy `flush_interval -1`. Don't put a caching CDN in front of the stream routes. - It sends **no heartbeat frame**. A stream that produces no tokens for a while can be culled by an idle timeout, so raise the proxy's read timeout to cover your longest quiet stretch. There is also **no server-side run timeout** under `skein start` — the engine supports one (`runTimeoutMs`), but no CLI flag or environment variable exposes it, so it is only reachable when you [embed skein in your own server](./embedding.md). With the image, the platform's request timeout is the only ceiling on a streaming run; set it generously (Cloud Run allows up to 60 minutes). Clients reconnect with `Last-Event-ID` and skein replays missed frames from a Redis stream (kept for an hour), so a dropped connection is recoverable. See [streaming.md](./streaming.md). ## Verify a deployment Every platform guide points here. Substitute your service's base URL. ```bash BASE=https://your-service.example.com # 1. Liveness — the same probe your platform uses. curl -s $BASE/ok # {"ok":true} # 2. Your graphs registered as assistants at boot. curl -s -X POST $BASE/assistants/search \ -H 'content-type: application/json' -d '{"limit":10}' # 3. An inline streaming run — exercises SSE and Postgres checkpointing end to end. THREAD=$(curl -s -X POST $BASE/threads -H 'content-type: application/json' -d '{}' \ | node -pe 'JSON.parse(require("fs").readFileSync(0)).thread_id') curl -N -X POST $BASE/threads/$THREAD/runs/stream \ -H 'content-type: application/json' \ -d '{"assistant_id":"agent","input":{"messages":[{"role":"user","content":"hi"}]}}' # 4. A BACKGROUND run — the one that catches CPU-throttling platforms. It returns immediately; # the work happens after the request ends, which is exactly when a throttled instance freezes. RUN=$(curl -s -X POST $BASE/threads/$THREAD/runs \ -H 'content-type: application/json' \ -d '{"assistant_id":"agent","input":{"messages":[{"role":"user","content":"hi"}]}}' \ | node -pe 'JSON.parse(require("fs").readFileSync(0)).run_id') # 5. Join it mid-flight (replayed from Redis), then confirm it finished. curl -N $BASE/threads/$THREAD/runs/$RUN/stream curl -s $BASE/threads/$THREAD/runs/$RUN | node -pe 'JSON.parse(require("fs").readFileSync(0)).status' ``` Step 4 is the one worth actually running. A background run that never leaves `pending`/`running` while the service is idle means the platform is suspending your instance between requests — see that platform's guide. ## Environment variables skein reads these and nothing else. Note there is **no `DATABASE_URL` or `REDIS_URL`** — those are platform names; map them onto skein's. | Variable | Required | Purpose | | ------------------------ | -------------------- | ----------------------------------------------------------------- | | `POSTGRES_URI` | yes (postgres store) | Postgres connection string (resources + checkpoints). | | `REDIS_URI` | yes (redis queue) | Redis connection string (run queue + stream pub/sub). | | `PORT` | usually injected | Port to bind. Defaults to 8123 — the port the image exposes. | | `HOST` | no | Host to bind. The image already passes `--host 0.0.0.0`. | | `DATABASE_SSL_NO_VERIFY` | no | `true` to skip TLS cert verification (self-signed database cert). | Everything else is **tuning**, and lives in one place so the numbers can't drift apart: **[performance.md](./performance.md#every-knob)** has every knob with its default and a suggested value for a small (256–512Mi) and a large (1–4Gi) deployment — run concurrency, the shutdown drain, page and stream bounds, the pool and statement timeouts, the heap monitor, and request logging. Two notes specific to the container: - The in-memory bus knobs (`SKEIN_MEMORY_BUS_*`) are **not** reachable from this image — `skein start` rejects `--queue memory`. They apply on the embedded path (`embedPostgresGraphs` with no `REDIS_URI`) and under `skein dev`; see [embedding.md](./embedding.md#going-to-production). - `PG_POOL_MAX` is per pool and skein opens **three** per instance, so budget three times your setting against the database's own connection cap — per replica. ## Compatibility assets and diagnostics `skein build` includes configured graph UI assets and `http.app` exports in the production artifact. Ship the whole artifact; production does not need TypeScript sources or the development UI compiler. After building, test your custom routes and UI assets from the artifact without the source tree, and check the generated API reference. See [configuration compatibility](./langgraph-cli-compat.md#graph-ui-and-custom-http-applications). Use Redis thread events for reconnect across API instances. SQLite and memory event replay are process-local, even when SQLite resource rows and checkpoints persist. Monitoring remains host-owned: configure an OpenTelemetry reader before exposing `/metrics`, and use the storage endpoint for exact checkpoint payload accounting. See [diagnostics](./observability.md#http-diagnostics). --- # Performance & memory How skein behaves under load, what bounds it, and which knob to reach for when something looks wrong. The short version: **every buffer in skein is bounded, and every bound has a knob.** The defaults are chosen so an ordinary deployment never loses a frame or truncates a page, which means they are not chosen to fit a small container. If you run on 256–512Mi, read [Sizing](#sizing) — a handful of knobs do almost all the work. ## What actually uses memory Four things, in the order they matter: 1. **Run frames in flight.** A streaming run produces frames faster than a client consumes them. Those frames live in the event bus until every subscriber has read them, and under `stream_mode: "values"` each frame is a copy of the whole graph state. This is the biggest and the most variable. 2. **Socket write buffers.** Every SSE connection whose client is slower than the graph. Bounded to the socket's own high-water mark (~64KB) — see [backpressure](#streaming-backpressure-drops-and-recovery). 3. **Rows read by one request.** A thread row carries the thread's mirrored graph state, so a search that returns 1000 threads is 1000 graph states, twice over (the rows, then the serialized response). 4. **The runtime itself.** ~130MB RSS to import an adapter and start a server, dominated by `@langchain/core` and `@langchain/langgraph`. This is a floor, not a variable — see [bundling.md](./bundling.md). Runs themselves are cheap when idle: a queued run is a row, not a process. ## Sizing ### Small: 256–512Mi, many replicas The shape most platforms default to. The defaults do **not** fit here — they are sized so a long, chatty run never loses a frame, and the worst case is roughly `MAX_FRAMES_PER_RUN × (concurrent runs + MAX_RETAINED_RUNS)`. At the defaults that ceiling is half a million frames. ```bash SKEIN_RUN_CONCURRENCY=3 SKEIN_MEMORY_BUS_MAX_FRAMES_PER_RUN=2000 # only when queue=memory SKEIN_MEMORY_BUS_MAX_RETAINED_RUNS=20 # only when queue=memory SKEIN_STREAM_BUFFER_FRAMES=128 # only when queue=redis SKEIN_MAX_PAGE_SIZE=200 PG_POOL_MAX=5 PG_STATEMENT_TIMEOUT_MS=15000 ``` Worked example, 512Mi with Redis: the runtime floor is ~130MB. Three concurrent runs, each with one SSE subscriber, buffer at most `3 × 128` frames in this process; at ~2KB per frame that is under a megabyte. The store pool holds 5 connections, the checkpointer pool another 5. A `POST /threads/search` returns at most 200 threads. That leaves the bulk of the container for the graph's own working set — which is where you actually want it, and which skein cannot size for you. Below ~345Mi also check [the heap-limit warning](./deploy.md#heap-size-vs-the-containers-memory-limit): Node's automatic heap sizing has a floor and stops adapting there. ### Large: 1–4Gi, high concurrency The defaults are close to right; raise concurrency to match the database. ```bash SKEIN_RUN_CONCURRENCY=25 PG_POOL_MAX=30 # ≥ concurrency, and skein opens THREE pools per instance SKEIN_REDIS_STREAM_MAXLEN=50000 SKEIN_STREAM_BUFFER_FRAMES=2048 PG_STATEMENT_TIMEOUT_MS=60000 ``` **Budget three pools per instance** (store + checkpointer + the per-thread execution claim), so 25 concurrency at `PG_POOL_MAX=30` is 90 connections against your database's cap, per replica. `skein start` warns at boot when concurrency exceeds `PG_POOL_MAX`, because the symptom otherwise is flat throughput with nothing pointing at the pool. The claim pool is the one that makes `PG_POOL_MAX ≥ concurrency` a hard requirement rather than advice: it holds one connection per _executing_ run for that run's whole duration (see [deploy.md](./deploy.md#connection-budget)), so a pool smaller than the concurrency simply caps how many runs can execute at once. ### The one that isn't about size `SKEIN_RUN_TIMEOUT_MS` is off by default and stays off unless you set it. A legitimate agent run takes minutes; a research or multi-step tool graph takes longer. A default here would turn slow-but-working into killed. Set it from your own graphs' worst honest case, not from a round number. ## Every knob Defaults are what skein uses when the variable is unset. "Small" and "Large" are the starting points above, not requirements. ### Runs | Variable | Default | Small | Large | What it bounds | | --------------------------------------------- | ------- | ----- | ----- | ----------------------------------------------------------------- | | `SKEIN_RUN_CONCURRENCY` / `N_JOBS_PER_WORKER` | 10 | 3 | 25 | Queued runs one instance executes at once (`--concurrency`, `-n`) | | `SKEIN_RUN_TIMEOUT_MS` | off | off | off | Abort a run executing longer than this (`--run-timeout`) | | `SKEIN_SHUTDOWN_GRACE_MS` | 5000 | 5000 | 15000 | How long `SIGTERM` lets in-flight runs finish before aborting | | `SKEIN_WEBHOOK_TIMEOUT_MS` | 5000 | 5000 | 5000 | One webhook POST. Sized to the shutdown budget — raise both | ### Streaming | Variable | Default | Small | Large | What it bounds | | ------------------------------------- | ------- | ----- | ----- | --------------------------------------------------------- | | `SKEIN_MEMORY_BUS_MAX_FRAMES_PER_RUN` | 10000 | 2000 | 10000 | Frames one run may buffer — in-memory bus only | | `SKEIN_MEMORY_BUS_MAX_RETAINED_RUNS` | 50 | 20 | 200 | Finished runs still replayable to a late `join` | | `SKEIN_REDIS_STREAM_MAXLEN` | 10000 | 2000 | 50000 | Approximate cap on a run's Redis stream (`0` = TTL only) | | `SKEIN_STREAM_BUFFER_FRAMES` | 512 | 128 | 2048 | Frames one slow subscriber may queue before being dropped | The socket write buffer has no knob: the socket's own high-water mark sets it, and nobody could size a second one sensibly. ### Storage | Variable | Default | Small | Large | What it bounds | | -------------------------- | ---------- | ----- | ----- | ------------------------------------------------------ | | `SKEIN_MAX_PAGE_SIZE` | 1000 | 200 | 1000 | Largest page any list/search returns | | `PG_POOL_MAX` | pg's 10 | 5 | 30 | Connections per pool — skein opens three per instance | | `PG_CONNECTION_TIMEOUT_MS` | 30000 | 30000 | 30000 | Waiting for a free connection (`0` = wait forever) | | `PG_IDLE_TIMEOUT_MS` | pg's 10000 | 10000 | 30000 | How long an unused pooled client is kept | | `PG_STATEMENT_TIMEOUT_MS` | 30000 | 15000 | 60000 | One statement server-side (`0` = off). Not per request | ### Diagnostics | Variable | Default | What it does | | ------------------------- | ------- | ------------------------------------------------------------------ | | `SKEIN_HEAP_WARN_PERCENT` | 85 | Warn above this % of the heap limit (`0` disables the monitor) | | `SKEIN_HEAP_SAMPLE_MS` | 30000 | How often the heap monitor samples | | `SKEIN_REQUEST_LOG` | see doc | A line per HTTP request. On for `skein dev`, off for `skein start` | Most of these can also be set in code — `worker: { maxConcurrency, shutdownGraceMs }` on any adapter, `embedPostgresGraphs`' options, `runTimeoutMs` on the deps, or a store constructor. Where a code path exists, the environment is still read **and validated** even when you pass the option, so a typo fails at boot rather than sitting unnoticed in a deployment that also passes the value. Four are environment-only: `SKEIN_WEBHOOK_TIMEOUT_MS`, `SKEIN_REQUEST_LOG` (or the CLI flag), and the two Redis stream bounds — setting those in code means constructing `RedisRunEventBus` yourself. The webhook timeout and the request-log switch also _fall back_ on a malformed value rather than throwing: both sit on paths where refusing to boot would be the more damaging failure. ## Streaming: backpressure, drops, and recovery **Backpressure.** The SSE write loop waits for the socket to drain before pulling the next frame, so a slow client's unwritten bytes stay in the socket's own buffer (~64KB) instead of accumulating in the process. Measured: a slow consumer's per-connection buffer went from ~1.26MB (the whole stream) to a constant ~67KB; 100 slow streams went from 125.5MB to 6.5MB, with no change to throughput or p99 and no frames lost. The graph does not slow down — the bus decouples it from the socket. **Drops.** Backpressure moves the problem rather than solving it: the frames queue in the bus instead. So the bus is bounded too, and a subscriber that falls behind the bound has its stream **ended** rather than being handed a silent gap. **Recovery.** An ended stream is not a lost run. Reconnect with `Last-Event-ID` and skein replays from that point — from the in-memory buffer, or from the Redis stream, whichever bus you run. The LangGraph SDK's `joinStream` does this for you. What you cannot recover is a frame that has aged out of `SKEIN_REDIS_STREAM_MAXLEN` or been evicted from the in-memory buffer, which is why those bounds are the ones to raise if reconnecting clients see gaps. **Retention after a run ends.** A finished run's frames stay replayable so a late `join` still works — for `SKEIN_MEMORY_BUS_MAX_RETAINED_RUNS` runs on the in-memory bus, or until the stream's TTL on Redis. Beyond that, joining a finished run completes immediately with no frames rather than hanging. See [streaming.md](./streaming.md) for the wire format and [runs-and-redis.md](./runs-and-redis.md) for what a frame costs in Redis. ## Query bounds Every list and search path is bounded — **including when the caller passes no `limit` at all**. Before that, one `POST /threads/search` with an empty body pulled an entire table into the heap and then serialized it into a single response string. - A client-supplied `limit` above 1000 is rejected. - An absent `limit` means the first `SKEIN_MAX_PAGE_SIZE` rows, not all of them. - `POST /threads/{id}/history` returns 100 checkpoints by default, capped at 1000, and pages with `before`. Each element is a checkpoint's whole graph state, so this is bounded harder than a row-based page — and separately from `SKEIN_MAX_PAGE_SIZE`, since history comes from the checkpointer rather than the store. - `GET /threads/{thread_id}/runs` and `POST /store/namespaces` page too, defaulting to **100** rows (`limit`/`offset`; a query `limit` above 1000 is clamped, not rejected). 100 matches what the LangGraph SDK sends for `store.listNamespaces`. `POST /store/namespaces` also applies the driver's `SKEIN_MAX_PAGE_SIZE` bound — it was the one list path that escaped it, and an `offset` with no `limit` used to be ignored outright on Postgres, silently answering from row 0. - A **wildcard** namespace prefix or a `suffix` cannot use the slice-equality form and is matched per-position instead. Neither form is index-backed: `store_items`' only relevant index is its `(namespace, key)` primary key, and Postgres will not derive a range scan from a slice expression, so both were already sequential scans. `filter` is pushed into the `WHERE` clause on both search paths (including the pgvector one), so the page is the top-_n_ of the **matched** set rather than the matched subset of a page. - Assistant search reports the unpaginated match count in `x-pagination-total`. Other collections do not: a total costs a second query, and for those it would be a count over exactly the rows the bound exists to avoid touching. Page them until they return fewer rows than you asked for. With auth configured, the ownership filter is pushed into the driver query, so a tenant's search is an indexed lookup rather than a full read filtered in JS. Details in [storage.md](./storage.md) and [agent-protocol.md](./agent-protocol.md#authentication--authorization). ## Triage: symptom → knob | Symptom | Likely cause | What to do | | -------------------------------------------------------- | ------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Restarts with no stack, no log line, nothing in metrics | OOM kill | Check the boot warning; below ~345Mi see [heap sizing](./deploy.md#heap-size-vs-the-containers-memory-limit). Then lower `SKEIN_RUN_CONCURRENCY` and the bus bounds. | | `heap pressure` warning, high `runs in flight` | Too much work at once | Lower `SKEIN_RUN_CONCURRENCY`, or scale out. | | `heap pressure` warning, high `buffered frames` | A slow SSE consumer | Lower `SKEIN_MEMORY_BUS_MAX_FRAMES_PER_RUN`, or move to Redis so frames live outside the process. | | `heap pressure` warning, neither, across episodes | A genuine leak | Capture a heap snapshot. Please open an issue. | | Throughput flat, adding concurrency does nothing | Runs queuing on the Postgres pool | Raise `PG_POOL_MAX` to ≥ concurrency (×2 for both pools). `skein start` warns about this at boot. | | Requests hang, then fail late | Unreachable or wedged database | `PG_CONNECTION_TIMEOUT_MS` bounds the wait; `PG_STATEMENT_TIMEOUT_MS` bounds one query. | | A query errors with `57014` | Statement timeout | The query is scanning something. Deep `OFFSET`, an unindexed `values` filter, or a very large checkpoint history — see [deploy.md](./deploy.md). Raise the timeout only after checking which. | | Streaming clients see gaps after reconnecting | Frames aged out before the reconnect | Raise `SKEIN_REDIS_STREAM_MAXLEN` (or the memory bus's frame bound). | | A stream ends early under load, on **Redis** | The subscriber's mailbox overflowed | Raise `SKEIN_STREAM_BUFFER_FRAMES`, or fix the consumer. `Last-Event-ID` recovers it from the durable stream. | | A stream ends early under load, on the **in-memory bus** | Frames were evicted out from under the subscriber | Raise `SKEIN_MEMORY_BUS_MAX_FRAMES_PER_RUN`. Reconnecting resumes with a gap — the buffer _is_ the replay log. `SKEIN_STREAM_BUFFER_FRAMES` does nothing here. | | Searches return fewer rows than expected | The page bound | Expected — page with `offset`. Raise `SKEIN_MAX_PAGE_SIZE` only if you know the rows are small. | | One thread's runs all stall behind each other | A run holding the thread lock | Runs on a thread are serialized by design. If it is a hung graph, set `SKEIN_RUN_TIMEOUT_MS`. | | Shutdown takes ~8s and kills in-flight work | The drain window | Raise `SKEIN_SHUTDOWN_GRACE_MS`, and your platform's termination grace with it. | ## Measuring it yourself Measure your application's graph with representative inputs and the intended storage backend. Record time to first frame, total duration, throughput, queue delay, and RSS before changing a limit. Use the same workload after the change and confirm final state and error behavior still match. Follow [Profiling your application](./profiling.md) for commands that run against an installed project. For a memory issue, compare retained memory after the same idle period, not just the peak during a run. Increase client count gradually; buffers should level off at the configured limits. Test slow readers separately from ordinary SDK clients, which may drain the socket before your application reads an event. ## Thread dispatch measurements Dispatch timings depend on your graph, backend, and workload. For your deployment, compare wait time for independent threads while another thread has a backlog. Use [queue configuration](./runs-and-redis.md#dispatch-by-thread) and the [pool budget](./deploy.md#connection-budget) to interpret the result. Historical repository measurements and benchmark commands are preserved in the [contributor testing guide](https://github.com/skein-js/skein-js/blob/main/docs/testing.md#performance-benchmark-evidence). --- # Profiling and performance tuning Skein Profile the skein-js application you deploy when runs are slow, memory keeps growing, or throughput stops improving. Start with one reproducible symptom and measure before changing configuration. ## The four tools and the tuning loop 1. Record the symptom and target: for example, time to first response, completion time, or memory after a batch of runs. 2. Reproduce it in a test deployment using your graph, storage, and representative inputs. Use controlled model/tool responses first, then repeat with the real dependencies. 3. Record a baseline after warm-up. Measure the client and server separately so a saturated load generator is not mistaken for a slow server. 4. Capture a CPU profile for slow execution, or compare heap profiles for retained memory. 5. Change one setting or application behavior, repeat the same workload, and run your [project tests](./testing-your-project.md) to check that output and state remain correct. Use [observability](./observability.md) to distinguish graph/model latency from queue delay and storage waits. The [symptom-to-setting table](./performance.md#triage-symptom-knob) helps choose a configuration change once you know the bottleneck. ## Reading the numbers Latency percentiles answer different questions. p50 is the ordinary request. p95/p99 are the tail your users notice during queueing, GC, pool waits, or a noisy dependency. For streams, record both **time to first frame** and total duration; an agent that starts responding in 150 ms and finishes in 20 seconds feels different from one silent for 10 seconds. Throughput is completed requests/second or frames/second. It means little without concurrency and saturation. Increase concurrency until throughput stops rising; then inspect CPU, event-loop delay, Postgres pool waits, Redis, and the load generator. The first saturated component is the limit. Memory has several layers: - **JS heap used** is live/recent JavaScript data. - **Heap capacity/limit** is what the collector/runtime reserved or may grow to; it is not usage. - **RSS** is resident process memory: JS heap plus runtime, native clients, stacks, code, and buffers. - **Native/external memory** includes socket buffers and runtime allocations outside the JS heap. - A healthy GC graph is a sawtooth. A leak has a rising _post-GC floor_ after the same work becomes unreachable. A bounded cache rises and then plateaus. For CPU profiles, **self time** is time in the function itself. **Total time** includes callees. High self time points at the hot implementation; high total but low self time says to descend into its children. Wide flame-graph boxes consumed more samples. GC width means allocation pressure, not necessarily that the collector itself is defective. ## Profile your installed server {#skeins-cross-runtime-benchmark} Run the following commands from your application directory, with `skein-js` installed locally. Build the production artifact and make its storage services and environment available as you would for your normal `skein start` command. Append your usual storage flags to the profiler commands if needed. If you embed skein-js, replace `node_modules/skein-js/dist/index.js start --runtime node` with your own built server entry point. Send representative traffic from a separate process while the profile is recording. Stop the server cleanly so the profile is written. Confirm that your output directory contains a profile before opening it in your runtime's tools. The repository's own benchmark fixtures require a skein-js checkout. Their commands and historical measurements are [contributor reference](https://github.com/skein-js/skein-js/blob/main/docs/testing.md#repository-profiling-fixtures), not a prerequisite for profiling your application. ## Node recipes Profile the built production entry, send representative traffic from another process, then terminate it cleanly so profiles flush: ```bash mkdir -p .profiles/node/cpu .profiles/node/heap node --cpu-prof --cpu-prof-dir=.profiles/node/cpu \ node_modules/skein-js/dist/index.js start --runtime node node --heap-prof --heap-prof-dir=.profiles/node/heap \ node_modules/skein-js/dist/index.js start --runtime node node --trace-gc node_modules/skein-js/dist/index.js start --runtime node node --inspect node_modules/skein-js/dist/index.js start --runtime node ``` Load `.cpuprofile` in Chrome DevTools’ Performance panel. Load `.heapprofile` in Memory. In an inspector session, take two heap snapshots only after the same forced-idle/GC point, repeat the workload between them, and compare retained size and retaining paths. Snapshots pause the process and can temporarily require roughly another heap’s worth of memory; capture them on a replica you can remove from traffic. ## Bun recipes ```bash mkdir -p .profiles/bun/cpu .profiles/bun/heap bun --cpu-prof --cpu-prof-dir=.profiles/bun/cpu \ node_modules/skein-js/dist/index.js start --runtime bun bun --heap-prof --heap-prof-dir=.profiles/bun/heap \ node_modules/skein-js/dist/index.js start --runtime bun ``` Bun has separate JavaScriptCore and native heaps. For an instrumented experiment, record `heapStats()` from `bun:jsc` for JS objects, `Bun.memoryUsage()` for Bun/native allocations, and RSS. A flat JS heap with rising RSS points away from retained JS objects and toward native/socket buffers. Bun’s heap profiler can also emit Markdown summaries; follow the options supported by the pinned Bun version used by the artifact. ## Deno recipes The production launcher needs explicit access to the artifact, environment, network, system data, and native dependencies. Deno’s profiler can produce the raw profile, a Markdown report, and an interactive flamegraph together: ```bash mkdir -p .profiles/deno/cpu deno run --allow-net --allow-env --allow-read=. --allow-sys --allow-ffi \ --cpu-prof --cpu-prof-dir=.profiles/deno/cpu \ --cpu-prof-md --cpu-prof-flamegraph \ node_modules/skein-js/dist/index.js start --runtime deno deno run --inspect --allow-net --allow-env --allow-read=. --allow-sys --allow-ffi \ node_modules/skein-js/dist/index.js start --runtime deno ``` Open the SVG directly or load `.cpuprofile` in DevTools. Deno reports transpiled JavaScript line numbers in CPU profiles, so use function names and sourcemaps to get back to TypeScript. ## Diagnostic checks {#five-diagnostic-exercises} ### 1. SSE backpressure Compare ordinary clients with deliberately slow readers against your test server. Watch RSS and connection count at increasing concurrency. Memory should settle within your configured limits; compare received output and final state to ensure a lower memory reading is not lost work. ### 2. Event-bus retention Complete a repeatable workload, disconnect clients, and compare memory after the same idle period. Configured retention can keep a bounded amount of history. A rising retained baseline across repeated workloads warrants a heap-profile comparison; see [streaming bounds](./performance.md#streaming). ### 3. Serialization cost Compare `stream_mode: "values"` and `"updates"` with the same graph and inputs. If full-state events dominate CPU and allocations, reduce event volume or payload size. Verify that your UI still receives what it needs and that final graph state matches. ### 4. Postgres pool saturation Increase concurrency gradually while observing database connections and queue delay. When throughput flattens, check the [connection budget](./deploy.md#connection-budget) before raising concurrency. Each instance has multiple pools; database capacity must cover all replicas. ### 5. Telemetry shutdown leak Stop a busy test server gracefully and check that it exits within your configured shutdown budget. Confirm that expected terminal run state and telemetry arrive. If an exporter delays exit, inspect its flush behavior and your host's shutdown order; do not infer a completed run from process exit alone. ## Benchmark traps - Warm-up/JIT: keep cold start as its own metric; do not mix it into steady state. - Coordinated omission: a closed-loop generator waits during stalls and under-samples the stall. - Noisy neighbours and CPU turbo: pin container resources and repeat. - A weak load generator: if its CPU is full, you measured the client. This is especially easy with a fast native Bun server. - Model/network variance: deterministic graphs establish runtime overhead; model-backed tests answer a different question. - Different hardware, regions, images, database sizes, or TLS paths: not a runtime comparison. - Average-only reporting: averages hide queue and GC tails. - Optimizing without a negative verification: a benchmark win can be dropped frames or skipped work. ## Comparing Skein with LangGraph Platform Treat Platform as a black box. Use the same graph, SDK client, client region, payloads, concurrency, warm-up, and observation window. Publish raw results and methodology. A “better” claim requires full protocol correctness plus a reproducible win in p99, memory, throughput, cold start, or operating cost, with no material regression in the others. If infrastructure cannot be made identical, label the result as an end-to-end deployment comparison rather than a runtime benchmark. Official references: [Node CLI profiling](https://nodejs.org/api/cli.html#--cpu-prof), [Bun benchmarking and profiling](https://bun.sh/docs/project/benchmarking), and [Deno CPU profiling](https://docs.deno.com/runtime/fundamentals/cpu_profiling/). --- # Deploy on Google Cloud Run Cloud Run runs your container, scales it, and bills per request — a good fit for skein, with **one sharp edge**: by default Cloud Run throttles a container's CPU to near-zero between requests, and skein's background runs do their work _after_ the request that created them has returned. Get that setting wrong and inline runs work perfectly while background runs mysteriously never finish. Everything platform-agnostic — environment variables, pool sizing, probes, scaling caveats — is in [deploy.md](./deploy.md). ## Before you start You need the `gcloud` CLI authenticated, a project with billing enabled, and Docker. Enable the APIs once: ```bash export PROJECT_ID=your-project REGION=us-central1 REPO=skein SERVICE=skein-app gcloud config set project $PROJECT_ID gcloud services enable run.googleapis.com artifactregistry.googleapis.com secretmanager.googleapis.com ``` ## 1. Build and push the image ```bash # Cloud Run runs x86. skein build doesn't pass --platform, so set this on Apple Silicon. export DOCKER_DEFAULT_PLATFORM=linux/amd64 export IMAGE=$REGION-docker.pkg.dev/$PROJECT_ID/$REPO/$SERVICE:v1 gcloud artifacts repositories create $REPO --repository-format=docker --location=$REGION gcloud auth configure-docker $REGION-docker.pkg.dev skein build -t $SERVICE docker tag $SERVICE $IMAGE docker push $IMAGE ``` ## 2. Provision Postgres + Redis Anything reachable works. Two shapes are common: **Managed outside GCP** (Neon, Supabase, Upstash, Redis Cloud) — simplest, no VPC needed, and the connection strings work as-is. Use the **direct** Postgres endpoint, not a pooled one ([why](./deploy.md#tls-to-the-database)). **GCP-native** — Cloud SQL and Memorystore: - **Cloud SQL** attaches over a Unix socket, which needs no TLS configuration. `POSTGRES_URI` is handed straight to `pg`, so the socket form works: ```text postgresql://USER:PASSWORD@/DATABASE?host=/cloudsql/PROJECT:REGION:INSTANCE ``` Add `--add-cloudsql-instances=PROJECT:REGION:INSTANCE` to the deploy below. - **Memorystore** is only reachable from inside your VPC, so the service needs Direct VPC egress or a Serverless VPC Access connector (`--network`/`--subnet`, or `--vpc-connector`). Store both URIs in Secret Manager rather than passing them as plain environment variables — `gcloud run services describe` prints env vars in cleartext: ```bash printf '%s' "$POSTGRES_URI" | gcloud secrets create skein-postgres-uri --data-file=- printf '%s' "$REDIS_URI" | gcloud secrets create skein-redis-uri --data-file=- ``` Grant the service's runtime service account `roles/secretmanager.secretAccessor`. ## 3. Deploy ```bash gcloud run deploy $SERVICE \ --image=$IMAGE \ --region=$REGION \ --port=8123 \ --set-secrets=POSTGRES_URI=skein-postgres-uri:latest,REDIS_URI=skein-redis-uri:latest \ --set-env-vars=PG_POOL_MAX=5,SKEIN_RUN_CONCURRENCY=5 \ --no-cpu-throttling \ --min-instances=1 \ --max-instances=3 \ --cpu=1 --memory=1Gi \ --timeout=3600 \ --no-allow-unauthenticated ``` The flags that matter, and why: | Flag | Why | | ------------------------------------------- | ------------------------------------------------------------------------------------------------------- | | `--port=8123` | Cloud Run injects `PORT`, and skein binds it. This tells Cloud Run which port to _send_ traffic to. | | `--no-cpu-throttling` + `--min-instances=1` | **Required for background runs.** See [below](#background-runs-need-cpu-outside-requests). | | `--timeout=3600` | The ceiling on a streaming SSE run; skein imposes none of its own. | | `--no-allow-unauthenticated` | skein's auth is off by default — see [the warning](./deploy.md#5-auth--read-this-before-you-expose-it). | | `PG_POOL_MAX=5` | Two pools per instance × instances vs. your database's cap ([budget](./deploy.md#connection-budget)). | Cloud Run's built-in health checking uses the container port; you can also declare an explicit startup probe on `GET /ok` with a ~30s failure budget, since boot runs migrations before listening. ## 4. Verify With `--no-allow-unauthenticated`, reach the service through an authenticated tunnel rather than opening it up: ```bash gcloud run services proxy $SERVICE --region=$REGION --port=8123 ``` Then run the [verification sequence](./deploy.md#verify-a-deployment) against `http://localhost:8123`. Step 4 — the background run — is the one that proves the CPU settings are right. ## Cloud Run caveats ### Background runs need CPU outside requests This is the big one. `POST /threads/{id}/runs` enqueues a run and responds immediately; the worker executes it afterwards. Under Cloud Run's default _CPU-allocated-during-requests_ model, the instance's CPU is throttled to near-zero the moment the response is sent, so that run stops making progress — and with `--min-instances=0` the instance may be shut down entirely. Set **`--no-cpu-throttling`** (CPU always allocated) and **`--min-instances=1`**. This costs more — you're paying for an always-on vCPU rather than per-request — but it's what makes background runs, webhooks and cron-shaped work behave. If you only use inline runs (`/runs/wait`, `/runs/stream`), the work happens _during_ the request and the defaults are fine. Scale to zero freely. ### Request timeout vs. streaming Default 300s, max 3600s. A long SSE run is a long request. skein sends no heartbeat frame, so a stream that goes quiet still counts against the timeout — set `--timeout` to cover your worst case. Don't put Cloud CDN in front of the streaming routes; it buffers. ### Connections vs. autoscaling Each instance opens two Postgres pools, so a burst to `--max-instances=10` with `PG_POOL_MAX=10` wants 200 connections. Cap `--max-instances`, keep `PG_POOL_MAX` small, or front Cloud SQL with the Auth Proxy or a pooler. ### Shutdown Cloud Run sends `SIGTERM` and SIGKILLs 10 seconds later by default. skein drains in-flight runs for 5s, aborts the rest so they land in a terminal status, and exits — comfortably inside that window. If you raise `SKEIN_SHUTDOWN_GRACE_MS`, raise Cloud Run's termination grace period to match, or you'll be killed mid-drain. Note that Cloud Run has no `init` process. The image is fine as-is, but if your graphs spawn child processes you'll want `tini` — add it via `dockerfile_lines` in `langgraph.json`. ### Multi-instance semantics With `--max-instances` above 1 and Postgres + Redis configured, cross-instance cancellation and the one-run-per-thread guard both hold — no session affinity needed. See [Scaling past one instance](./deploy.md#scaling-past-one-instance) for what that costs in Postgres connections. --- # Bundling skein Most people never read this page: if you deploy the `skein build` image, or run `next start` with skein mounted as a route handler, bundling is already handled. You need this page when **you** own the bundler — an rspack/webpack server build, an esbuild bundle for Lambda, a Next.js app with an unusual config — and skein is inside its module graph. The short version: **skein is ESM-only, and everything except the CLI bundles cleanly.** Two packages must stay external, and only if you use the `langgraph.json` on-ramp. ## skein is ESM-only Every `@skein-js/*` package is `"type": "module"` and ships a single ESM entry. There is no CommonJS build and there won't be one. That does **not** mean you can't `require()` it. Each library package exposes a `default` export condition, so Node's `require(esm)` resolves it: ```js const { embedPostgresGraphs } = require("@skein-js/runtime"); // works on Node 20.19+ / 22.12+ ``` `require(esm)` landed unflagged in **Node 20.19** and **22.12**. On an older Node you get `ERR_REQUIRE_ESM` telling you to use `import()` instead — which is the honest answer, and what a CJS-emitting bundler should be configured to do. Plain `import` works on any Node ≥ 20. > Before 0.10.0 the packages declared only `types` + `import` conditions, so `require()` failed with > `ERR_PACKAGE_PATH_NOT_EXPORTED` — which reads like the package is broken rather than like a module > format mismatch. If you're pinned below 0.10.0, that's the fix. > > The `skein-js` CLI keeps the old shape deliberately: it has no exports, and its entry point runs a > command as a side effect. `require()`-ing it should fail. **`require()` does not tree-shake.** `require("@skein-js/runtime")` eagerly loads `@skein-js/config` and therefore `@langchain/langgraph-api`, even if you only use `embedPostgresGraphs` and never touch a `langgraph.json`. It works, it just costs ~0.5s of cold start. Prefer `import` where your toolchain allows it. ## What must stay external | Package | Bundle it? | Why | | -------------------------- | ------------------------- | --------------------------------------------------------------------------------------- | | `@langchain/langgraph-api` | **No** — mark external | skein's graph loader `import()`s a path computed at runtime; no bundler can follow that | | `@typescript/vfs` | **No** — mark external | pulled in by the same loader | | `skein-js` (the CLI) | **Never** | a `bin` with no exports and top-level await; importing it runs a command | | every other `@skein-js/*` | **Yes** — bundles cleanly | including `@skein-js/storage-postgres` as of 0.10.0 | The first two only matter if you point skein at a `langgraph.json` (`buildRuntime`, the CLI, the `{ config }` form of any adapter). If you embed graphs in code — `embedPostgresGraphs`, `embedInMemoryGraphs`, the `{ deps }` form — nothing reaches them: `@langchain/langgraph-api` is loaded with `await import()` at the two points that genuinely need it (analysing a graph's schema, and adapting a user's `Auth` instance), and the in-memory runtime loader is behind a dynamic import on the `{ config }` branch. So on an embedded path they are never in the module graph at all, rather than being present and merely tree-shakeable. One trade-off worth knowing: because `@langchain/langgraph-api` is now loaded on demand, a bundling mistake around it (the `serverExternalPackages` config below) surfaces when something first asks for a graph schema rather than at startup. The container boots and passes its probes, and `GET /assistants/{id}/schemas` returns a 500. Bake your schemas at build time (`skein build` does) and the path is never taken at all. Two consequences for the public API, both of which exist to keep that graph clean: - `SkeinConfigError` is importable from `@skein-js/config/errors` as well as the root, and internal code uses the subpath. The root barrel is the `langgraph.json` loader. - `readLanggraphDevState` / `loadSnapshotIntoStore` / `describeSnapshot` live at `@skein-js/server-kit/dev`, not on the root barrel — they carry `superjson` and `node:fs/promises`, and only `skein dev` / `skein import` call them. They are deliberately **not** re-exported from the root or from `@skein-js/express`: a re-export is still a static import, which would undo the split. ## Copy-paste configs **Next.js** (`next.config.mjs`): ```js export default { serverExternalPackages: ["@langchain/langgraph-api", "@typescript/vfs"], }; ``` **webpack / rspack** (server build): ```js export default { target: "node", externals: [ { "@langchain/langgraph-api": "commonjs @langchain/langgraph-api" }, { "@typescript/vfs": "commonjs @typescript/vfs" }, ], }; ``` **esbuild**: ```bash esbuild server.ts --bundle --platform=node --format=esm \ --external:@langchain/langgraph-api --external:@typescript/vfs ``` Prefer `--format=esm` if you can. With `--format=cjs`, anything the bundle `require()`s at runtime still needs Node 20.19+, per above. ## The "Critical dependency" warning webpack and rspack emit this when they meet skein's graph loader: ```text Critical dependency: the request of a dependency is an expression ``` It's expected and harmless — that expression is the `import()` of your graph module, resolved from `langgraph.json` at runtime. Externalizing `@langchain/langgraph-api` removes most of it; to silence the rest: ```js // next.config.mjs export default { serverExternalPackages: ["@langchain/langgraph-api", "@typescript/vfs"], webpack: (config) => { config.ignoreWarnings = [ ...(config.ignoreWarnings ?? []), { message: /Critical dependency: the request of a dependency is an expression/ }, ]; return config; }, }; ``` ## Postgres migrations are compiled in `@skein-js/storage-postgres` needs **no skein-side externals**. Its schema migrations ship as string constants inside `dist/index.js`, so the package makes no filesystem access at runtime — it imports only `node:crypto`, `pg`, and `@skein-js/core`. (`pg` itself has one wrinkle — see below.) You do not need to copy a separate migrations directory into your bundle. Migrations run at startup. If an older bundled version fails to find its migration files, upgrade or externalize `@skein-js/storage-postgres` and keep it installed at runtime. See [storage](./storage.md) for upgrades. ### One caveat, from `pg` rather than skein `pg` has an optional native binding it reaches for at runtime: `pg/lib/native/client.js` does `require('pg-native')`, and `pg-native` is an optional peer that is normally not installed. pg wraps that call in a `try`/`catch` specifically so bundlers tolerate it, and **esbuild does** — it leaves a runtime `require` and emits nothing. **webpack and rspack are stricter** and report: ```text Module not found: Can't resolve 'pg-native' ``` You do not want the native binding; tell the bundler to ignore it: ```js // webpack / rspack import webpack from "webpack"; export default { plugins: [new webpack.IgnorePlugin({ resourceRegExp: /^pg-native$/ })], }; ``` Next.js keeps `pg` on its built-in server-externals list, so this never surfaces there. ## What `skein build` inlines vs. externalizes The section above is about bundling **skein**. This one is about the bundler skein itself runs: `skein build` compiles your graphs (plus auth, custom embed, custom telemetry sinks) into `.skein/build`, and the split it makes there is the reason the production image is small and the monorepo case works at all. **Inlined into the artifact** — your own source, including anything reached through a `tsconfig` `paths` alias or a workspace link (`@myorg/js`, the Nx/Turborepo/pnpm-workspace pattern). Those files exist nowhere a package manager could install them from, so resolving them once on the build host is what dissolves the "my Docker build context doesn't contain my monorepo" problem. **Left external and pinned** — every published `node_modules` package. `skein build` records each one at the exact version installed on the build host and writes it into the artifact's `package.json`, which the image installs with `npm install --omit=dev`. Externalizing is not a limitation to work around; it is load-bearing: - **One copy of each library.** The image installs `skein-js`, which brings `@langchain/langgraph` and `@langchain/core`, and that runtime is what imports your graph bundle. Inline `@langchain/core` into the graph and there are two copies: `instanceof BaseMessage` starts failing, and config/callbacks propagate through a different `AsyncLocalStorage` than the one the runtime reads. - **Native addons can't be inlined.** `pg-native`, `sharp`, `better-sqlite3` and friends are platform binaries; they have to be installed for the image's platform. - **Package-relative asset reads survive.** A bundler rewrites `import.meta.url`/`__dirname` to the output location, which breaks packages that load workers, wasm, or data files from beside themselves (pdfjs, `tiktoken`, …) — the same hazard that made skein [compile its own SQL in](#postgres-migrations-are-compiled-in). - **Cheaper rebuilds.** `COPY package.json` + install is a cached Docker layer; a graph edit re-ships only the bundle. The one thing a bundler structurally cannot see is a package imported **by name at runtime** — `initChatModel` doing `import("@langchain/" + provider)`, a plugin loaded from config. Those never appear in the module graph, so declare them under `dependencies` in `langgraph.json`: ```json { "graphs": { "agent": "./src/graph.ts:graph" }, "dependencies": ["@langchain/openai"] } ``` (skein pins the packages behind a declared `store.index.embed` provider and a declared `telemetry` provider for you — the field is for the ones only your code knows about.) `skein build` fails on the **host** if the artifact would ship an import it does not install, so a missing pin is a build-time error with a package name in it rather than an `ERR_MODULE_NOT_FOUND` from inside `docker build`. ## See also - [deploy.md](./deploy.md) — deploying the built image, env vars, probes, scaling - [embedding.md](./embedding.md) — the in-code on-ramp, which avoids the graph loader entirely - [storage.md](./storage.md) — the Postgres driver and its schema