How I made AI chat resumable

DANYLO KACHUR · 2026-07-06 · 10 min read

This is the story of one chat window, and the stack I kept rebuilding underneath it. All I wanted was an AI assistant you could reload without losing your place. It took a couple of discarded architectures, a month of freezes, and — eventually — forking a database server.

close, but no standard

The goal: a chat that outlives the session. Close the tab, reload an hour later, and you're back exactly where you left off — mid-token if that's where you were.

The default tool for this is the Vercel AI SDK (v6 at the time), and I started there. The awkwardness wasn't durability, at first — it was the agent itself. It has no primitive for a graph interrupt, multi-agent runs, or the other durable, multi-agent specifics I needed.

Everything in that gap rides in as a generic data-* part instead — a genuinely expandable escape hatch, but it adds up. My client-side switch ended up with a dozen bespoke data-* cases: one for a pending message, one for a graph interrupt, one for a subagent snapshot, one for a renamed conversation, one for compaction, and so on, each hand-parsed and hand-rendered. Workable. Proprietary in practice, if not in name.

Durability made it worse. The message list lives on the client, the transport is ephemeral SSE, and resumability is a bolt-on: a Redis-backed resumable-stream that replays bytes into a client still doing the bookkeeping by hand. My first cut had that shape: a Redis stream read over SSE. Redis is ephemeral, so the durable copy of the conversation lived in the Postgres checkpoint, and resuming meant reconciling the two by hand. The durable thing sat beside the state instead of being it.

two sources of truth is zero sources of truth.

So I switched to AG-UI: an event protocol built for exactly the agent shape the AI SDK made awkward — text delta, tool call, state patch, run finished, each a typed event instead of a part you reconstruct meaning from. It's transport-agnostic and supported across most agent frameworks, which is what let me put assistant-ui on screen with its AG-UI runtime underneath. A better primitive for the agent — but still silent on durability, so I swapped Redis for durable streams: an append-only log over HTTP that outlives the connection, the tab, and the process that wrote it. Reconnecting stops being an emergency; it's just the read path now.

And here's what took me weeks to see: a durable pipe is still just a pipe. It carries events; it isn't state. There was no source of truth — I was replaying the log and rebuilding the conversation by hand on the client, every time. A nicer transport, the same bookkeeping. That gap cost me a month.

the final boss

Most of that month lives on a branch I named dev-3472-final-boss-of-agui. The symptom never changed: the chat froze — typing dots forever, on a conversation that had already finished or died.

Here's the cause, in one line: a durable log doesn't close when its writer crashes. So silence is silence. A dead agent and a slow one send the exact same signal — nothing — and the client can't tell them apart.

I tried to make the client tell them apart anyway. A watchdog that gave up after thirty seconds of quiet. Reconnects on tab focus, on network changes, on a prayer. It worked, right up until it declared a perfectly healthy backend dead.

Turns out this is a known problem: distributed systems calls it a failure detector, telling live from dead by watching the wire, and there's a proof that a perfect one is impossible over an asynchronous network. I didn't know that proof existed. I just re-derived it, production incident by production incident, at 2am.

a closed stream is not a dead producer. an open one is not a live producer either.

Every fix uncovered another version of the same gap: reads 401ing on an expired token, resumed tabs 404ing on a TTL-evicted stream (#7310), a subagent swallowing an error into ninety seconds of silence, a sandbox stuck for two days (#8008). One root cause — I was asking the client to infer liveness from a transport that doesn't carry it.

The fix wasn't another heuristic. It was a rule: stop guessing, ask the source of truth — is this run still running? Which assumes there is a source of truth to ask. There wasn't. So I built one.

make the log the truth

New rule: the log is the source of truth, and state is something you derive from it — state = fold(log). So I needed something to fold into: a normalized, queryable, reactive projection of the log. I'd been reaching for TanStack DB for exactly this shape of problem — a client-side store that holds normalized collections in memory and keeps queries over them live via differential dataflow — a new event patches the one row it touches, not the whole result set, so updates land sub-millisecond. Two ways to point it at a durable log:

• a raw local collection — I own the events and fold them in by hand;
• Electric's StreamDB — a collection that syncs itself straight off a durable stream.

I took StreamDB: a primitive built on top of durable state — the exact shape I'd been hand-rolling. Normalized rows, last-writer-wins per row, a client that replays the log and never mutates by hand:

const db = createStreamDB({
  streamOptions: { url, params: { key: conversationId } }, // ?key=
  state: schema,                 // messages, chunks, runs
});
await db.preload();              // replay the log → rows
// last-writer-wins per row; the client never mutates, it folds

What is that fold actually doing? StreamDB routes each typed change event on the stream into a collection and keeps the rows caught up to the latest offset — an insert adds a row, an update replaces it, a delete drops it. And because it's a real collection, not just a replay buffer, you get a live query on top for free: a subagent's run is its own row, tagged with an agent id, and .where() narrows to one without re-folding anything:

Every freeze fix collapsed into a read. Is the run still going? Ask the log. The UI isn't a copy of the conversation, it's a window into it — the stream is the only source of truth.

the ui isn't where the truth lives. it's just where you look at it.

the read path: one conversation out of many

Then the read path, and the worst hack of the lot — the one that worked, which is what made it dangerous. One stream carried every conversation in a workspace, and the log had no notion of a key. So to show you one conversation, I read the whole log and kept the 2% that was yours:

// no keyed reads. want ONE conversation? read everything, keep 2%.
const all   = await readWholeStream(factoryUrl); // every frame
const state = foldEverything(all);           // keep one row
const mine  = state.messages.filter(m => m.threadId === conversationId);
// cold load = O(all conversations), every time.

Cold load scaled with the workspace, not with you. The next challenge was the opposite direction: multiple concurrent conversations, and the UX that requires. Give every conversation its own stream and switching conversations means switching StreamDB's URL — which remounts the collection and re-folds it from cold every time. It feels laggy and slow. One way out is a stable ref: keep a single StreamDB instance mounted and swap the URL underneath it, a kind of switchbox you plug in front of it. Doable, but a bit awkward — you're routing around the sync layer instead of using it. The way I did it: give the log an index. One stream, tagged per conversation on write, filtered by key on read, so switching conversations is just a different ?key= — no remount, no re-fold of anyone else's.

POST /log   Stream-Key: conv_7        # tag the write
GET  /log?key=conv_7                   # read ONLY conv_7 — no re-fold

That single index — key on write, filter on read — turns cold load and resume from "proportional to everything" into "proportional to your conversation."

the rust server had no key

One problem: the server I most wanted couldn't do it. The best durable-streams server is Electric's, in Rust, and it's genuinely kernel-speed — sendfile, splice, the wire format stored as-is on disk so a read is more or less a file range, around 860k appends a second. But every read scans the whole stream. No keys. Prisma's Bun server can filter by key, but it isn't native, and Bun still has no mature multi-core forking to lean on — no sharing a listener across processes the way the Rust server does, so it's single-process-bound in a way the Rust one isn't.

So I built rheoDS: keyed reads on the Rust server. A Stream-Key header on write, ?key= on read. Reads come from coalesced key spans, resident-cache-first, so the hot path rarely touches disk; writes go through a per-stream journal, fsynced at ack; live tails still work over long-poll and SSE. I benchmarked it against Bun on the same Linux kernel:

The 50× is the honest one. Not reading other people's conversations turns out to be a fantastic optimization.

the runtime, and the honest part

The last piece ties it together: react-durable-streams (private for now), a durable runtime for assistant-ui on its ExternalStore pattern, modeled on react-ag-ui. Using it is one hook:

import { AssistantRuntimeProvider, Thread } from "@assistant-ui/react";
import { useDurableStreamsRuntime } from "@assistant-ui/react-durable-streams";

function Chat({ conversationId }: { conversationId: string }) {
  // one keyed durable stream per conversation; messages = fold(log)
  const runtime = useDurableStreamsRuntime({
    url: "https://rheods.example/log",
    conversationId,            // → Stream-Key on write, ?key= on read
  });
  return (
    <AssistantRuntimeProvider runtime={runtime}>
      <Thread />
    </AssistantRuntimeProvider>
  );
}

And the inside is exactly the thesis — a thin fold over the keyed log, wired into assistant-ui's ExternalStore:

const db = createStreamDB({
  streamOptions: { url, params: { key: conversationId } }, // keyed read
  state: schema,                              // messages, chunks, runs
});

const runtime = useExternalStoreRuntime({
  messages:  assembleMessages(db.collections),        // → messages
  isRunning: db.collections.runs.get(RUN)?.status === "running",
  onNew:     (m) => db.actions.send(m),                 // append, keyed
});

Open the tab, fold the log, and you're where you left off. Mid-token, if that's where you were.

One tradeoff I'm still weighing: a conversation you've opened stays warm and switches instantly; one you haven't costs a keyed fetch. The old whole-stream hack, having already replayed everything, had cold conversations in hand too. I chose the key anyway.

Which is the payoff I wanted from that first pipe: durable, resumable chat as a primitive — a log you can trust, a fold you can replay, and typing dots that stop when the producer does.