Streaming & optimistic UI

In one line: a model can take seconds to answer, so don't make the interface wait — stream the reply token-by-token as it's generated, and render the user's own action optimistically the instant they take it.

What it is

Model-backed interfaces have a latency problem: a full response can take several seconds, and a UI that blocks on it feels broken. Two patterns hide that latency:

• Streaming — the model emits tokens incrementally, and you paint each one as it lands, so the user sees the answer forming instead of staring at a spinner.
• Optimistic UI — when the user sends a message or takes an action, you show the result immediately (their message bubble, a pending assistant placeholder) on the assumption it'll succeed, then reconcile with the real server response when it arrives.

Why it matters

Perceived performance is the product here. A streamed answer feels dramatically faster than the same answer delivered all at once a beat later, even though the total time is identical — because feedback starts immediately and the user can begin reading. Optimism does the same for input: the conversation never freezes between "I sent that" and "the server agreed."

The discipline is in the edges. Optimism is a bet, so you need a rollback path when the bet loses (the request failed), and streaming needs to announce itself to assistive tech without flooding it. Get those right and the UI feels instant and trustworthy.

See it

In streaming mode, reveal the reply token-by-token and watch the caret. Switch to optimistic mode and send: the user's message and a pending placeholder appear instantly, then resolve when the "server" responds.

How it works

Streaming. The transport (an SSE stream, a fetch ReadableStream, or an SDK's async iterator) yields chunks. You append each chunk to the in-progress message in state, so React re-renders with a little more text each tick. Wrap the growing text in an aria-live="polite" region so screen readers are kept current — but update the message in place rather than announcing every token, or you'll spam them.

Optimistic. On send, push the user's message and a pending assistant entry into state immediately, then fire the request. When it resolves, replace the pending entry with the real result (reconcile by id). If it rejects, remove the optimistic entry and surface a retry — the rollback. Throughout, keep the input enabled so the user is never blocked.

Build it

async function send(text: string) {
  const userMsg = { id: crypto.randomUUID(), role: "user", text };
  const pending = { id: crypto.randomUUID(), role: "assistant", text: "", pending: true };
 
  // Optimistic: show both instantly, before any network round-trip.
  setMessages((m) => [...m, userMsg, pending]);
 
  try {
    const stream = await chat(text); // async iterator of tokens
    for await (const token of stream) {
      // Append each token to the pending message as it arrives.
      setMessages((m) =>
        m.map((msg) => (msg.id === pending.id ? { ...msg, text: msg.text + token } : msg)),
      );
    }
    setMessages((m) =>
      m.map((msg) => (msg.id === pending.id ? { ...msg, pending: false } : msg)),
    );
  } catch {
    // Rollback: drop the optimistic placeholder and let the user retry.
    setMessages((m) => m.filter((msg) => msg.id !== pending.id));
    setError("Couldn't send. Try again.");
  }
}

Make it yours

• In streaming mode, notice you can start reading before the reply is finished — that's the whole win.
• In optimistic mode, the user bubble shows before the server replies; the placeholder is what gets reconciled.
• Imagine the request failing: the optimistic placeholder is removed and a retry appears — never a message that silently lies about succeeding.

Reproduce it with an LLM

Pitfalls & accessibility

Related

Further reading

• MDN — Using readable streams