Streaming response

Problem Context

• The system may emit output deltas, message events, tool-call events, citation events, moderation events, usage metadata, and final completion events at different times.
• Text, Markdown, tables, JSON, code, citations, and generated files may be invalid or incomplete until the stream closes cleanly.
• Users may want to start reading, stop when enough information arrives, continue after stopping, retry after network failure, copy partial output, or wait for final source verification.
• Streaming can happen inside a chat interface, side panel, document editor, code assistant, workflow run, voice transcript, or background job detail surface.
• Accessibility, reduced motion, mobile scroll, text selection, virtualized transcripts, and screen reader announcement rate all affect whether streaming is usable.

Selection Rules

• Choose streaming response when partial generated output is useful before the full answer is ready and the product can explain partial versus final state.
• Use chat interface when the design problem is the broader conversation transcript, history, turns, and composer rather than one active response lifecycle.
• Use progress bar when the system reports measurable operation completion instead of readable generated output.
• Use loading spinner when the wait is short and neither partial output nor measurable progress can be shown.
• Use feed when users browse finished chronological items rather than watch one answer being generated.
• Delay apply, publish, commit, structured export, final citation trust, and high-risk copy actions until the stream reaches a verified terminal state.
• Show Stop generation when stopping is technically possible and the partial answer can be preserved or explicitly discarded.
• Show Continue or Retry when a stream stops, stalls, times out, or fails while useful partial output remains.
• Use a pending source or pending tool state when text arrives before citations, retrieval, function output, or safety checks.
• Do not stream raw chunks if they will expose unsafe intermediate content, broken syntax, misleading partial conclusions, or private tool data.

Required States

• Queued or receiving state before first output arrives.
• First output state that confirms generation has started.
• Active streaming state with partial-answer label, stop control, and stable output region.
• Pending citation, pending tool call, pending moderation, and pending structured-validation states.
• Stalled or slow-stream state with elapsed status and recovery threshold.
• Stopped-by-user state that labels partial output and offers continue, retry, copy partial, or discard.
• Interrupted, failed, rate-limited, offline, and timeout states that preserve prompt and partial output where safe.
• Complete state with final answer, final citations or source status, copy/apply eligibility, and completion metadata.

Interaction Contract

• The UI never presents an active stream as a complete answer until the terminal event or final validation state arrives.
• Partial output remains attached to the prompt, source scope, tool state, and stream status that produced it.
• Stop generation changes the state to stopped or partial instead of silently converting the text into a finished answer.
• Retry clearly states whether it continues from the partial output, restarts the same prompt, or regenerates a new answer.
• Copy, apply, cite, export, and publish actions indicate whether they operate on partial output or final output.
• Streaming updates do not steal focus, break text selection, collapse the composer, or force scroll when users are reading older content.
• Screen reader announcements summarize meaningful stream milestones instead of announcing every token.
• If safety, citation, tool, or structured-output checks revise or remove streamed content, the UI explains the revision.

Implementation Checklist

• Model stream lifecycle separately from prompt draft, submitted prompt, final response, citation resolution, tool execution, and history state.
• Track event types such as response started, output delta, tool call delta, citation pending, moderation hold, error, stopped, completed, and final metadata.
• Render partial output in stable blocks that tolerate incomplete Markdown, code fences, tables, links, JSON, and citations.
• Add Stop, Continue, Retry, Regenerate, Copy partial, Copy final, Discard, View events, and Report issue controls according to product risk.
• Throttle visual updates and live announcements to meaningful chunks, sentence boundaries, or elapsed intervals.
• Preserve prompt, partial output, selected text, scroll position, source panel state, and focus through stream failure and route changes where possible.
• Gate apply, publish, final copy, download, or structured export until final validation passes when early chunks can be misleading.
• Test slow first token, long stream, network interruption, server error event, content-filter hold, citation delay, tool delay, malformed Markdown, malformed JSON, mobile keyboard, reduced motion, and screen reader output.

Common Generated-UI Mistakes

• Showing a blinking cursor with no state, stop control, or elapsed feedback.
• Letting partial output look finished before the final event arrives.
• Copying or applying a streamed answer before citations, tool results, or validation complete.
• Erasing partial output after a network failure.
• Announcing every token through a live region.
• Forcing scroll to the newest token while users are reading or selecting earlier text.
• Using streaming to hide long-running tool work that should have visible steps or approval.

Critique Questions

• Which events prove the response has started, is still partial, has stalled, has stopped, has failed, or is complete?
• Can users tell whether text, citations, tool results, and structured output are pending or final?
• What actions are safe on partial output, and which require final validation?
• What remains available after stop, retry, timeout, offline, rate limit, or moderation hold?
• How does streaming behave when users scroll, select text, copy, inspect citations, use mobile keyboards, or rely on screen readers?
• Would a spinner, progress bar, chat interface, or workflow trace be clearer than streaming partial text?