Building a Real-Time Chat UI with SSE Streaming

Building a Streaming Chat Interface

Chat in an AI platform isn't just request-response. Displaying tokens as the LLM generates them is essential for a good user experience. Here's how we implemented an SSE (Server-Sent Events)-based streaming chat UI in XGen.

SSE vs. WebSocket — Why We Chose SSE

We initially considered WebSockets, but since chat streaming is mostly one-directional (server → client), SSE was the better fit. Connection management is simpler, and in HTTP/2 environments we get multiplexing for free.

Core Implementation: the useStreamChat Hook

function useStreamChat(workflowId: string) {
  const [messages, setMessages] = useState<ChatMessage[]>([]);
  const [isStreaming, setIsStreaming] = useState(false);

  const sendMessage = useCallback(async (content: string) => {
    const userMessage: ChatMessage = {
      id: crypto.randomUUID(),
      role: 'user',
      content,
      timestamp: new Date(),
    };
    setMessages((prev) => [...prev, userMessage]);
    setIsStreaming(true);

    const assistantId = crypto.randomUUID();
    setMessages((prev) => [
      ...prev,
      { id: assistantId, role: 'assistant', content: '', timestamp: new Date() },
    ]);

    const response = await fetch(`/api/workflow/${workflowId}/chat`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ message: content }),
    });

    const reader = response.body!.getReader();
    const decoder = new TextDecoder();

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;
      
      const chunk = decoder.decode(value);
      const lines = chunk.split('\n').filter((l) => l.startsWith('data: '));
      
      for (const line of lines) {
        const data = JSON.parse(line.slice(6));
        if (data.type === 'token') {
          setMessages((prev) =>
            prev.map((m) =>
              m.id === assistantId
                ? { ...m, content: m.content + data.content }
                : m
            )
          );
        }
      }
    }
    setIsStreaming(false);
  }, [workflowId]);

  return { messages, sendMessage, isStreaming };
}

Chat Bubble Component

We render in-progress and completed messages differently. While streaming, a cursor animation is shown; markdown rendering is applied only after the message is complete.

const ChatBubble: React.FC<{ message: ChatMessage; isStreaming: boolean }> = ({
  message,
  isStreaming,
}) => (
  <div className={`flex ${message.role === 'user' ? 'justify-end' : 'justify-start'}`}>
    <div className="max-w-[80%] rounded-lg p-3">
      {isStreaming ? (
        <span>{message.content}<span className="animate-pulse">▊</span></span>
      ) : (
        <MarkdownRenderer content={message.content} />
      )}
    </div>
  </div>
);

Performance Issues and Solutions

When tokens arrive rapidly, setMessages can fire dozens of times per second. Early on this caused noticeable jank. We fixed it by batching updates with requestAnimationFrame and using a useRef buffer to hold intermediate state. The perceived performance improvement was substantial.