SSE Streaming in FastAPI: A Practical Implementation Guide

What Is SSE?

Server-Sent Events is an HTTP protocol that provides one-way, real-time data streaming from server to client. It is ideal for LLM token streaming.

SSE vs WebSocket

graph LR
    subgraph SSE
        A1[서버] -->|단방향| B1[클라이언트]
    end

    subgraph WebSocket
        A2[서버] <-->|양방향| B2[클라이언트]
    end

FastAPI SSE Implementation

from fastapi import FastAPI
from fastapi.responses import StreamingResponse
import asyncio

app = FastAPI()

async def generate_stream(prompt: str):
    async for chunk in llm.astream(prompt):
        yield f"data: {json.dumps({'content': chunk.content})}\n\n"
    yield "data: [DONE]\n\n"

@app.post("/api/chat/stream")
async def chat_stream(request: ChatRequest):
    return StreamingResponse(
        generate_stream(request.prompt),
        media_type="text/event-stream",
        headers={
            "Cache-Control": "no-cache",
            "Connection": "keep-alive",
            "X-Accel-Buffering": "no",  # Disable Nginx buffering
        }
    )

Frontend Consumption

async function consumeSSE(prompt: string) {
  const response = await fetch('/api/chat/stream', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ prompt })
  });

  const reader = response.body!.getReader();
  const decoder = new TextDecoder();

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    const text = decoder.decode(value);
    for (const line of text.split('\n')) {
      if (line.startsWith('data: ') && line !== 'data: [DONE]') {
        const data = JSON.parse(line.slice(6));
        appendToUI(data.content);
      }
    }
  }
}

Nginx Configuration Caveats

If Nginx buffers the SSE response, streaming will not work:

location /api/chat/stream {
    proxy_pass http://backend:8000;
    proxy_buffering off;
    proxy_cache off;
    proxy_read_timeout 600s;
    proxy_set_header Connection '';
    chunked_transfer_encoding off;
}

This pattern was used across all AI projects — PlateeRAG-1, hr_blog2.0, web-front, and web.