Building a RAG System with Qdrant Vector DB

RAG (Retrieval-Augmented Generation) is the most practical approach to addressing LLM hallucination. This post walks through the entire process of building a RAG system using Qdrant as the vector DB, step by step.

Why Qdrant

After comparing several vector DBs (Pinecone, Weaviate, Milvus, Chroma), we went with Qdrant.

Performance: gRPC support delivers fast responses at scale
Filtering: Powerful metadata-based filtering
Self-hosting: Simple deployment with Docker
Python SDK: Intuitive API

Running Qdrant with Docker

# docker-compose.yml
services:
  qdrant:
    image: qdrant/qdrant:latest
    ports:
      - "6333:6333"
      - "6334:6334"
    volumes:
      - qdrant_data:/qdrant/storage
    environment:
      QDRANT__SERVICE__GRPC_PORT: 6334

Embedding and Storing Documents

from qdrant_client import QdrantClient
from qdrant_client.models import Distance, VectorParams, PointStruct
from openai import AsyncOpenAI
import uuid

client = QdrantClient(host="localhost", port=6333)
openai_client = AsyncOpenAI()

COLLECTION_NAME = "documents"
VECTOR_SIZE = 1536  # text-embedding-3-small

# Create collection
client.create_collection(
    collection_name=COLLECTION_NAME,
    vectors_config=VectorParams(
        size=VECTOR_SIZE,
        distance=Distance.COSINE,
    ),
)

async def embed_text(text: str) -> list[float]:
    response = await openai_client.embeddings.create(
        model="text-embedding-3-small",
        input=text,
    )
    return response.data[0].embedding

async def store_document(text: str, metadata: dict):
    vector = await embed_text(text)
    point = PointStruct(
        id=str(uuid.uuid4()),
        vector=vector,
        payload={"text": text, **metadata},
    )
    client.upsert(
        collection_name=COLLECTION_NAME,
        points=[point],
    )

Retrieval and Context Construction

async def search_similar(query: str, top_k: int = 5) -> list[dict]:
    query_vector = await embed_text(query)
    results = client.search(
        collection_name=COLLECTION_NAME,
        query_vector=query_vector,
        limit=top_k,
        score_threshold=0.7,
    )
    return [
        {"text": hit.payload["text"], "score": hit.score}
        for hit in results
    ]

async def generate_rag_response(query: str) -> str:
    contexts = await search_similar(query)
    context_text = "\n\n".join([c["text"] for c in contexts])

    response = await openai_client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": f"다음 문서를 참고하세요:\n{context_text}"},
            {"role": "user", "content": query},
        ],
        stream=True,
    )
    return response

Usage in the XGen Platform

In XGen, we exposed RAG as a node-based pipeline via VectorStore and Retriever nodes, letting users assemble RAG pipelines through drag-and-drop.

class VectorStoreNode(BaseNode):
    async def execute(self, inputs: dict) -> dict:
        documents = inputs.get("documents", [])
        collection = inputs.get("collection", "default")
        for doc in documents:
            await store_document(doc["text"], doc.get("metadata", {}))
        return {"status": "stored", "count": len(documents)}

class RetrieverNode(BaseNode):
    async def execute(self, inputs: dict) -> dict:
        query = inputs.get("query", "")
        top_k = inputs.get("top_k", 5)
        results = await search_similar(query, top_k)
        return {"contexts": results}

Combined with Qdrant's filtering capabilities, you can build fine-grained RAG pipelines that restrict retrieval to specific categories, date ranges, or any other metadata criteria.

Building a RAG System with Qdrant Vector DB