Why Embedding Model Selection Matters
40–60% of a RAG system's performance is determined by the embedding model. Even with a great LLM, poor retrieval quality leads to hallucinations.
Model Selection Decision Tree
flowchart TD
A[RAG Project Start] --> B{Language?}
B -->|English only| C[text-embedding-3-large]
B -->|Multilingual/Korean| D{Budget?}
D -->|Sufficient| E[OpenAI ada-002/3]
D -->|Limited| F{Domain-specific?}
F -->|Required| G[Custom training prj_ecellm approach]
F -->|Not required| H[bge-m3 or e5-large]
E --> I{Performance tuning?}
I -->|Required| J[Add Reranker]
I -->|Not required| K[Use as-is]
G --> J
H --> J
Embedding Model Comparison
| Model | Dimensions | Korean | Cost | Speed | Recommended Use |
|---|---|---|---|---|---|
| text-embedding-3-small | 1536 | ⭐⭐ | Paid | Fast | General RAG |
| text-embedding-3-large | 3072 | ⭐⭐ | Paid | Moderate | High-precision RAG |
| bge-m3 | 1024 | ⭐⭐⭐ | Free | Moderate | Multilingual RAG |
| multilingual-e5-large | 1024 | ⭐⭐⭐ | Free | Slow | Academic research |
| ko-sbert-nli | 768 | ⭐⭐⭐⭐ | Free | Fast | Korean-specialized |
Lessons from Production Experience
1. Dimensionality vs. Performance
From experiments in prj_ecellm, the performance gap between 768 and 1024 dimensions was smaller than expected — roughly 2% by Retrieval@10. Storage costs, however, differ by 33%.
2. The Impact of a Reranker
Adding a Cross-Encoder Reranker consistently delivered an average nDCG improvement of 22%. It's the highest ROI optimization available.
3. ROI of Custom Training
Results from domain-specific fine-tuning:
- Investment: ~2 weeks of development time + GPU costs
- Return: 12% improvement in Retrieval@10
- Conclusion: Worth it when the domain is well-defined and sufficient data is available
Recommended Production Setup
graph LR
A[Query] --> B[Bi-Encoder bge-m3]
B -->|Top-100| C[Cross-Encoder Reranker]
C -->|Top-5| D[LLM Generation]
style B fill:#e3f2fd
style C fill:#fff8e1
style D fill:#e8f5e9
This setup delivers the best performance per dollar. Everything here is based on hands-on experience from the prj_ecellm, Embedding-Is-All-RAG-Need, and Contextifier projects.