What RAG Solves
LLMs have two persistent problems: a knowledge cutoff (training data only goes up to a certain date) and hallucination (fabricating non-existent facts). RAG solves both by using an external knowledge base — retrieve relevant documents in real time, then feed them as context to the LLM.
Technology Stack
| Component | Role | Recommendation |
|---|---|---|
| Document Processing | Parse + chunk | Unstructured, LlamaIndex |
| Embedding | Text to vectors | BGE-M3, text-embedding-3-large |
| Vector DB | Store and search | Milvus, Qdrant, Chroma |
| LLM | Generate answers | GPT-4o, Claude, Qwen |
Key Decision 1: Chunking
Chunking strategy directly impacts retrieval quality. Too fine loses semantics, too coarse adds noise.
from langchain.text_splitter import RecursiveCharacterTextSplitter
splitter = RecursiveCharacterTextSplitter(
chunk_size=500,
chunk_overlap=50,
separators=["\n\n", "\n", ".", "。", " "]
)
chunks = splitter.split_documents(documents)Key Decision 2: Hybrid Retrieval
Simple vector similarity search misses relevant documents. Production systems use hybrid retrieval:
from langchain.retrievers import EnsembleRetriever
hybrid = EnsembleRetriever(
retrievers=[dense_retriever, sparse_retriever],
weights=[0.7, 0.3]
)RAG is currently the most effective solution against LLM Hallucination Research. For personal practice, see Building a Personal RAG Knowledge Base.