Why RAG hallucinates, and how to prevent it
Retrieval-Augmented Generation (RAG) lets a language model answer from your documents. Hallucinations occur when the model gets “creative” without enough evidence. The fix isn't magic, it's architecture: better retrieval quality and strict grounding.
1. Hybrid search instead of vectors only
Pure vector search finds meaning but loses exact terms (part numbers, names). Classic keyword search (BM25) is the opposite. We combine both and fuse the results via Reciprocal Rank Fusion and measurably improves hit quality.
2. Reranking: separating signal from noise
From the top-50 candidates, a cross-encoder reranker (e.g. bge-reranker-v2-m3) selects the five most relevant passages. Only those go to the model, less noise, more precise answers.
3. Grounding and “not found”
Every answer is tied to concrete source passages and returned with a citation. If the system finds no evidence, it honestly answers “not found” instead of guessing. In archive contexts, audit-proof retention (GoBD) is added.