The mechanics of evidence-grounded archive search
An archive system is only as trustworthy as the source behind each answer. We separate retrieval from generation. Search runs two methods in parallel: BM25 for exact terms, file numbers and proper names, and a vector search over self-hosted embeddings for semantic proximity. Both result lists are merged through Reciprocal Rank Fusion. A cross-encoder reranking step narrows the top 50 candidates down to the five most relevant passages. Only those reach the answer.
Before output, an NLI sentence-level grounding check tests every generated sentence against the cited sources. If the model finds no support, the system replies not found rather than guessing. This abstention is the point: no invented references, no hallucinated case numbers. An eval_harness runs the RAGAS Faithfulness metric ahead of every release.
To navigate large holdings, PageIndex provides a document tree down to section level. The knowledge graph links entities, relations, paths and communities, which makes cross-comparisons across thousands of files practical. Where audit integrity is mandatory, we implement the framework defined by GoBD, section 147 of the German Fiscal Code and WORM audit logging.
- Hybrid search combining BM25 and vector retrieval with rank fusion
- Reranking from top-50 to top-5 before any answer
- Sentence-level grounding with abstention when evidence is missing
- PageIndex and knowledge graph for structure and cross-references
- Audit integrity aligned with GoBD and section 147 AO
This architecture runs in production, including a large-scale AI archive with millions of documents. Built for archives, law firms, public authorities and research. Your data stays on our servers in Germany.