A production-grade 7-stage retrieval-augmented generation system for research document intelligence
Research analysts and strategy consultants spend hours manually reading dense documents to locate specific data points, cross-reference claims across sources, and build evidence-grounded views. Keyword search is shallow. A basic LLM chatbot is unreliable: it hallucinates facts and cannot cite sources.
This project builds a system that solves both problems. Every answer is retrieved before it is generated, and every claim is cited to a specific source document and page number.
all-MiniLM-L6-v2 (384-dimensional bi-encoder). Top 20 candidates retrieved by cosine similarity.rank-bm25 library with a custom regex tokenizer. Handles exact keyword matches that dense retrieval misses. Top 20 candidates retrieved independently from the dense results.cross-encoder/ms-marco-MiniLM-L-6-v2 rescores each (query, chunk) pair with full attention across both. More accurate than bi-encoder similarity but too slow to run on the full corpus, so it runs only on the 20 RRF candidates.compound-beta-mini generates the final answer. System prompt enforces strict grounding: no external knowledge, every claim must be cited, refuse to answer if context is insufficient. temperature=0.1 for near-deterministic factual extraction.| Component | Library or Service |
|---|---|
| PDF Parsing | PyMuPDF (fitz) |
| Chunking | LangChain RecursiveCharacterTextSplitter |
| Embeddings | sentence-transformers all-MiniLM-L6-v2 |
| Vector Store | ChromaDB (persistent, embedded mode) |
| Sparse Retrieval | rank-bm25 (BM25Okapi) |
| Hybrid Fusion | Reciprocal Rank Fusion (custom implementation) |
| Query Engineering | HyDE, Query Rewriting, Multi-Query |
| Reranking | cross-encoder/ms-marco-MiniLM-L-6-v2 |
| LLM Generation | Groq API (compound-beta-mini) |
| API Framework | FastAPI plus Uvicorn |
| Containerisation | Docker plus Docker Compose |
| Deployment | Railway (Dockerised) |
| Evaluation | RAGAS 0.2.14 (local, no OpenAI calls) |
| Tracing | LangSmith with traceable decorators |
This project was built by diagnosing real retrieval failures, not by following a framework tutorial. Each challenge below was encountered during development, diagnosed through controlled experiments, and fixed with a targeted solution.
A query for North America revenue returned a ratio analysis table (where "revenue" appeared six times as a denominator) and a Swiss subsidiary row (where "NA" matched "North America" semantically). The correct answer page containing the actual geographic revenue figure was not in the top 3 results at all.
BM25 sparse retrieval. BM25 tokenizes "NA" as a single token with no geographic match and scores the ratio table near zero because it contains no geographic tokens. BM25 found the correct page at rank 3. MRR improved from 0.00 to 0.33.
BM25 alone had MRR 0.33. Combining dense and BM25 with RRF dropped MRR to 0.00. The correct page was absent from the dense top-10 entirely, so RRF amplified the dense false positives rather than correcting them. RRF rewards cross-system agreement but cannot penalise single-system confident errors.
HyDE query expansion. Generating a hypothetical answer passage using document vocabulary (INR crore, North America, YoY, geographic mix) moved the correct page to dense rank 1. HyDE plus BM25 hybrid achieved MRR 1.00.
After HyDE and hybrid retrieval got the correct page to rank 1, adding the cross-encoder dropped it to rank 2. The MS MARCO-trained cross-encoder had learned that pages dense with geographic names are relevant to geographic queries, which is true on the web but wrong in financial documents. It ranked a corporate overview page containing a country office list above the actual revenue table.
Page-level deduplication after reranking. Keeping only the highest-scoring chunk per unique source page removed a duplicate chunk that was diluting the correct page's ranking and allowed its score to compete directly against the false positive. Zero additional compute cost. MRR restored to 1.00.
Note on Faithfulness: The 0.385 score is a measurement artefact. The cross-encoder faithfulness metric penalises terse one-liner answers produced for numerical fact queries, because short answer sentences score in a different logit range than full sentences when paired with long context paragraphs. Conceptual questions where the model produced multi-sentence answers all scored 1.000.
Evaluated on 13 ground-truth QA pairs across the Infosys FY2025-26 annual report and AGM transcript corpus.
deep-research-rag/ ├── app.py # Portfolio demo UI (FastAPI + dark-mode HTML, port 7860) ├── src/ │ ├── ingestion/ # PDF parsing, chunking, embedding, ChromaDB ingest │ ├── retrieval/ # Dense retriever, BM25 retriever, RRF fusion │ ├── reranking/ # Cross-encoder reranker │ ├── generation/ # HyDE, LLM generation, citation extraction │ ├── pipeline/ # Full 7-stage pipeline orchestration │ ├── evaluation/ # RAGAS evaluation │ └── api/ # FastAPI application ├── chroma_db/ # Pre-populated vector store (committed to repo) ├── data/ │ └── ground_truth.json ├── Dockerfile ├── docker-compose.yml └── requirements.txt
Explore more projects from BuildWithAV
← Back to Projects