Self-Improving RAG
Self-Improving RAG is a judge-driven multi-agent retrieval system that improves financial document question answering through feedback-driven self-correction. Expert-level financial QA needs grounded verification to catch numeric hallucinations and audit trails for regulatory compliance, two attributes that standard single-pass RAG systems lack. The system decomposes answering into three specialized agents for retrieval, reasoning, and judging, coordinated by an orchestrator. When the judge scores an answer below a dynamic threshold, the system retries with escalated strategies such as broader retrieval and more careful prompting, and the best answer is always kept.
What exists today
- A peer-reviewed workshop paper accepted at the AFA Workshop @ ICLR 2026, written with collaborators from Cal Poly, Hofstra, and Pepperdine. Read the paper.
- An open-source implementation covering the full multi-agent pipeline, retrieval tools, and evaluation harness. Browse the code.
- Benchmark results on FinanceBench: 86% oracle-guided accuracy, a 62.3 point improvement over a single-pass RAG baseline, with recovery of roughly 4 in 10 initially incorrect answers through targeted retry.
- An audit-first design where every agent decision is logged with provenance, confidence scores, and reasoning traces.
Who it is for
The system is aimed at teams that answer questions over regulated, high-stakes document collections: financial analysts working with SEC filings and earnings reports, compliance teams that need auditable answer trails, and researchers studying retrieval-augmented generation and self-correcting agent loops.
What is next
Current development focuses on scaling evaluation beyond FinanceBench to broader financial corpora, hardening the retrieval and judging pipeline for larger document collections, and building a hosted demo so that analysts and researchers can try the system on their own documents. This work is compute-intensive: it requires sustained model inference for the three-agent pipeline, embedding generation and vector search over large filing corpora, storage for documents and audit logs, and batch compute for evaluation sweeps.
Status and contact
Self-Improving RAG is an independent research project led by Junjie Xiong, an EECS student at UC Berkeley. It is currently unincorporated and under active development. For questions or collaboration, email junjiexiong@berkeley.edu.