UniversalRAG: Universal Retrieval-Augmented Generation Framework

Large Language Models LLMs excel in various tasks, yet hallucination limits their applicability in high-accuracy, domain-specific scenarios. Retrieval-Augmented Generation RAG mitigates this issue by integrating external knowledge retrieval, but existing systems struggle with multimodal, multi-format corpora common in industrial settings, and targeted evaluation datasets remain scarce. This paper introduces UniversalRAG, a modular, plug-and-play RAG framework supporting diverse document formats with adaptive indexing, retrieval, and generation agents, enhancing RAG adaptability and output quality. To validate its effectiveness, we develop the FACT dataset Fact-based Augmented Corpus Testing for RAG evaluation. Experimental results show that UniversalRAG, when paired with GPT-4o, achieves a 73.68 score, a 8.54-point improvement over the naive RAG baseline, significantly outperforming traditional methods. Ablation studies confirm the essential roles of indexing, retrieval, and generation agents in system performance. This work not only introduces a versatile RAG framework but also fills a critical gap in end-to-end evaluation, advancing RAG system development and assessment.