Cultural Heritage Assistant: A Lightweight Retrieval Augmented Generation Method Enhanced Vision-Language Model for Cultural Heritage

External knowledge makes the Vision-Language Model VLM mo -re versatile. However, Traditional methods often fail to address the nuanced challenges of cultural heritage, mainly when dealing with unfamiliar or complex artifact-related queries. This limitation is evident in Vision-Language Models, which struggle to generate responses without exposure to domain knowledge. Frequent retraining to accommodate new artifacts or knowledge domains is computationally expensive and impractical. To overcome these limitations, we propose Cultural Heritage Assistant, a lightweight Retrieval-Augmented Generation RAG method designed for enhancing small-scale VLMs. Our approach integrates visual and textual retrieval modules to augment the input context, enabling the model to generate professional and accurate responses for cultural heritage queries. Experimental results on the constructed Hemudu Artifacts Visual Question-Answering dataset demonstrate the effectiveness of this approach. This method offers a solution for preserving and disseminating cultural heritage, bridging the gap between advanced VLM capabilities and domain-specific expertise.