LLM-driven Interactive document classification through Keyword Feedback

Document classification offers a concise comprehension of document content, which is crucial for document organization and management in real application. However, practical scenarios pose challenges due to limited annotated data and dynamic changes in document categories. In this paper, we propose an LLM-driven interactive document classification framework based on keyword feedback, which operates with minimal input—just the documents to be classified. We achieve this by first introducing an unsupervised learning based document classification framework. Then a keyword interaction process is designed to iteratively enhance the classifier's performance. The representative keyword explanations is generated in each iteration, which offer the most significant features or characteristics within each category. Crucially, an LLM feedback module is designed for interaction which offers category description and keyword feedback, facilitating seamless cooperation to enhance classification performance. Experimental results on benchmark datasets demonstrated that our framework significantly improves classifier accuracy when compared to methods lacking feedback with few feedback iterations.