CIAF: Cross-modal Inconsistency Aware Framework for Multimodal Fact-Checking

In the multimodal information era, fact-checking faces growing challenges as misinformation becomes increasingly complex. Content that appears plausible in a single modality may reveal subtle inconsistencies across modalities, which often overlooked by traditional methods. Existing approaches mainly fuse multimodal features but rarely explicitly model fine-grained cross-modal inconsistencies, limiting both accuracy and interpretability. To address this issue, we propose a Cross-modal Inconsistency Aware Framework (CIAF) that leverages multimodal sentiment cues to identify and localize implicit discrepancies between textual claims and corresponding images. Specifically, CIAF first performs a joint analysis of sentiment elements across textual and visual modalities. It then introduces a novel relational discrepancy attention (RDA) mechanism to model semantic and emotional interactions between modalities, dynamically weighting potential inconsistency signals. Furthermore, we design task-oriented prompt templates to guide the model’s reasoning process toward cross-modal consistency assessment, thereby enhancing both precision and robustness in multimodal fact-checking. Experiments on two challenging multimodal fact-checking benchmarks show that our approach delivers superior performance and interpretability, underscoring the importance of modeling fine-grained cross-modal inconsistencies for robust fact verification.