F3ND: Bridging the Semantic Gap with Tri-modal Self-Attention for Enhanced Fake News Detection

The rapid development of Internet technology and the rise of social media have promoted the spread of fake news, most of which contain both text and image content. However, multimodal methods will inevitably encounter the challenge of semantic gap when analyzing these news. To solve this problem, we propose F3ND, a tri-modal integrated self-attention framework for fake news detection. F3ND combines unimodal text and image features with fused multimodal features. The fused features act as a bridge to enhance the correlation between the two unimodal features, which effectively solves the semantic gap problem when understanding multimodal content. At the same time, we introduce a self-attention mechanism to dynamically assign weights to different features, retaining the discriminative information in unimodal features that helps to determine whether the news is fake. Our experiments on Weibo and Weibo21 datasets show that F3ND can achieve better performance than many previous baseline models, proving the robustness and effectiveness of our method.