KG-MMIL: A Knowledge-Guided Multi-modal Multi-Instance Learning Framework for Interpretable Renal Tumor Ultrasound Diagnosis

Current methods for diagnosing renal tumors via ultrasound primarily rely on learning image features; however, the model’s decision-making process lacks interpretability, exhibiting typical “black-box” characteristics that limit its clinical application. To address this issue, this paper proposes a knowledge-guided multimodal multi-instance learning fusion model (KG-MMIL) to enhance the model’s discriminative performance and interpretability.First, a knowledge-guided multimodal feature enhancement method is designed. By constructing a knowledge-guided attention mechanism, medical text knowledge is converted into prior weights and fused with data-driven attention weights. This guides the model to focus on key regions and features of diagnostic significance, thereby improving the medical consistency and interpretability of feature representations.Second, to address the challenges of fusion caused by multimodal feature heterogeneity, we propose a multi-stream residual parallel fusion method. Through a multi-path residual structure, this method enables deep interaction and information compensation across modalities, effectively mitigating information loss and gradient propagation issues in traditional fusion methods, thereby enhancing overall feature expressiveness.Experimental results on a renal tumor ultrasound dataset demonstrate that the proposed KG-MMIL model achieves a precision of 91.9% and an AUC of 0.961. Compared to the second-best multimodal method, precision is improved by 1.7 percentage points while maintaining competitive performance in terms of AUC, validating the effectiveness and superiority of this approach.