Intelligent Diagnosis for Breast Cancer Based on Multi-Modal Hierarchical Fusion of Ultrasound Images and Clinical Semantic Features

Ultrasound imaging plays a key role in breast cancer screening and diagnosis due to its advantages of non-invasive, real-time and low cost. This paper proposes an intelligent breast cancer diagnosis method based on Multi-modal Hierarchical Fu-sion Network MHFNet , aiming to fully integrate complementary information from B-mode image, Doppler image and clinical semantic features. In MHFNet, a Semantic-augmented ResNet SAR was constructed to achieve the deep fusion of image and semantic features. And Hierarchical Semantic Fusion HSF mod-ule and Semantic Integration Bottleneck SIB are designed to enhance the inter-action and fusion of multi-modal information layer by layer. Finally, a unified multi-modal feature representation was developed for breast cancer diagnosis. The experimental results show that the proposed multi-modal classification fusion method is superior to other comparison algorithms, which fully verifies the posi-tive role of multi-modal information complementarity in improving the diagnostic performance of breast cancer.