A review of deep learning-based information fusion techniques for multimodal medical image classification
Résumé
Multimodal medical imaging plays a pivotal role in clinical diagnosis and research, as it combines information
from various imaging modalities to provide a more comprehensive understanding of the underlying pathology.
Recently, deep learning-based multimodal fusion techniques have emerged as powerful tools for improving
medical image classification. This review offers a thorough analysis of the developments in deep learningbased
multimodal fusion for medical classification tasks. We explore the complementary relationships among
prevalent clinical modalities and outline three main fusion schemes for multimodal classification networks:
input fusion, intermediate fusion (encompassing single-level fusion, hierarchical fusion, and attention-based
fusion), and output fusion. By evaluating the performance of these fusion techniques, we provide insight
into the suitability of different network architectures for various multimodal fusion scenarios and application
domains. Furthermore, we delve into challenges related to network architecture selection, handling incomplete
multimodal data management, and the potential limitations of multimodal fusion. Finally, we spotlight the
promising future of Transformer-based multimodal fusion techniques and give recommendations for future
research in this rapidly evolving field.
Origine | Fichiers éditeurs autorisés sur une archive ouverte |
---|---|
Licence |