Multi-modal medical Transformers: A meta-analysis for medical image segmentation in oncology - IMT Atlantique
Article Dans Une Revue Computerized Medical Imaging and Graphics Année : 2023

Multi-modal medical Transformers: A meta-analysis for medical image segmentation in oncology

Résumé

Multi-modal medical image segmentation is a crucial task in oncology that enables the precise localization and quantification of tumors. The aim of this work is to present a meta-analysis of the use of multi-modal medical Transformers for medical image segmentation in oncology, specifically focusing on multi-parametric MR brain tumor segmentation (BraTS2021), and head and neck tumor segmentation using PET-CT images (HECKTOR2021). The multi-modal medical Transformer architectures presented in this work exploit the idea of modality interaction schemes based on visio-linguistic representations: (i) single-stream, where modalities are jointly processed by one Transformer encoder, and (ii) multiple-stream, where the inputs are encoded separately before being jointly modeled. A total of fourteen multi-modal architectures are evaluated using different ranking strategies based on dice similarity coefficient (DSC) and average symmetric surface distance (ASSD) metrics. In addition, cost indicators such as the number of trainable parameters and the number of multiply-accumulate operations (MACs) are reported. The results demonstrate that multi-path hybrid CNN- Transformer-based models improve segmentation accuracy when compared to traditional methods, but come at the cost of increased computation time and potentially larger model size.
Fichier non déposé

Dates et versions

hal-04266909 , version 1 (31-10-2023)

Identifiants

Citer

Gustavo Andrade-Miranda, Vincent Jaouen, Olena Tankyevych, Catherine Cheze Le Rest, Dimitris Visvikis, et al.. Multi-modal medical Transformers: A meta-analysis for medical image segmentation in oncology. Computerized Medical Imaging and Graphics, 2023, 110, pp.102308. ⟨10.1016/j.compmedimag.2023.102308⟩. ⟨hal-04266909⟩
48 Consultations
0 Téléchargements

Altmetric

Partager

More