Improved automatic diabetic retinopathy severity classification using deep multimodal fusion of UWF-CFP and OCTA images - IMT Atlantique Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Improved automatic diabetic retinopathy severity classification using deep multimodal fusion of UWF-CFP and OCTA images

Sophie Bonnin

Résumé

Diabetic Retinopathy (DR), a prevalent and severe complication of diabetes, affects millions of individuals globally, underscoring the need for accurate and timely diagnosis. Recent advancements in imaging technologies, such as Ultra-WideField Color Fundus Photography (UWF-CFP) imaging and Optical Coherence Tomography Angiography (OCTA), provide opportunities for the early detection of DR but also pose significant challenges given the disparate nature of the data they produce. This study introduces a novel multimodal approach that leverages these imaging modalities to notably enhance DR classification. Our approach integrates 2D UWF-CFP images and 3D high-resolution 6x6 mm3 OCTA (both structure and flow) images using a fusion of ResNet50 and 3D-ResNet50 models, with Squeeze-and-Excitation (SE) blocks to amplify relevant features. Additionally, to increase the model's generalization capabilities, a multimodal extension of Manifold Mixup, applied to concatenated multimodal features, is implemented. Experimental results demonstrate a remarkable enhancement in DR classification performance with the proposed multimodal approach compared to methods relying on a single modality only. The methodology laid out in this work holds substantial promise for facilitating more accurate, early detection of DR, potentially improving clinical outcomes for patients.
Fichier non déposé

Dates et versions

hal-04169377 , version 1 (24-07-2023)

Identifiants

  • HAL Id : hal-04169377 , version 1

Citer

Mostafa El Habib Daho, Yihao Li, Rachid Zeghlache, Yapo Cédric Atse, Hugo Le Boité, et al.. Improved automatic diabetic retinopathy severity classification using deep multimodal fusion of UWF-CFP and OCTA images. Ophthalmic Medical Image Analysis Workshop, Oct 2023, Vancouver (Canada), Canada. ⟨hal-04169377⟩
33 Consultations
0 Téléchargements

Partager

Gmail Facebook X LinkedIn More