Multi-Modal Biomedical Data Fusion Using Transformer Networks for Early Detection and Severity Assessment of Neurodegenerative Diseases

Abstract

Neurodegenerative diseases are hard to detect in early stages due to clinical, neuroimaging, genomic and electrophysiological findings being variable, and not often studied together. The aim of this study is to propose a multimodal biomedical data-fusion framework using modality-specific encoders with cross-modal transformer networks to combine clinical variables, magnetic resonance images, genomic markers and electroencephalographic signals. The proposed approach builds a patient representation that is unified across self-attention and cross-modal attention and is leveraged to achieve binary early detection, mild–moderate–severe classification, and continuous severity-score estimation. The framework will be expected to provide an overall detection accuracy of about 94.2%, a sensitivity of about 93.5%, a specificity of about 94.8%, an F1-score of about 93.9% and an ROC–AUC of 0.968. The accuracy of the severity classification is expected to be 90.6% and the mean absolute error of the continuous severity estimation is expected to be about 4.7 points and R^2 of 0.89. These expected outcomes should be validated by adequate computation and clinical tests.