Multimodal Deep Learning for Depression Detection: A Systematic Review of Artificial Intelligence Techniques, Data Fusion Strategies, and Emerging Trends

Abstract

Depression is one of the most widespread mental health conditions worldwide and impacts approximately 280 million people around the globe. The primary way to diagnose and treat depressive disorders is through clinical interviews and predominantly self-reports, which are very subjective, often very time and labor intensive, and susceptible to social desirability bias. Conversely, recent advances in artificial intelligence (primarily deep learning) are creating new opportunities for objectively measuring depression in a scalable, rapid, and non-invasive manner. The current systematic review examines—through established PRISMA guidelines—deep learning approaches to detecting and assessing depression using multimodal datasets (text, audio, video, and images) and includes a look at five databases (PubMed, IEEE Xplore, ACM Digital Library, Web of Science, and Scopus) from 2015 to 2024. Of 2,847 initial records screened, 127 studies met the inclusion criteria. Overall, the analysis showed that multimodal (i.e. text/audio) approaches consistently performed better than unimodal (i.e. text only) approaches, with an increase of 5% to 15% in classification accuracy identified for each approach. Of the studies reviewed, text/audio (43% of studies), audio/video (31%), and total four-modal approaches (12%) were the most frequently used combinations. Late fusion methods dominated the literature (52% of studies), although advances using transformer networks for joint fusion have recently shown promise. Common architecture configurations included CNN/LSTM hybrid models for temporal data, BERT style models for text data, and attention-based multimodal transformer networks. Several barriers limiting this body of work exist, including lack of available datasets and limited cross-cultural generalizability.