Detecting Depression from Tweets through Topic Modeling and Lexicon-based Sentiment Fusion

Abstract

Mental disorders such as depression constitute the largest share of concerns in mental well-being across the globe, having a severe influence on an individual's emotional, social and professional lives. With the increased application of social networks in the modern age, opportunities have opened up to detect the signs of depression among people using various tools based on textual content shared on these platforms. This paper describes a hybrid NLP approach towards detecting depression through analysis of the textual corpus of tweets based on the usage of LDA, VADER and TF-IDF features. The method includes a complete process of text preprocessing, consisting of tokenization, lemmatization, stop-word removal, punctuation elimination and de-noising to create a meaningful text corpus. The resulting topics generated using LDA, sentiments expressed through VADER and statistical weight of words captured using TF-IDF are used to generate feature vectors. The constructed feature matrix is further used to train and test a range of machine learning/deep learnimg models, such as SVM, LR, KNN, RF and LSTM. The evaluation of the proposed approach is carried out based on publicly available Twitter depression dataset consisting of over one million labeled tweets. As a result, it is shown that the suggested approach can capture the themes, emotions and context connected to depression. The experiments carried out using a number of different models have revealed that SVM provides the highest values of classification accuracy (95.1%). On the other hand, LSTM achieves accuracy of 96.5%.