A Study on the Automatic Classification of Tweets Related to Mental Health Literacy

Users on Twitter often post various tweets describing their moods. Analyzing these tweets can aid in understanding an individual’s psychological state, which will be beneficial to research aimed at promoting public mental health. This study intends to perform automatic classification on tweets related to mental health literacy. Techniques including traditional machine learning as well as AI technologies like BERT, SetFit, GPT-3, and GPT-4 are used to automatically classify them into 11 items across five dimensions, with each item having five related intensity scores. The goal is to achieve a machine prediction effectiveness of over 0.8 with limited human-annotated training data, to ensure the machine can effectively assist in mental health research. The results show that using SetFit, most items can achieve a Macro F1 score of about 0.8, with only two items scoring around 0.65. The contribution of this study lies in presenting and comparing the effectiveness of various natural language processing paradigms in text comprehension and analysis on these difficult tasks.