The Feasibility of Automated Topic Analysis: An Empirical Evaluation of Deep Learning Techniques Applied to Skew-Distributed Chinese Text Classificationψ

Yuen Hsien Tseng*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


Text classification (TC) is the task of assigning predefined categories (or labels) to texts for information organization, knowledge management, and many other applications. Normally the categories are topical in library science applications, although they can be any labels suitable for an application. Thus, TC often requires topical analysis which relies on human knowledge. However, in recent decades, machine learning (ML) techniques have been applied to TC for efficiency, as long as a sufficient number of training texts are available for each category. Nevertheless, in real-world cases, the number of texts (documents) for each category is often highly skewed for a certain TC task. This leads to the problem of predicting labels for small categories, which is viable for humans but challenging for machines. Deep learning (DL) is an emerging class of machine learning (ML) which was inspired by human neural networks. This study aims to evaluate whether DL techniques are feasible for the mentioned problem by comparing the performance of four off- the-shelf DL methods (CNN, RCNN, fastText, and BERT) with four traditional ML techniques on five skew-distributed datasets (four in Chinese, and one in English for comparison). Our results show that BERT is effective for moderately skewed datasets, but is still not feasible for highly skewed TC tasks. The other three DL-aware methods (CNN, RCNN, fastText) do not show any advantage in comparison with traditional methods such as SVM for the five TC tasks, although they captured extra language knowledge in the pretrained word representation. To facilitate future study, all of the Chinese datasets used in this study have been released publicly, together with all of the adapted machine learning and evaluation source codes for verification and for further study at Skewed_TxtClf.

Translated title of the contribution主題分析自動化的可行性: 深度學習技術應用於偏態分佈之 中文文件分類的實證評估ψ
Original languageEnglish
Pages (from-to)121-144
Number of pages24
JournalJournal of Educational Media and Library Sciences
Issue number1
Publication statusPublished - 2020


  • Deep learning
  • Performance evaluation
  • Real-world corpus
  • Text categorization

ASJC Scopus subject areas

  • Information Systems
  • Library and Information Sciences
  • Archaeology
  • Conservation


Dive into the research topics of 'The Feasibility of Automated Topic Analysis: An Empirical Evaluation of Deep Learning Techniques Applied to Skew-Distributed Chinese Text Classificationψ'. Together they form a unique fingerprint.

Cite this