Automated Diagnosis of Novice Programmers' Misconceptions Based on Machine Learning

This study proposed a mechanism for building a dynamic model to advise novice programmers of their misconceptions; it was mainly through monitoring the programming process and diagnosing their codes directly on the learning platform. It aims to provide a personal supervisor for suggestions of misconceptions and improve the previous lack of objective diagnosis. The development of the model was started by reviewing the misconceptions found in the prior researches and then designing the corresponding programming test to elicit these related misconceptions. After collecting 1213 students' codes, these codes were preprocessed by extracting features and encoding these codes to feature vectors. The clustering analysis was conducted to confirm that these feature vectors can indeed classify misconceptions from students' codes. As a result of data mining, this study summarized 9 categories of misconceptions; then, these feature vectors and their multi-labels were used to build a diagnosis model using supervised machine learning skills. In the experiment, there were 27 questions with 5 languages (Python, C++, C, Java, and Javascript) in the programming test, the clustering analysis adopted spectral clustering, the feature vector of one student's codes was 183 dimensions, the machine learning was implemented by multilayer perceptron, and the result demonstrated that the proposed mechanism performed 97.9% accuracy of the diagnosis model. The periodic development will dynamically change the categories of misconceptions and diagnosis models in response to students' codes until stable.