Reliability, validity, and strictness of adult learner online self-rating based on teacher and achievement criteria for different individual characteristics and academic achievement

Although many studies have shown that student self-assessment/rating is reliable or valid, many teachers still hesitate to implement student self-assessment/rating in their teaching. Therefore, whether or not student self-rating is reliable or valid has been a crucial concern. The study aimed to investigate the reliability, validity, and strictness of adult students’ online self-ratings by examining the consistency and difference between self-rating and teacher rating, and the consistency between self-rating and term project grade. There were 368 graduate students who used an online assessment platform to rate their own presentations. The results showed that adult students’ self-ratings had significant but low inter-rater consistency reliability and teacher-referenced validity, but inadequate teacher-referenced strictness, compared with teacher/expert criteria. Self-rating had significant but low achievement-referenced validity, compared with academic achievement criteria. Age, work experience, paper-pencil self-assessment experience, and online self-assessment experience affected the inter-rater consistency reliability and teacher-referenced validity of student self-rating. Individual characteristics and academic achievement had no effect on self-rating strictness. Implications and recommendations for academic theory and educational practice are proposed.