Measuring AI literacy in medical students: scale development and validation within a self-determination theory framework

Background: Artificial intelligence (AI) is increasingly integrated into healthcare, making AI literacy an essential competency for medical students. Existing assessments are often generic, lack validation in medical education, and are not grounded in learning theory. This study developed and validated the AI Literacy Scale for Medical Students (ALSMS) within a self-determination theory (SDT) framework. Methods: We used a split-sample validation design (N = 518; exploratory factor analysis [EFA], n = 204; confirmatory factor analysis [CFA], n = 314). Candidate facets were derived from prior AI literacy instruments and a previously developed framework, then organized according to SDT. EFA refined the first-order structure, and CFA cross-validated the retained structure and compared prespecified first-order and SDT-aligned higher-order models. Results: EFA identified nine factors organized into the SDT domains of competence, relatedness, and autonomy. CFA supported the correlated nine-factor structure and demonstrated strong psychometric properties. Model comparisons identified two theory-consistent, well-fitting solutions: a correlated nine-factor model and an SDT-aligned second-order model with Ethics loading on Autonomy. Unidimensional and some hierarchical general-factor models showed poorer fit or identification problems, supporting the construct’s multidimensionality. Conclusions: This study provides initial validity evidence for interpreting ALSMS scores as indicators of medical students’ AI literacy within an SDT-informed framework. The findings highlight the significance of integrating ethics into autonomy-supportive curricula and underscore the potential utility of ALSMS for curriculum design, advising, and the evaluation of AI literacy initiatives in medical education.