Evaluation of the psychometric properties of the general literacy test for university students

Po-Hsi Chen, Hsin Ying Huang, Yu Hsin Chen, Tai Ting Yeh, Shao Tsu Su

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

This study evaluated the psychometric properties of the General Literacy Test for University Students. To develop the assessment framework, the educational objectives of general literacy courses of universities in Taiwan as well as the core competencies of Assessment and Teaching of 21st Century Skills were all reviewed and considered. The general literacy test is composed of nine literacy domains: communication and collaboration, esthetics, information, lifelong learning, career, leadership, problem solving, social concerns and citizenship, and scientific thinking. The items of the general literacy test were developed into a multimedia assessment. In addition, booklets including two to three testlets on each of the nine literacy domains were organized. Overall, 10,958 students from 20 universities participated. The results demonstrated that testlet effect could be ignored and the partial credit model could be used to analyze data. In addition, the items fit the partial credit model closely and the test demonstrated favorable construct validity. In addition, small differences existed between gender and grades. The results were consistent with those of previous studies and showed that the test’s criterion-related validity was acceptable. Furthermore, the standard error of ability ranged from .20 to .60 logit and the reliabilities of the booklets were more than .69 in the nine literacy domains. The evidence supports the high degree of reliability and validity of basic literacy assessment.

Original languageEnglish
Pages (from-to)95-126
Number of pages32
JournalJournal of Research in Education Sciences
Volume60
Issue number3
DOIs
Publication statusPublished - 2015 Sep 1

Fingerprint

psychometrics
literacy
university
evaluation
student
credit
lifelong learning
construct validity
multimedia
Taiwan
aesthetics
citizenship
career
leadership
communication
gender
ability
Teaching
evidence

Keywords

  • Literacy assessment
  • Online test
  • Reliability
  • Scenario based test
  • Validity

ASJC Scopus subject areas

  • Education

Cite this

Evaluation of the psychometric properties of the general literacy test for university students. / Chen, Po-Hsi; Huang, Hsin Ying; Chen, Yu Hsin; Yeh, Tai Ting; Su, Shao Tsu.

In: Journal of Research in Education Sciences, Vol. 60, No. 3, 01.09.2015, p. 95-126.

Research output: Contribution to journalArticle

Chen, Po-Hsi ; Huang, Hsin Ying ; Chen, Yu Hsin ; Yeh, Tai Ting ; Su, Shao Tsu. / Evaluation of the psychometric properties of the general literacy test for university students. In: Journal of Research in Education Sciences. 2015 ; Vol. 60, No. 3. pp. 95-126.
@article{0c738dcc14564938a80eecbcf0fd63bb,
title = "Evaluation of the psychometric properties of the general literacy test for university students",
abstract = "This study evaluated the psychometric properties of the General Literacy Test for University Students. To develop the assessment framework, the educational objectives of general literacy courses of universities in Taiwan as well as the core competencies of Assessment and Teaching of 21st Century Skills were all reviewed and considered. The general literacy test is composed of nine literacy domains: communication and collaboration, esthetics, information, lifelong learning, career, leadership, problem solving, social concerns and citizenship, and scientific thinking. The items of the general literacy test were developed into a multimedia assessment. In addition, booklets including two to three testlets on each of the nine literacy domains were organized. Overall, 10,958 students from 20 universities participated. The results demonstrated that testlet effect could be ignored and the partial credit model could be used to analyze data. In addition, the items fit the partial credit model closely and the test demonstrated favorable construct validity. In addition, small differences existed between gender and grades. The results were consistent with those of previous studies and showed that the test’s criterion-related validity was acceptable. Furthermore, the standard error of ability ranged from .20 to .60 logit and the reliabilities of the booklets were more than .69 in the nine literacy domains. The evidence supports the high degree of reliability and validity of basic literacy assessment.",
keywords = "Literacy assessment, Online test, Reliability, Scenario based test, Validity",
author = "Po-Hsi Chen and Huang, {Hsin Ying} and Chen, {Yu Hsin} and Yeh, {Tai Ting} and Su, {Shao Tsu}",
year = "2015",
month = "9",
day = "1",
doi = "10.6209/JORIES.2015.60(3).04",
language = "English",
volume = "60",
pages = "95--126",
journal = "Journal of Research in Education Sciences",
issn = "2073-753X",
publisher = "國立臺灣師範大學",
number = "3",

}

TY - JOUR

T1 - Evaluation of the psychometric properties of the general literacy test for university students

AU - Chen, Po-Hsi

AU - Huang, Hsin Ying

AU - Chen, Yu Hsin

AU - Yeh, Tai Ting

AU - Su, Shao Tsu

PY - 2015/9/1

Y1 - 2015/9/1

N2 - This study evaluated the psychometric properties of the General Literacy Test for University Students. To develop the assessment framework, the educational objectives of general literacy courses of universities in Taiwan as well as the core competencies of Assessment and Teaching of 21st Century Skills were all reviewed and considered. The general literacy test is composed of nine literacy domains: communication and collaboration, esthetics, information, lifelong learning, career, leadership, problem solving, social concerns and citizenship, and scientific thinking. The items of the general literacy test were developed into a multimedia assessment. In addition, booklets including two to three testlets on each of the nine literacy domains were organized. Overall, 10,958 students from 20 universities participated. The results demonstrated that testlet effect could be ignored and the partial credit model could be used to analyze data. In addition, the items fit the partial credit model closely and the test demonstrated favorable construct validity. In addition, small differences existed between gender and grades. The results were consistent with those of previous studies and showed that the test’s criterion-related validity was acceptable. Furthermore, the standard error of ability ranged from .20 to .60 logit and the reliabilities of the booklets were more than .69 in the nine literacy domains. The evidence supports the high degree of reliability and validity of basic literacy assessment.

AB - This study evaluated the psychometric properties of the General Literacy Test for University Students. To develop the assessment framework, the educational objectives of general literacy courses of universities in Taiwan as well as the core competencies of Assessment and Teaching of 21st Century Skills were all reviewed and considered. The general literacy test is composed of nine literacy domains: communication and collaboration, esthetics, information, lifelong learning, career, leadership, problem solving, social concerns and citizenship, and scientific thinking. The items of the general literacy test were developed into a multimedia assessment. In addition, booklets including two to three testlets on each of the nine literacy domains were organized. Overall, 10,958 students from 20 universities participated. The results demonstrated that testlet effect could be ignored and the partial credit model could be used to analyze data. In addition, the items fit the partial credit model closely and the test demonstrated favorable construct validity. In addition, small differences existed between gender and grades. The results were consistent with those of previous studies and showed that the test’s criterion-related validity was acceptable. Furthermore, the standard error of ability ranged from .20 to .60 logit and the reliabilities of the booklets were more than .69 in the nine literacy domains. The evidence supports the high degree of reliability and validity of basic literacy assessment.

KW - Literacy assessment

KW - Online test

KW - Reliability

KW - Scenario based test

KW - Validity

UR - http://www.scopus.com/inward/record.url?scp=84942866884&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84942866884&partnerID=8YFLogxK

U2 - 10.6209/JORIES.2015.60(3).04

DO - 10.6209/JORIES.2015.60(3).04

M3 - Article

AN - SCOPUS:84942866884

VL - 60

SP - 95

EP - 126

JO - Journal of Research in Education Sciences

JF - Journal of Research in Education Sciences

SN - 2073-753X

IS - 3

ER -