Establishing the Criterion-related, Construct, and Content Validities of a Simulation-based Assessment of Inquiry Abilities

Pai Hsing Wu, Hsin-Kai Wu, Ying-Shao Hsu

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

The emphasis on scientific inquiry has increased the importance in developing the fundamental abilities to conduct scientific investigations and urged a need for valid assessments of students' inquiry abilities. We took advantage of the advanced technology to develop a simulation-based assessment of inquiry abilities (SAIA) that allowed students to generate scientific explanations and demonstrate their experimental abilities. This paper describes the validation of the assessment. Data were collected from 48 12th-grade students at a local high school who were categorized into three groups based on their program majors. Both quantitative and qualitative approaches were utilized to validate SAIA. The quantitative results showed that SAIA was aligned with a validated reasoning-skill test (criterion-related validity), discriminated variance among different groups (construct validity), and was highly suitable for examining inquiry abilities (content validity). Additionally, we utilized the think-aloud technique in order to identify the performances exhibited by students while they accomplished the SAIA tasks. The protocol analysis indicated that in general, students demonstrated the expected abilities in SAIA and that their SAIA scores accurately reflected their performance levels of inquiry abilities. The results suggested that SAIA was a valid assessment for evaluating the inquiry abilities of high school students. This study also provided systemic strategies for validating simulation-based assessments.

Original languageEnglish
Pages (from-to)1630-1650
Number of pages21
JournalInternational Journal of Science Education
Volume36
Issue number10
DOIs
Publication statusPublished - 2014 Jan 1

Fingerprint

simulation
ability
student
construct validity
school
performance
Group

Keywords

  • Inquiry ability
  • Simulation-based assessment
  • Validation

ASJC Scopus subject areas

  • Education

Cite this

@article{d2d07461ab684737b30e2c3cdc75f8fb,
title = "Establishing the Criterion-related, Construct, and Content Validities of a Simulation-based Assessment of Inquiry Abilities",
abstract = "The emphasis on scientific inquiry has increased the importance in developing the fundamental abilities to conduct scientific investigations and urged a need for valid assessments of students' inquiry abilities. We took advantage of the advanced technology to develop a simulation-based assessment of inquiry abilities (SAIA) that allowed students to generate scientific explanations and demonstrate their experimental abilities. This paper describes the validation of the assessment. Data were collected from 48 12th-grade students at a local high school who were categorized into three groups based on their program majors. Both quantitative and qualitative approaches were utilized to validate SAIA. The quantitative results showed that SAIA was aligned with a validated reasoning-skill test (criterion-related validity), discriminated variance among different groups (construct validity), and was highly suitable for examining inquiry abilities (content validity). Additionally, we utilized the think-aloud technique in order to identify the performances exhibited by students while they accomplished the SAIA tasks. The protocol analysis indicated that in general, students demonstrated the expected abilities in SAIA and that their SAIA scores accurately reflected their performance levels of inquiry abilities. The results suggested that SAIA was a valid assessment for evaluating the inquiry abilities of high school students. This study also provided systemic strategies for validating simulation-based assessments.",
keywords = "Inquiry ability, Simulation-based assessment, Validation",
author = "Wu, {Pai Hsing} and Hsin-Kai Wu and Ying-Shao Hsu",
year = "2014",
month = "1",
day = "1",
doi = "10.1080/09500693.2013.871660",
language = "English",
volume = "36",
pages = "1630--1650",
journal = "International Journal of Science Education",
issn = "0950-0693",
publisher = "Taylor and Francis Ltd.",
number = "10",

}

TY - JOUR

T1 - Establishing the Criterion-related, Construct, and Content Validities of a Simulation-based Assessment of Inquiry Abilities

AU - Wu, Pai Hsing

AU - Wu, Hsin-Kai

AU - Hsu, Ying-Shao

PY - 2014/1/1

Y1 - 2014/1/1

N2 - The emphasis on scientific inquiry has increased the importance in developing the fundamental abilities to conduct scientific investigations and urged a need for valid assessments of students' inquiry abilities. We took advantage of the advanced technology to develop a simulation-based assessment of inquiry abilities (SAIA) that allowed students to generate scientific explanations and demonstrate their experimental abilities. This paper describes the validation of the assessment. Data were collected from 48 12th-grade students at a local high school who were categorized into three groups based on their program majors. Both quantitative and qualitative approaches were utilized to validate SAIA. The quantitative results showed that SAIA was aligned with a validated reasoning-skill test (criterion-related validity), discriminated variance among different groups (construct validity), and was highly suitable for examining inquiry abilities (content validity). Additionally, we utilized the think-aloud technique in order to identify the performances exhibited by students while they accomplished the SAIA tasks. The protocol analysis indicated that in general, students demonstrated the expected abilities in SAIA and that their SAIA scores accurately reflected their performance levels of inquiry abilities. The results suggested that SAIA was a valid assessment for evaluating the inquiry abilities of high school students. This study also provided systemic strategies for validating simulation-based assessments.

AB - The emphasis on scientific inquiry has increased the importance in developing the fundamental abilities to conduct scientific investigations and urged a need for valid assessments of students' inquiry abilities. We took advantage of the advanced technology to develop a simulation-based assessment of inquiry abilities (SAIA) that allowed students to generate scientific explanations and demonstrate their experimental abilities. This paper describes the validation of the assessment. Data were collected from 48 12th-grade students at a local high school who were categorized into three groups based on their program majors. Both quantitative and qualitative approaches were utilized to validate SAIA. The quantitative results showed that SAIA was aligned with a validated reasoning-skill test (criterion-related validity), discriminated variance among different groups (construct validity), and was highly suitable for examining inquiry abilities (content validity). Additionally, we utilized the think-aloud technique in order to identify the performances exhibited by students while they accomplished the SAIA tasks. The protocol analysis indicated that in general, students demonstrated the expected abilities in SAIA and that their SAIA scores accurately reflected their performance levels of inquiry abilities. The results suggested that SAIA was a valid assessment for evaluating the inquiry abilities of high school students. This study also provided systemic strategies for validating simulation-based assessments.

KW - Inquiry ability

KW - Simulation-based assessment

KW - Validation

UR - http://www.scopus.com/inward/record.url?scp=84901694075&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84901694075&partnerID=8YFLogxK

U2 - 10.1080/09500693.2013.871660

DO - 10.1080/09500693.2013.871660

M3 - Article

VL - 36

SP - 1630

EP - 1650

JO - International Journal of Science Education

JF - International Journal of Science Education

SN - 0950-0693

IS - 10

ER -