Peer assessment of webpage design: Behavioral sequential analysis based on eye tracking evidence

Ting Chia Hsu, Shao Chen Chang, Nan Cen Liu

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

This study employed an eye-tracking machine to record the process of peer assessment. Each web page was divided into several regions of interest (ROIs) based on the frame design and content. A total of 49 undergraduate students with a visual learning style participated in the experiment. This study investigated the peer assessment attitudes of the participants and found that they possessed highly positive attitudes towards and understanding of peer assessment. After comparing the results of the peer assessments and evaluation by experts, high consistency occurred when the design of the web page was concise; however, the consistency decreased when the web page content was too diverse. After comparing the peer assessment attitudes of the participants and their web page design scores, it was found that the web pages with concise designs attracted the visual-style students' attention more, and that there was a significant negative correlation for those students who possessed a more negative attitude toward peer assessment. In addition, the study further analyzed the visual-style students' serial behaviors in the peer assessment process for each web page design. After comparing the evidence of each student's eye movements and his/her evaluation results, it was found that the students who gave higher or lower scores had different eye movements. For the website scored as having the best design, the fixations and behaviors of the assessors giving higher scores were highly consistent with those of the experts, implying that the few assessors giving lower scores were relatively poor at peer assessment. On the contrary, for the website which was scored as having the worst design, the fixations and behaviors of the assessors giving lower scores were highly consistent with those of the experts. Consequently, from the eye fixation hotspot evidence, when the students were more concentrated on the peer assessment, their evaluated results were closer to those of the two experts. Finally, the study found that the eye fixation hotspots were the same as the key points planned by the student designers of the website which scored the highest, which provided the student designers with additional important eye-tracking feedback from the peer assessment activities.

Original languageEnglish
Pages (from-to)305-321
Number of pages17
JournalEducational Technology and Society
Volume21
Issue number2
Publication statusPublished - 2018 Jan 1

Fingerprint

Websites
Students
evidence
student
expert
Eye movements
website
sequential analysis
student movement
evaluation
Feedback
experiment
learning

Keywords

  • Eye tracking
  • Hotspots
  • Peer assessment
  • Region of interest
  • Serial behavior

ASJC Scopus subject areas

  • Education
  • Sociology and Political Science
  • Engineering(all)

Cite this

Peer assessment of webpage design : Behavioral sequential analysis based on eye tracking evidence. / Hsu, Ting Chia; Chang, Shao Chen; Liu, Nan Cen.

In: Educational Technology and Society, Vol. 21, No. 2, 01.01.2018, p. 305-321.

Research output: Contribution to journalArticle

@article{32b538ee518c435ca2c55072b6e0c89d,
title = "Peer assessment of webpage design: Behavioral sequential analysis based on eye tracking evidence",
abstract = "This study employed an eye-tracking machine to record the process of peer assessment. Each web page was divided into several regions of interest (ROIs) based on the frame design and content. A total of 49 undergraduate students with a visual learning style participated in the experiment. This study investigated the peer assessment attitudes of the participants and found that they possessed highly positive attitudes towards and understanding of peer assessment. After comparing the results of the peer assessments and evaluation by experts, high consistency occurred when the design of the web page was concise; however, the consistency decreased when the web page content was too diverse. After comparing the peer assessment attitudes of the participants and their web page design scores, it was found that the web pages with concise designs attracted the visual-style students' attention more, and that there was a significant negative correlation for those students who possessed a more negative attitude toward peer assessment. In addition, the study further analyzed the visual-style students' serial behaviors in the peer assessment process for each web page design. After comparing the evidence of each student's eye movements and his/her evaluation results, it was found that the students who gave higher or lower scores had different eye movements. For the website scored as having the best design, the fixations and behaviors of the assessors giving higher scores were highly consistent with those of the experts, implying that the few assessors giving lower scores were relatively poor at peer assessment. On the contrary, for the website which was scored as having the worst design, the fixations and behaviors of the assessors giving lower scores were highly consistent with those of the experts. Consequently, from the eye fixation hotspot evidence, when the students were more concentrated on the peer assessment, their evaluated results were closer to those of the two experts. Finally, the study found that the eye fixation hotspots were the same as the key points planned by the student designers of the website which scored the highest, which provided the student designers with additional important eye-tracking feedback from the peer assessment activities.",
keywords = "Eye tracking, Hotspots, Peer assessment, Region of interest, Serial behavior",
author = "Hsu, {Ting Chia} and Chang, {Shao Chen} and Liu, {Nan Cen}",
year = "2018",
month = "1",
day = "1",
language = "English",
volume = "21",
pages = "305--321",
journal = "Educational Technology and Society",
issn = "1436-4522",
publisher = "IEEE Computer Society",
number = "2",

}

TY - JOUR

T1 - Peer assessment of webpage design

T2 - Behavioral sequential analysis based on eye tracking evidence

AU - Hsu, Ting Chia

AU - Chang, Shao Chen

AU - Liu, Nan Cen

PY - 2018/1/1

Y1 - 2018/1/1

N2 - This study employed an eye-tracking machine to record the process of peer assessment. Each web page was divided into several regions of interest (ROIs) based on the frame design and content. A total of 49 undergraduate students with a visual learning style participated in the experiment. This study investigated the peer assessment attitudes of the participants and found that they possessed highly positive attitudes towards and understanding of peer assessment. After comparing the results of the peer assessments and evaluation by experts, high consistency occurred when the design of the web page was concise; however, the consistency decreased when the web page content was too diverse. After comparing the peer assessment attitudes of the participants and their web page design scores, it was found that the web pages with concise designs attracted the visual-style students' attention more, and that there was a significant negative correlation for those students who possessed a more negative attitude toward peer assessment. In addition, the study further analyzed the visual-style students' serial behaviors in the peer assessment process for each web page design. After comparing the evidence of each student's eye movements and his/her evaluation results, it was found that the students who gave higher or lower scores had different eye movements. For the website scored as having the best design, the fixations and behaviors of the assessors giving higher scores were highly consistent with those of the experts, implying that the few assessors giving lower scores were relatively poor at peer assessment. On the contrary, for the website which was scored as having the worst design, the fixations and behaviors of the assessors giving lower scores were highly consistent with those of the experts. Consequently, from the eye fixation hotspot evidence, when the students were more concentrated on the peer assessment, their evaluated results were closer to those of the two experts. Finally, the study found that the eye fixation hotspots were the same as the key points planned by the student designers of the website which scored the highest, which provided the student designers with additional important eye-tracking feedback from the peer assessment activities.

AB - This study employed an eye-tracking machine to record the process of peer assessment. Each web page was divided into several regions of interest (ROIs) based on the frame design and content. A total of 49 undergraduate students with a visual learning style participated in the experiment. This study investigated the peer assessment attitudes of the participants and found that they possessed highly positive attitudes towards and understanding of peer assessment. After comparing the results of the peer assessments and evaluation by experts, high consistency occurred when the design of the web page was concise; however, the consistency decreased when the web page content was too diverse. After comparing the peer assessment attitudes of the participants and their web page design scores, it was found that the web pages with concise designs attracted the visual-style students' attention more, and that there was a significant negative correlation for those students who possessed a more negative attitude toward peer assessment. In addition, the study further analyzed the visual-style students' serial behaviors in the peer assessment process for each web page design. After comparing the evidence of each student's eye movements and his/her evaluation results, it was found that the students who gave higher or lower scores had different eye movements. For the website scored as having the best design, the fixations and behaviors of the assessors giving higher scores were highly consistent with those of the experts, implying that the few assessors giving lower scores were relatively poor at peer assessment. On the contrary, for the website which was scored as having the worst design, the fixations and behaviors of the assessors giving lower scores were highly consistent with those of the experts. Consequently, from the eye fixation hotspot evidence, when the students were more concentrated on the peer assessment, their evaluated results were closer to those of the two experts. Finally, the study found that the eye fixation hotspots were the same as the key points planned by the student designers of the website which scored the highest, which provided the student designers with additional important eye-tracking feedback from the peer assessment activities.

KW - Eye tracking

KW - Hotspots

KW - Peer assessment

KW - Region of interest

KW - Serial behavior

UR - http://www.scopus.com/inward/record.url?scp=85046083161&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85046083161&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:85046083161

VL - 21

SP - 305

EP - 321

JO - Educational Technology and Society

JF - Educational Technology and Society

SN - 1436-4522

IS - 2

ER -