TY - JOUR
T1 - Peer assessment of webpage design
T2 - Behavioral sequential analysis based on eye tracking evidence
AU - Hsu, Ting Chia
AU - Chang, Shao Chen
AU - Liu, Nan Cen
N1 - Funding Information:
This study is supported in part by the Ministry of Science and Technology in the Republic of China under contract numbers MOST 105-2628-S-003-002-MY3. This work is also partly supported by the “Aim for the Top University Project” of the National Taiwan Normal University and the Ministry of Education, Taiwan, R.O.C. The authors would like to thank Prof. Hong-Fa Ho who provided the eye-tracking equipment and technical support.
PY - 2018
Y1 - 2018
N2 - This study employed an eye-tracking machine to record the process of peer assessment. Each web page was divided into several regions of interest (ROIs) based on the frame design and content. A total of 49 undergraduate students with a visual learning style participated in the experiment. This study investigated the peer assessment attitudes of the participants and found that they possessed highly positive attitudes towards and understanding of peer assessment. After comparing the results of the peer assessments and evaluation by experts, high consistency occurred when the design of the web page was concise; however, the consistency decreased when the web page content was too diverse. After comparing the peer assessment attitudes of the participants and their web page design scores, it was found that the web pages with concise designs attracted the visual-style students' attention more, and that there was a significant negative correlation for those students who possessed a more negative attitude toward peer assessment. In addition, the study further analyzed the visual-style students' serial behaviors in the peer assessment process for each web page design. After comparing the evidence of each student's eye movements and his/her evaluation results, it was found that the students who gave higher or lower scores had different eye movements. For the website scored as having the best design, the fixations and behaviors of the assessors giving higher scores were highly consistent with those of the experts, implying that the few assessors giving lower scores were relatively poor at peer assessment. On the contrary, for the website which was scored as having the worst design, the fixations and behaviors of the assessors giving lower scores were highly consistent with those of the experts. Consequently, from the eye fixation hotspot evidence, when the students were more concentrated on the peer assessment, their evaluated results were closer to those of the two experts. Finally, the study found that the eye fixation hotspots were the same as the key points planned by the student designers of the website which scored the highest, which provided the student designers with additional important eye-tracking feedback from the peer assessment activities.
AB - This study employed an eye-tracking machine to record the process of peer assessment. Each web page was divided into several regions of interest (ROIs) based on the frame design and content. A total of 49 undergraduate students with a visual learning style participated in the experiment. This study investigated the peer assessment attitudes of the participants and found that they possessed highly positive attitudes towards and understanding of peer assessment. After comparing the results of the peer assessments and evaluation by experts, high consistency occurred when the design of the web page was concise; however, the consistency decreased when the web page content was too diverse. After comparing the peer assessment attitudes of the participants and their web page design scores, it was found that the web pages with concise designs attracted the visual-style students' attention more, and that there was a significant negative correlation for those students who possessed a more negative attitude toward peer assessment. In addition, the study further analyzed the visual-style students' serial behaviors in the peer assessment process for each web page design. After comparing the evidence of each student's eye movements and his/her evaluation results, it was found that the students who gave higher or lower scores had different eye movements. For the website scored as having the best design, the fixations and behaviors of the assessors giving higher scores were highly consistent with those of the experts, implying that the few assessors giving lower scores were relatively poor at peer assessment. On the contrary, for the website which was scored as having the worst design, the fixations and behaviors of the assessors giving lower scores were highly consistent with those of the experts. Consequently, from the eye fixation hotspot evidence, when the students were more concentrated on the peer assessment, their evaluated results were closer to those of the two experts. Finally, the study found that the eye fixation hotspots were the same as the key points planned by the student designers of the website which scored the highest, which provided the student designers with additional important eye-tracking feedback from the peer assessment activities.
KW - Eye tracking
KW - Hotspots
KW - Peer assessment
KW - Region of interest
KW - Serial behavior
UR - http://www.scopus.com/inward/record.url?scp=85046083161&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85046083161&partnerID=8YFLogxK
M3 - Article
AN - SCOPUS:85046083161
VL - 21
SP - 305
EP - 321
JO - Educational Technology and Society
JF - Educational Technology and Society
SN - 1436-4522
IS - 2
ER -