DEVELOPMENT OF AN ENGLISH ORAL ASSESSMENT SYSTEM WITH THE GEPT DATASET

  • Hao Chien Lu
  • , Chung Chun Wang
  • , Jhen Ke Lin
  • , Berlin Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In the rapidly evolving intersection of artificial intelligence and educational technology, multi-perspective assessments are transforming language learning. However, much of the existing research focuses on single-dimensional evaluations, often neglecting the potential of a comprehensive, multi-dimensional oral assessment system that addresses acoustic features, content, and language use simultaneously. Few studies have developed such systems based on established language proficiency tests like the General English Profi-ciency Test (GEPT). This study breaks new ground by creating an integrated, multi-dimensional oral assessment system tailored to the Intermediate level of the GEPT Speaking Test, specifically targeting the Picture Description section. We introduce two novel architectural innovations: the first integrates gram-mar error correction using GECToR and ERRANT into the Language Use Module, while the second enhances the Content Module by combining image and question analy-sis through Blip2 and large language models (LLMs). Our system demonstrates robust performance, achieving 71 % ac-curacy on familiar content and 68% accuracy on unfamiliar content.

Original languageEnglish
Title of host publication2024 27th Conference on the Oriental COCOSDA International Committee for the Co-Ordination and Standardisation of Speech Databases and Assessment Techniques, O-COCOSDA 2024 - Proceedings
EditorsMing-Hsiang Su, Jui-Feng Yeh, Yuan-Fu Liao, Chi-Chun Lee, Yu Taso
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9798331506032
DOIs
Publication statusPublished - 2024
Event27th Conference on the Oriental COCOSDA International Committee for the Co-Ordination and Standardisation of Speech Databases and Assessment Techniques, O-COCOSDA 2024 - Hsinchu, Taiwan
Duration: 2024 Oct 172024 Oct 19

Publication series

Name2024 27th Conference on the Oriental COCOSDA International Committee for the Co-Ordination and Standardisation of Speech Databases and Assessment Techniques, O-COCOSDA 2024 - Proceedings

Conference

Conference27th Conference on the Oriental COCOSDA International Committee for the Co-Ordination and Standardisation of Speech Databases and Assessment Techniques, O-COCOSDA 2024
Country/TerritoryTaiwan
CityHsinchu
Period2024/10/172024/10/19

Keywords

  • GEC
  • LLM
  • automatic speech assessment

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Information Systems
  • Information Systems and Management
  • Safety, Risk, Reliability and Quality
  • Library and Information Sciences
  • Language and Linguistics

Fingerprint

Dive into the research topics of 'DEVELOPMENT OF AN ENGLISH ORAL ASSESSMENT SYSTEM WITH THE GEPT DATASET'. Together they form a unique fingerprint.

Cite this