Artificial intelligence as an automated essay scoring tool: A focus on ChatGPT

Basit Öğe Kaydı
dc.contributor.authorUyar, Ahmet Can
dc.contributor.authorBuyukahiska, Dilek
dc.date.accessioned2025-05-04T16:45:54Z
dc.date.available2025-05-04T16:45:54Z
dc.date.issued2025
dc.departmentSivas Cumhuriyet Üniversitesi
dc.description.abstractThis study explores the effectiveness of using ChatGPT, an Artificial Intelligence (AI) language model, as an Automated Essay Scoring (AES) tool for grading English as a Foreign Language (EFL) learners' essays. The corpus consists of 50 essays representing various types including analysis, compare and contrast, descriptive, narrative, and opinion essays written by 10 EFL learners at the B2 level. Human raters and ChatGPT (4o mini version) scored the essays using the International English Language Testing System (IELTS) TASK 2 Writing band descriptors. Adopting a quantitative approach, the Wilcoxon signed-rank tests and Spearman correlation tests were employed to compare the scores generated, revealing a significant difference between the two methods of scoring, with human raters assigning higher scores than ChatGPT. Similarly, significant differences with varying degrees were also evident for each of the various types of essays, suggesting that the genre of the essays was not a parameter affecting the agreement between human raters and ChatGPT. After all, it was discussed that while ChatGPT shows promise as an AES tool, the observed disparities suggest that it has not reached sufficient proficiency for practical use. The study emphasizes the need for improvements in AI language models to meet the nuanced nature of essay evaluation in EFL contexts.
dc.identifier.doi10.21449/ijate.1517994
dc.identifier.endpage32
dc.identifier.issn2148-7456
dc.identifier.issue1
dc.identifier.scopusqualityN/A
dc.identifier.startpage20
dc.identifier.urihttps://doi.org/10.21449/ijate.1517994
dc.identifier.urihttps://hdl.handle.net/20.500.12418/35273
dc.identifier.volume12
dc.identifier.wosWOS:001429124500002
dc.identifier.wosqualityN/A
dc.indekslendigikaynakWeb of Science
dc.language.isoen
dc.publisherIzzet Kara
dc.relation.ispartofInternational Journal of Assessment Tools in Education
dc.relation.publicationcategoryMakale - Uluslararası Hakemli Dergi - Kurum Öğretim Elemanı
dc.rightsinfo:eu-repo/semantics/openAccess
dc.snmzKA_WOS_20250504
dc.subjectAutomated essay scoring
dc.subjectArtificial intelligence
dc.subjectChatGPT
dc.subjectForeign language writing
dc.subjectWriting evaluation
dc.titleArtificial intelligence as an automated essay scoring tool: A focus on ChatGPT
dc.typeArticle

Dosyalar

Koleksiyon

WoS İndeksli Yayınlar Koleksiyonu