SHAPed automated essay scoring: explaining writing features’ contributions to english writing organization

David Boulanger, Vivekanandan Kumar

Research output: Chapter in Book/Report/Conference proceedingPublished Conference contributionpeer-review

4 Citations (Scopus)

Abstract

This study applies the state of the art in explainable AI techniques to shed light on the automated essay scoring (AES) process. By means of linear regression and Shapley values, SHAP (Shapley Additive Explanations) approximates a complex AES predictive model implemented as a deep neural network and an ensemble regression. This study delves into the essentials of the automated assessment of ‘organization’, a key rubric in writing. Specifically, it explores whether the organization and connections between ideas and/or events are clear and logically sequenced. Built on findings from previous work, this paper, in addition to improving the generalizability and interpretability of the AES model, highlights the means to identify important ‘writing features’ (both global and local) and hint at the best ranges of feature values. By associating ‘organization’ with ‘writing features’, it provides a mechanism to hypothesize causal relationships among variables and shape machine-learned formative feedback in human-friendly terms for the consumption of teachers and students. Finally, it offers an in-depth discussion on linguistic aspects implied by the findings.

Original languageEnglish
Title of host publicationIntelligent Tutoring Systems - 16th International Conference, ITS 2020, Proceedings
EditorsVivekanandan Kumar, Christos Troussas
Pages68-78
Number of pages11
DOIs
Publication statusPublished - 2020
Event16th International Conference on Intelligent Tutoring Systems, ITS 2020 - Athens, Greece
Duration: 8 Jun. 202012 Jun. 2020

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12149 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference16th International Conference on Intelligent Tutoring Systems, ITS 2020
Country/TerritoryGreece
CityAthens
Period8/06/2012/06/20

Keywords

  • Automated essay scoring
  • Deep learning
  • Explainable artificial intelligence
  • Linguistic analysis
  • Organization
  • Rubric
  • SHAP

Fingerprint

Dive into the research topics of 'SHAPed automated essay scoring: explaining writing features’ contributions to english writing organization'. Together they form a unique fingerprint.

Cite this