TY - GEN
T1 - SHAPed automated essay scoring
T2 - 16th International Conference on Intelligent Tutoring Systems, ITS 2020
AU - Boulanger, David
AU - Kumar, Vivekanandan
N1 - Publisher Copyright:
© Springer Nature Switzerland AG 2020.
PY - 2020
Y1 - 2020
N2 - This study applies the state of the art in explainable AI techniques to shed light on the automated essay scoring (AES) process. By means of linear regression and Shapley values, SHAP (Shapley Additive Explanations) approximates a complex AES predictive model implemented as a deep neural network and an ensemble regression. This study delves into the essentials of the automated assessment of ‘organization’, a key rubric in writing. Specifically, it explores whether the organization and connections between ideas and/or events are clear and logically sequenced. Built on findings from previous work, this paper, in addition to improving the generalizability and interpretability of the AES model, highlights the means to identify important ‘writing features’ (both global and local) and hint at the best ranges of feature values. By associating ‘organization’ with ‘writing features’, it provides a mechanism to hypothesize causal relationships among variables and shape machine-learned formative feedback in human-friendly terms for the consumption of teachers and students. Finally, it offers an in-depth discussion on linguistic aspects implied by the findings.
AB - This study applies the state of the art in explainable AI techniques to shed light on the automated essay scoring (AES) process. By means of linear regression and Shapley values, SHAP (Shapley Additive Explanations) approximates a complex AES predictive model implemented as a deep neural network and an ensemble regression. This study delves into the essentials of the automated assessment of ‘organization’, a key rubric in writing. Specifically, it explores whether the organization and connections between ideas and/or events are clear and logically sequenced. Built on findings from previous work, this paper, in addition to improving the generalizability and interpretability of the AES model, highlights the means to identify important ‘writing features’ (both global and local) and hint at the best ranges of feature values. By associating ‘organization’ with ‘writing features’, it provides a mechanism to hypothesize causal relationships among variables and shape machine-learned formative feedback in human-friendly terms for the consumption of teachers and students. Finally, it offers an in-depth discussion on linguistic aspects implied by the findings.
KW - Automated essay scoring
KW - Deep learning
KW - Explainable artificial intelligence
KW - Linguistic analysis
KW - Organization
KW - Rubric
KW - SHAP
UR - http://www.scopus.com/inward/record.url?scp=85086228646&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-49663-0_10
DO - 10.1007/978-3-030-49663-0_10
M3 - Published Conference contribution
AN - SCOPUS:85086228646
SN - 9783030496623
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 68
EP - 78
BT - Intelligent Tutoring Systems - 16th International Conference, ITS 2020, Proceedings
A2 - Kumar, Vivekanandan
A2 - Troussas, Christos
Y2 - 8 June 2020 through 12 June 2020
ER -