Assessing learning analytics systems impact by summative measures

Rebecca Guillot; Jeremie Seanosky; Isabelle Guillot; David Boulanger; Claudia Guillot; Vivekanandan Kumar; Shawn N. Fraser; Kinshuk

doi:10.1109/ICALT.2018.00051

Assessing learning analytics systems impact by summative measures

Rebecca Guillot, Jeremie Seanosky, Isabelle Guillot, David Boulanger, Claudia Guillot, Vivekanandan Kumar, Shawn N. Fraser, Kinshuk

School of Computing and Information systems

Research output: Chapter in Book/Report/Conference proceeding › Published Conference contribution › peer-review

4 Citations (Scopus)

Abstract

This paper introduces a randomized study conducted among a group of 48 student Java programmers to assess the impact of learning analytics (LA) on their academic performance. The LA system design incorporated both cognitive and metacognitive tools to help learners take possession of their learning processes. Participation was voluntary and data about potential confounding factors were also collected to minimize bias by blocking on two or more factors (future work). This paper summarily explores the relationships between students' programming expertise, coding assignments, user experience and satisfaction, and academic performance. The results of this preliminary exploration are inconclusive as to whether the LA system made a difference in academic performance. Nevertheless, they seem to indicate that LA was beneficial to student programmers and that summative measures such as grades are not a proper metric to measure the usefulness of LA systems.

Original language	English
Title of host publication	Proceedings - IEEE 18th International Conference on Advanced Learning Technologies, ICALT 2018
Editors	Nian-Shing Chen, Maiga Chang, Ronghuai Huang, K. Kinshuk, Kannan Moudgalya, Sahana Murthy, Demetrios G Sampson
Pages	188-190
Number of pages	3
DOIs	https://doi.org/10.1109/ICALT.2018.00051
Publication status	Published - 10 Aug. 2018
Event	18th IEEE International Conference on Advanced Learning Technologies, ICALT 2018 - Bombay, India Duration: 9 Jul. 2018 → 13 Jul. 2018

Publication series

Name	Proceedings - IEEE 18th International Conference on Advanced Learning Technologies, ICALT 2018

Conference

Conference	18th IEEE International Conference on Advanced Learning Technologies, ICALT 2018
Country/Territory	India
City	Bombay
Period	9/07/18 → 13/07/18

Keywords

Academic performance
Coding
Competence assessment
Learning analytics
Summative feedback
User experience

Access to Document

10.1109/ICALT.2018.00051

Cite this

Guillot, R., Seanosky, J., Guillot, I., Boulanger, D., Guillot, C., Kumar, V., Fraser, S. N., & Kinshuk (2018). Assessing learning analytics systems impact by summative measures. In N.-S. Chen, M. Chang, R. Huang, K. Kinshuk, K. Moudgalya, S. Murthy, & D. G. Sampson (Eds.), Proceedings - IEEE 18th International Conference on Advanced Learning Technologies, ICALT 2018 (pp. 188-190). Article 8433489 (Proceedings - IEEE 18th International Conference on Advanced Learning Technologies, ICALT 2018). https://doi.org/10.1109/ICALT.2018.00051

Guillot, Rebecca ; Seanosky, Jeremie ; Guillot, Isabelle et al. / Assessing learning analytics systems impact by summative measures. Proceedings - IEEE 18th International Conference on Advanced Learning Technologies, ICALT 2018. editor / Nian-Shing Chen ; Maiga Chang ; Ronghuai Huang ; K. Kinshuk ; Kannan Moudgalya ; Sahana Murthy ; Demetrios G Sampson. 2018. pp. 188-190 (Proceedings - IEEE 18th International Conference on Advanced Learning Technologies, ICALT 2018).

@inproceedings{d465a3276f9f4e1193fcbd10095b6e87,

title = "Assessing learning analytics systems impact by summative measures",

abstract = "This paper introduces a randomized study conducted among a group of 48 student Java programmers to assess the impact of learning analytics (LA) on their academic performance. The LA system design incorporated both cognitive and metacognitive tools to help learners take possession of their learning processes. Participation was voluntary and data about potential confounding factors were also collected to minimize bias by blocking on two or more factors (future work). This paper summarily explores the relationships between students' programming expertise, coding assignments, user experience and satisfaction, and academic performance. The results of this preliminary exploration are inconclusive as to whether the LA system made a difference in academic performance. Nevertheless, they seem to indicate that LA was beneficial to student programmers and that summative measures such as grades are not a proper metric to measure the usefulness of LA systems.",

keywords = "Academic performance, Coding, Competence assessment, Learning analytics, Summative feedback, User experience",

author = "Rebecca Guillot and Jeremie Seanosky and Isabelle Guillot and David Boulanger and Claudia Guillot and Vivekanandan Kumar and Fraser, {Shawn N.} and Kinshuk",

note = "Publisher Copyright: {\textcopyright} 2018 IEEE.; 18th IEEE International Conference on Advanced Learning Technologies, ICALT 2018 ; Conference date: 09-07-2018 Through 13-07-2018",

year = "2018",

month = aug,

day = "10",

doi = "10.1109/ICALT.2018.00051",

language = "English",

isbn = "9781538660492",

series = "Proceedings - IEEE 18th International Conference on Advanced Learning Technologies, ICALT 2018",

pages = "188--190",

editor = "Nian-Shing Chen and Maiga Chang and Ronghuai Huang and K. Kinshuk and Kannan Moudgalya and Sahana Murthy and Sampson, {Demetrios G}",

booktitle = "Proceedings - IEEE 18th International Conference on Advanced Learning Technologies, ICALT 2018",

}

Guillot, R, Seanosky, J, Guillot, I, Boulanger, D, Guillot, C, Kumar, V , Fraser, SN & Kinshuk 2018, Assessing learning analytics systems impact by summative measures. in N-S Chen, M Chang, R Huang, K Kinshuk, K Moudgalya, S Murthy & DG Sampson (eds), Proceedings - IEEE 18th International Conference on Advanced Learning Technologies, ICALT 2018., 8433489, Proceedings - IEEE 18th International Conference on Advanced Learning Technologies, ICALT 2018, pp. 188-190, 18th IEEE International Conference on Advanced Learning Technologies, ICALT 2018, Bombay, India, 9/07/18. https://doi.org/10.1109/ICALT.2018.00051

Assessing learning analytics systems impact by summative measures. / Guillot, Rebecca; Seanosky, Jeremie; Guillot, Isabelle et al.
Proceedings - IEEE 18th International Conference on Advanced Learning Technologies, ICALT 2018. ed. / Nian-Shing Chen; Maiga Chang; Ronghuai Huang; K. Kinshuk; Kannan Moudgalya; Sahana Murthy; Demetrios G Sampson. 2018. p. 188-190 8433489 (Proceedings - IEEE 18th International Conference on Advanced Learning Technologies, ICALT 2018).

Research output: Chapter in Book/Report/Conference proceeding › Published Conference contribution › peer-review

TY - GEN

T1 - Assessing learning analytics systems impact by summative measures

AU - Guillot, Rebecca

AU - Seanosky, Jeremie

AU - Guillot, Isabelle

AU - Boulanger, David

AU - Guillot, Claudia

AU - Kumar, Vivekanandan

AU - Fraser, Shawn N.

AU - Kinshuk,

PY - 2018/8/10

Y1 - 2018/8/10

N2 - This paper introduces a randomized study conducted among a group of 48 student Java programmers to assess the impact of learning analytics (LA) on their academic performance. The LA system design incorporated both cognitive and metacognitive tools to help learners take possession of their learning processes. Participation was voluntary and data about potential confounding factors were also collected to minimize bias by blocking on two or more factors (future work). This paper summarily explores the relationships between students' programming expertise, coding assignments, user experience and satisfaction, and academic performance. The results of this preliminary exploration are inconclusive as to whether the LA system made a difference in academic performance. Nevertheless, they seem to indicate that LA was beneficial to student programmers and that summative measures such as grades are not a proper metric to measure the usefulness of LA systems.

AB - This paper introduces a randomized study conducted among a group of 48 student Java programmers to assess the impact of learning analytics (LA) on their academic performance. The LA system design incorporated both cognitive and metacognitive tools to help learners take possession of their learning processes. Participation was voluntary and data about potential confounding factors were also collected to minimize bias by blocking on two or more factors (future work). This paper summarily explores the relationships between students' programming expertise, coding assignments, user experience and satisfaction, and academic performance. The results of this preliminary exploration are inconclusive as to whether the LA system made a difference in academic performance. Nevertheless, they seem to indicate that LA was beneficial to student programmers and that summative measures such as grades are not a proper metric to measure the usefulness of LA systems.

KW - Academic performance

KW - Coding

KW - Competence assessment

KW - Learning analytics

KW - Summative feedback

KW - User experience

UR - http://www.scopus.com/inward/record.url?scp=85052501306&partnerID=8YFLogxK

U2 - 10.1109/ICALT.2018.00051

DO - 10.1109/ICALT.2018.00051

M3 - Published Conference contribution

AN - SCOPUS:85052501306

SN - 9781538660492

T3 - Proceedings - IEEE 18th International Conference on Advanced Learning Technologies, ICALT 2018

SP - 188

EP - 190

BT - Proceedings - IEEE 18th International Conference on Advanced Learning Technologies, ICALT 2018

A2 - Chen, Nian-Shing

A2 - Chang, Maiga

A2 - Huang, Ronghuai

A2 - Kinshuk, K.

A2 - Moudgalya, Kannan

A2 - Murthy, Sahana

A2 - Sampson, Demetrios G

T2 - 18th IEEE International Conference on Advanced Learning Technologies, ICALT 2018

Y2 - 9 July 2018 through 13 July 2018

ER -

Guillot R, Seanosky J, Guillot I, Boulanger D, Guillot C, Kumar V et al. Assessing learning analytics systems impact by summative measures. In Chen NS, Chang M, Huang R, Kinshuk K, Moudgalya K, Murthy S, Sampson DG, editors, Proceedings - IEEE 18th International Conference on Advanced Learning Technologies, ICALT 2018. 2018. p. 188-190. 8433489. (Proceedings - IEEE 18th International Conference on Advanced Learning Technologies, ICALT 2018). doi: 10.1109/ICALT.2018.00051

Assessing learning analytics systems impact by summative measures

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this