Enhancing Access to Educational Data for Educators and Learning Designers: Staged Evaluation of the Academic Analytics Tool

To meet student demand for responsive, adaptable, and up-to-date online courses, educators and learning designers need tools to analyse student interactions with their peers, educators and learning resources. Learning Management Systems (LMSs) store large volumes of detailed user data, but offer only limited, pre-set reports and visualizations to inform analytical studies for educators or learning designers to better understand how students actually learn in online courses. Existing software tools which mine LMS data offer limited usability for non-technical users, and do not permit full, unlimited data exploration. The Academic Analytics Tool (AAT) allows educators and learning designers to investigate student behaviours and outcomes using the data stored on LMS databases. To optimize the usability and application potential of AAT, a staged evaluation was conducted. This paper aims at introducing the latest version of AAT and the outcomes of a staged evaluation to investigate the perceived usefulness, effectiveness and user-friendliness of the latest version of AAT. The evaluation showed that users considered AAT to be useful, effective and relatively user-friendly considering the complex tasks that it can perform. Providing educators and learning designers with a tool to investigate their own research questions using LMS data allows them to increase their understanding of how learners learn in online or blended courses as well as how well their teaching strategies and course/learning designs support students. By doing so, AAT streamlines the ability of educators and learning designers to continuously improve their courses.