A Generative Artificial Intelligence Empowered Chatbot: System Usability and Student Teachers’ Experience

Stavros A. Nikou, Arjun Guliya, Suraj Van Verma, Maiga Chang

    Research output: Chapter in Book/Report/Conference proceedingPublished Conference contributionpeer-review

    Abstract

    Generative Artificial Intelligence empowered conversational agents (chatbots) seem to be increasingly used in various settings including education. While student teachers are key stakeholders in supporting and improving education, not many studies exist in student teachers’ views on the educational use of chatbots. The current study performs a usability evaluation and explores student teachers’ views on the academic use of the VIP-Bot, an advanced academic Discord chatbot, which leverages the OpenAI’s gpt-3.5-turbo-instruct model. Student teachers, within the context of the formative task of writing a literature review, interacted with the chatbot and self-reported their experiences through an online survey. The usability evaluation returned a relatively high SUS score (76.36) for the chatbot. Moreover, student teachers’ view on the chatbot acceptance, effectiveness and motivation were positive. The chatbot can be helpful in developing ideas and initiating further engagement with the literature. Academic misconduct concerns have been expressed if the chatbot is not used properly. The study, as a usability evaluation, is an essential step in further chatbot development and, as an investigation of student teachers’ views, it is an essential step on the chatbot employment in teaching and learning.

    Original languageEnglish
    Title of host publicationGenerative Intelligence and Intelligent Tutoring Systems - 20th International Conference, ITS 2024, Proceedings
    EditorsAngelo Sifaleras, Fuhua Lin
    Pages330-340
    Number of pages11
    DOIs
    Publication statusPublished - 2024
    Event20th International Conference on Generative Intelligence and Intelligent Tutoring Systems, ITS 2024 - Thessaloniki, Greece
    Duration: 10 Jun. 202413 Jun. 2024

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume14798 LNCS
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Conference

    Conference20th International Conference on Generative Intelligence and Intelligent Tutoring Systems, ITS 2024
    Country/TerritoryGreece
    CityThessaloniki
    Period10/06/2413/06/24

    Keywords

    • Discord bot
    • Generative Pre-trained Transformer (GPT)
    • acceptance
    • chatbot
    • effectiveness
    • gpt-3.5-turbo-instruct
    • motivation
    • system usability

    Fingerprint

    Dive into the research topics of 'A Generative Artificial Intelligence Empowered Chatbot: System Usability and Student Teachers’ Experience'. Together they form a unique fingerprint.

    Cite this