Learning by Building Chatbot: A System Usability Study and Teachers’ Views About the Educational Uses of Chatbots

This article describes an experiment to investigate the usability of the “Learning by Building Chatbot” environment and the views of in-service teachers about the potential educational uses of chatbots. Chatbots are text or voice-based conversational interfaces that use natural language to simulate human conversations serving as virtual assistants to users. They have been used in a variety of areas such as e-commerce and healthcare while their use in education is relatively new. Chatbots have the potential to offer new dynamic forms of interactions, but there is often a burden to users to deploy and maintain their own chatbots. Our research group has proposed a web-based environment, “Learning by Building Chatbot”, that alleviates this burden by offering educators and students without any programming knowledge the opportunity to deploy and maintain their own chatbots for learning and training. The environment is a block-based, visual editing environment of creating RiveScript-powered chatbots for learning and training, without needing to know any RiveScript. Twelve in-service teachers registered in a postgraduate program in technology-enhanced learning used this environment to create their own chatbots. Afterwards, they completed the System Usability Scale (SUS) questionnaire. Results have shown that the visualized editing environment for building chatbot is marginally acceptable and fair to use with the SUS score 57.50. Participants found the system ease to use and felt quite confident to use it. Its functions are well integrated while there are a few inconsistencies in the current version. Moreover, participants self-reported their views about the educational uses of chatbots. They agreed that chatbots, if used appropriately, can be a valuable educational tool because they can automate administrative and teaching tasks and can be supportive and engaging for students. However, since chatbot development is challenging, teacher should be well supported to integrate chatbots in the educational practice if they intend to do so.