TY - GEN
T1 - Real-time Multi-module Student Engagement Detection System
AU - Ravi, Pooja
AU - Ali Akber Dewan, M.
N1 - Funding Information:
This work was supported by Mitacs Globalink Research Internship Program and NSERC Discovery Grant, Canada.
Publisher Copyright:
© 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
PY - 2023
Y1 - 2023
N2 - We present a method to aggregate four different facial cues to help identify distraction among online learners: facial emotion detection, micro-sleep tracking, yawn detection, and iris distraction detection. In our proposed method, the first module identifies facial emotions using both 2D and 3D convolutional neural networks (CNNs) which facilitates comparison between spatiotemporal and solely spatial features. The other three modules use a 3D facial mesh to localize the eye and lip coordinates in order to track a student’s facial landmarks and identify iris positions as well as signs of micro-sleep like yawns or drowsiness. The results from each module are combined to form an all-encompassing label displayed on an integrated user interface that can further be used to provide real-time alerts to students and instructors when required. From our experiments, the emotion, micro-sleep, yawn, and iris monitoring modules individually achieved 72.5%, 95%, 97%, and 93% accuracy scores, respectively.
AB - We present a method to aggregate four different facial cues to help identify distraction among online learners: facial emotion detection, micro-sleep tracking, yawn detection, and iris distraction detection. In our proposed method, the first module identifies facial emotions using both 2D and 3D convolutional neural networks (CNNs) which facilitates comparison between spatiotemporal and solely spatial features. The other three modules use a 3D facial mesh to localize the eye and lip coordinates in order to track a student’s facial landmarks and identify iris positions as well as signs of micro-sleep like yawns or drowsiness. The results from each module are combined to form an all-encompassing label displayed on an integrated user interface that can further be used to provide real-time alerts to students and instructors when required. From our experiments, the emotion, micro-sleep, yawn, and iris monitoring modules individually achieved 72.5%, 95%, 97%, and 93% accuracy scores, respectively.
KW - 2D and 3D CNNs
KW - Facial landmark detection
KW - Online learning
KW - Spatiotemporal features
KW - Student engagement detection
UR - http://www.scopus.com/inward/record.url?scp=85172235626&partnerID=8YFLogxK
U2 - 10.1007/978-981-99-2100-3_22
DO - 10.1007/978-981-99-2100-3_22
M3 - Published Conference contribution
AN - SCOPUS:85172235626
SN - 9789819920990
T3 - Lecture Notes in Networks and Systems
SP - 261
EP - 278
BT - Communication and Intelligent Systems - Proceedings of ICCIS 2022
A2 - Sharma, Harish
A2 - Shrivastava, Vivek
A2 - Bharti, Kusum Kumari
A2 - Wang, Lipo
T2 - 4th International Conference on Communication and Intelligent Systems, ICCIS 2022
Y2 - 19 December 2022 through 20 December 2022
ER -