Detecting learner drowsiness based on facial expressions and head movements in online courses

Abstract

Drowsiness is a major factor that hinders learning. To improve learning efficiency, it is important to understand students' physical status such as wakefulness during online coursework. In this study, we have proposed a drowsiness estimation method based on learners' head and facial movements while viewing video lectures. To examine the effectiveness of head and facial movements in drowsiness estimation, we collected learner video data recorded during e-learning and applied a deep learning approach under the following conditions: (a) using only facial movement data, (b) using only head movement data, and (c) using both facial and head movement data.We achieved an average F1-macro score of 0.74 in personalized models for detecting learner drowsiness using both facial and head movement data.

Noriko Takemura

Guest Associate Professor

She is working on ambient intelligence and gait recognition using pattern recognition and machine learning.

Hajime Nagahara

Professor

He is working on computer vision and pattern recognition. His main research interests lie in image/video recognition and understanding, as well as applications of natural language processing techniques.