| Title |
Learner’s Concentration Analysis System using Deep Learning based Facial State Detection Model |
| Authors |
여동훈(Dong Hun Yeo) ; 라이언(I Eon Na) ; 황병일(Byeong Il Hwang) ; 김동주(Dong Ju Kim) ; 황도경(Do Kyung Hwang) |
| DOI |
https://doi.org/10.5573/ieie.2023.60.1.37 |
| Keywords |
Student concentration analysis; Real-time inference; Object detection; Face landmark; CNN network |
| Abstract |
This paper proposes a learner concentration analysis system through deep learning-based facial state detection models and algorithms in a face-to-face and non-face-to-face mixed educational environment. Previously, learners' concentration was analyzed using the facial emotion analysis model, posture detection using the image segmentation model, and eye tracking and brain wave. The proposed method detects drowsiness and abnormal behavior through eye blink and head position states for the learner's concentration analysis. We propose a deep learning model-based concentration analysis algorithm that yields concentration through quantitative values, not random confidence values inferred by the model. The algorithm performs drowsiness recognition through eye blink detection and abnormal behavior recognition through head pose detection based on the learner's facial state collected by the camera. It calculates the concentration state in a certain period based on the recognition data. We used face landmarks and object detectors to detect facial conditions and constructed the system by adopting more advantageous object detectors by comparing the accuracy and operating speed. In order to verify the system, learner concentration analysis system calculated through scenarios of 20 and 100 students in non-face-to-face and face-to-face situations, and showed accuracies of 90% and 93%. |