Eye gaze pattern analysis for fatigue detection based on GP-BCNN with ESM

This paper presents a robust fatigue detection system based on binocular consistency, which integrates artificial modulation into deep learning to guide the learning process and removes the extreme cases of dynamic objects through screening mechanism. Specifically, we first build a dual-stream bidirectional convolutional neural network (BCNN) for eye gaze pattern detection, which uses binocular consistency for information interaction. Then we incorporate vectorized local integral projection features which named projection vectors and Gabor filters into BCNN to construct GP-BCNN that not only enhances the resistance of deep learned features to the orientation and scale changes, but strengthens the learning of texture information. Finally, an eye screening mechanism (ESM) based on pupil distance is proposed to eliminate the detected errors caused by the occluded eyes when the lateral face is detected. Demonstrated by introducing binocular consistency and artificial modulation to convolutional neural network (CNN), GP-BCNN improves the widely used CNNs architectures and yields a 2.9% promotion in the average accuracy rate compared with the results obtained by CNN alone. Our approach obtains the state-of-the-art results in fatigue detection and has the generalization potential in general image recognition tasks.