Development and Evaluation of a Mouse Emulator Using Multi-modal Real-Time Head Tracking Systems with Facial Gesture Recognition as a Switching Mechanism
Authors: 
Shivanand P. Guness, Farzin Deravi, Konstantinos Sirlantzis, Matthew G. Pepper, Mohammed SakelAuthors Info & Claims
Pages 218 - 232
Abstract
The objective of this study is to evaluate and compare the performance of a set of low-cost multi-modal head tracking systems incorporating facial gestures as a switching mechanism. The proposed systems are aimed to enable severely disabled patients to access a computer. In this paper, we are comparing RGB (2D) and RGB-D (3D) sensors for both head tracking and facial gesture recognition. System evaluations and usability assessment were carried out on 21 healthy individuals. Two types of head tracking systems were compared - a web camera-based and another using the Kinect sensor. The two facial switching mechanisms were eye blink and eyebrows movement. Fitts’ Test is used to evaluate the proposed systems. Movement Time (MT) was used to rank the performance of the proposed systems. The Kinect-Eyebrows system had the lowest MT, followed by the Kinect-Blink, Webcam-Blink and Webcam-Eyebrows systems. The 3D Kinect systems performed better than the 2D Vision systems for both gestures. Both Kinect systems have the lowest MT and best performance, thus showing the advantage of using depth.
References
[1]
Kinect for Windows. http://www.microsoft.com/en-us/kinectforwindows/
[2]
[3]
Tobii Technology. http://www.tobii.com/
[4]
Abascal J and Nicolle CMoving towards inclusive design guidelines for socially and ethically aware HCIInteract. Comput.2005175484-505http://iwc.oxfordjournals.org/cgi/doi/10.1016/j.intcom.2005.03.002
[5]
Accot, J., Zhai, S.: Beyond Fitts’ law: models for trajectory-based HCI tasks. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, pp. 295–301 (1997). http://dl.acm.org/citation.cfm?id=258760
[6]
Betke M, Gips J, and Fleming P The camera mouse: visual tracking of body features to provide computer access for people with severe disabilities IEEE Trans. Neural Syst. Rehabil. Eng. 2002 10 1 1-10 A Publication of the IEEE Engineering in Medicine and Biology Society
[7]
Betke M Nakashima H, Aghajan H, and Augusto JC Intelligent interfaces to empower people with disabilities Handbook of Ambient Intelligence and Smart Environments 2010 Boston Springer 409-432
[8]
Bradski G The OpenCV library Dr. Dobb’s J. Softw. Tools 2000 25 120-125
[9]
Cloud, R., Betke, M., Gips, J.: Experiments with a camera-based human-computer interface system. In: Proceedings of the 7th ERCIM Workshop “User Interfaces for All,” UI4ALL 2002, pp. 103–110 (2002). http://cstest.bc.edu/~gips/UI4ALL-2002.pdf www.cs.bu.edu/faculty/betke/papers/Cloud-Betke-Gips-UI4ALL-2002.pdf
[10]
Cox DR and Snell EJ Analysis of Binary Data 1989 Boca Raton CRC Press
[11]
del Val L, Jiménez MI, Alonso A, de la Rosa R, Izquierdo A, and Carrera A Lytras MD, Ordonez De Pablos P, Ziderman A, Roulstone A, Maurer H, and Imber JB Assistance system for disabled people: a robot controlled by blinking and wireless link Knowledge Management, Information Systems, E-Learning, and Sustainability Research 2010 Heidelberg Springer 383-388
[12]
Douglas, S.A., Kirkpatrick, A.E., MacKenzie, I.S.: Testing pointing device performance and user assessment with the ISO 9241, part 9 standard. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 1999, pp. 215–222. ACM, New York (1999).
[13]
Fitts PM and Radford B Information capacity of discrete motor responses under different cognitive sets J. Exp. Psychol. 1966 71 475-482
[14]
Fitts, P.: The information capacity of the human motor system in controlling the amplitude of movement. J. Exp. Psychol. 47(6) (1954). http://psycnet.apa.org/journals/xge/47/6/381/
[15]
Grauman K, Betke M, Lombardi J, Gips J, and Bradski G Communication via eye blinks and eyebrow raises: video-based human-computer interfaces Univ. Access Inf. Soc. 2003 2 4 359-373
[16]
Guness, S.P., Deravi, F., Sirlantzis, K., Pepper, M.G., Sakel, M.: Evaluation of vision-based head-trackers for assistive devices. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, pp. 4804–4807 (2012). http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6347068
[17]
Gustavsson, A., et al.: Cost of disorders of the brain in Europe 2010. Eur. Neuropsychopharmacol. J. Eur. Coll. Neuropsychopharmacol. 21(10), 718–79 (2011). http://www.ncbi.nlm.nih.gov/pubmed/21924589
[18]
ISO TC 159/SC 4: ISO 9241–9:2000, ergonomic requirements for office work with visual display terminals (VDTs) - part 9: requirements for non-keyboard input devices. International Organization for Standardization (2002)
[19]
Karson, C.N.: Spontaneous eye-blink rates and dopaminergic systems. Brain: J. Neurol. 106(Pt 3), 643–53 (1983). http://www.ncbi.nlm.nih.gov/pubmed/6640274
[20]
Kass M, Witkin A, and Terzopoulos D Snakes: active contour models Int. J. Comput. Vis. 1988 1 4 321-331
[21]
King DE Dlib-ml: a machine learning toolkit J. Mach. Learn. Res. (JMLR) 2009 10 1755-1758 http://dl.acm.org/citation.cfm?id=1577069.1755843
[22]
Krolak, A., Strumillo, P.: Vision-based eye blink monitoring system for human-computer interfacing. In: 2008 Conference on Human System Interactions. Institute of Electrical & Electronics Engineers (IEEE), May 2008.
[23]
MacKenzie IS Fitts’ law as a research and design tool in human-computer interaction Hum. -Comput. Interact. 1992 7 1 91-139
[24]
MacKenzie, I.S., Buxton, W.: A tool for the rapid evaluation of input devices using Fitts’ law models. ACM SIGCHI Bull. 25(3), 58–63 (1993). http://portal.acm.org/citation.cfm?doid=155786.155801
[25]
Malik, K., Smolka, B.: Eye blink detection using local binary patterns. In: 2014 International Conference on Multimedia Computing and Systems (ICMCS), pp. 385–390, April 2014.
[26]
Missimer, E., Betke, M.: Blink and wink detection for mouse pointer control. ACM Press, New York (2010). http://portal.acm.org/citation.cfm?doid=1839294.1839322
[27]
Ojala T, Pietikäinen M, and Harwood D A comparative study of texture measures with classification based on featured distributions Pattern Recogn. 1996 29 1 51-59
[28]
Pan, G., Sun, L., Wu, Z., Lao, S.: Eyeblink-based anti-spoofing in face recognition from a generic webcamera. In: 2007 IEEE 11th International Conference on Computer Vision. Institute of Electrical & Electronics Engineers (IEEE) (2007).
[29]
Pistori, H.: Computer vision and digital inclusion of persons with special needs: overview and state of art. In: Computational Modelling of Objects Represented in Images (2018). https://www.taylorfrancis.com/books/e/9781351377133/chapters/10.1201%2F9781315106465-6
[30]
Royal College of Physicians: Medical rehabilitation in 2011 and beyond. Report of a working party, November 2010, Royal College of Physicians & British Society of Rehabilitation Medicine. RCP, London (2010)
[31]
Tuisku, O., Surakka, V., Vanhala, T., Rantanen, V., Lekkala, J.: Wireless face interface: using voluntary gaze direction and facial muscle activations for human-computer interaction. Interact. Comput. 24(1), 1–9 (2012). http://iwc.oxfordjournals.org/cgi/doi/10.1016/j.intcom.2011.10.002
[32]
Viola P and Jones MJ Robust real-time face detection Int. J. Comput. Vis. 2004 57 2 137-154
[33]
Viola P and Jones M Robust real-time object detection Int. J. Comput. Vis. 2001 57 137-154
[34]
Ward, D.J., Blackwell, A.F., MacKay, D.J.C.: Dasher–a data entry interface using continuous gestures and language models, pp. 129–137. ACM Press (2000). http://portal.acm.org/citation.cfm?doid=354401.354427
[35]
Wilson GF An analysis of mental workload in pilots during flight using multiple psychophysiological measures Int. J. Aviat. Psychol. 2002 12 1 3-18
[36]
World Health Organization: World report on disability. Technical report, WHO, Geneva (2011). http://whqlibdoc.who.int/publications/2011/9789240685215_eng.pdf
[37]
Yunqi, L., Meiling, Y., Xiaobing, S., Xiuxia, L., Jiangfan, O.: Recognition of eye states in real time video, pp. 554–559, January 2009. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4769528
[38]
Zhang, X., Kulkarni, H., Morris, M.R.: Smartphone-based gaze gesture communication for people with motor disabilities. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI 2017, pp. 2878–2889. ACM, New York (2017).
Recommendations
Multi-scenario gesture recognition using Kinect
CGAMES '12: Proceedings of the 2012 17th International Conference on Computer Games: AI, Animation, Mobile, Interactive Multimedia, Educational & Serious Games (CGAMES)Hand gesture recognition (HGR) is an important research topic because some situations require silent communication with sign languages. Computational HGR systems assist silent communication, and help people learn a sign language. In this article, a ...
Gliding and saccadic gaze gesture recognition in real time
Eye movements can be consciously controlled by humans to the extent of performing sequences of predefined movement patterns, or 'gaze gestures'. Gaze gestures can be tracked noninvasively employing a video-based eye tracking system. Gaze gestures hold ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Jan 2021
766 pages
ISBN:978-3-030-68789-2
DOI:10.1007/978-3-030-68790-8
© Springer Nature Switzerland AG 2021.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 10 January 2021
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Jan 2025