More Web Proxy on the site http://driver.im/

research-article

FaceInput: A Hand-Free and Secure Text Entry System through Facial Vibration

Authors:

Kaishun WuAuthors Info & Claims

2019 16th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON)

Pages 1 - 9

https://doi.org/10.1109/SAHCN.2019.8824990

Published: 10 June 2019 Publication History

Abstract

Wearable wristbands have become prevailing in the recent days because of their small and portable property. However, the limited size of the touch screen causes the problems of fat fingers and screen occlusion. Furthermore, it is not available for users whose hands are fully occupied with other tasks. To break this bottleneck, we propose a portable, hand-free and secure text-entry system, called FaceInput, which firstly uses a single small form factor sensor to accomplish a practical user input via facial vibrations. To sense the tiny facial vibration signals, we design and implement a double-stage amplifier whose maximum gain is 225. To enhance the input accuracy and robustness, we design a set of novel schemes for FaceInput based on the Mel-frequency cepstral coefficient (MFCC) concept and a hidden Markov model (HMM) to process the vibration signals, and an online calibration and adaptation scheme to recover the error due to temporal instability. Extensive experiments have been conducted on 30 human subjects during the period of one month. The results demonstrate that FaceInput can be successful to sense the tiny facial vibrations and robust to fight against various confounding factors. The average recognition accuracy is 98.2%. Furthermore, by enabling the runtime calibration and adaptation scheme that updates and enlarges the training data set, the accuracy can reach 100%.

References

[1]

K. Zhu, X. Ma, H. Chen, and M. Liang, (2017). Tripartite Effects: Exploring Users Mental Model of Mobile Gestures under the Influence of Operation, Handheld Posture, and Interaction Space. The International Journal of HumanComputer Interaction, Vol. 33, No. 6 (pp. 443-459).

[2]

R. Nandakumar, V. Iyer, D. Tan, and S. Gollakota, (2016). Fingerio: Using active sonar for fine-grained finger tracking. In Proc. ACM CHI (pp. 1515-1525).

[3]

W. Wang, A. X. Liu, and K. Sun, (2016, October). Device-free gesture tracking using acoustic signals. In Proc. ACM MobiCom (pp. 82-94).

[4]

P. C. Wong, K. Zhu, and H.b. Fu, (2018). FingerT9: Leveraging thumb-to-finger interaction for same-side-hand text entry on smartwatches. In Proc. ACM CHI (pp. 178).

[5]

W. Chen, M. Guan, L. Wang, R. Ruby, K. Wu.(2017, July). FLoc: Device free passive indoor localization in complex environments. In Proc. IEEE ICC (pp. 1-6).

[6]

L. A. Leiva, A. Sahami, A. Catala, N. Henze, and A. Schmidt, (2015). Text entry on tiny qwerty soft keyboards. In Proc. ACM CHI (pp. 669-678).

[7]

K. Sun, Y. Wang, C. Yu, Y. Yan, H. Wen, and Y. Shi, (2017). Float: One-Handed and Touch-Free Target Selection on Smartwatches. In Proc. ACM CHI (pp. 692-704).

[8]

W. Chen, M. Guan, Y. Huang, L. Wang, R. Ruby, W. Hu, and K. Wu, (2018, June). ViType: A Cost Efficient On-Body Typing System through Vibration. In Proc. IEEE SECON (pp. 1-9).

[9]

V. Lakshmipathy, C. Schmandt, and N. Marmasse, (2003, November). TalkBack: a conversational answering machine. In Proc. ACM UIST (pp. 41-50).

[10]

N. Carlini, P. Mishra, T. Vaidya, Y. Zhang, M. Sherr, C. Shields, D. Wagner, and W. Zhou, (2016). Hidden Voice Commands. In USENIX Security Symposium (pp. 513530).

[11]

C. Kasmi and J. L. Esteves (2015). IEMI Threats for Information Security: Remote Command Injection on Modern Smartphones. In IEEE Transactions on Electromagnetic Compatibility, Vol. 57, No. 6 (pp. 17521755).

[12]

R. Martin, (2016). Listen Up: Your AI Assistant Goes Crazy For NPR Too. http://www.npr.org/2016/03/06/469383361/listen-up-your-ai-assistant-goescrazy-for-npr-too.

[13]

B. Fasel, and J. Luettin, (2003). Automatic facial expression analysis: a survey. In Pattern recognition, Vol. 36, No. 1 (pp. 259-275).

[14]

J. S. Agustin, J. P. Hansen, D. W. Hansen, and H. Skovsgaard, (2009). Low-cost gaze pointing and EMG clicking. In Proc. ACM CHI (pp. 3247-3252).

[15]

D. J. C. Matthies, J. N. Antons, F. Heidmann, R. Wettach, and R. Schleicher, (2012). NeuroPad: use cases for a mobile physiological interface. In Proc. ACM NordiCHI (pp. 795-796).

[16]

A. Bulling, D. Roggen, and G. Trster, (2009). Wearable EOG goggles: eye-based interaction in everyday environments, In Proc. ACM CHI (pp. 3259-3264).

[17]

S. Ishimaru, K. Kunze, Y. Uema, K. Kise, M. Inami, and K. Tanaka, (2014). Smarter Eyewear: using commercial EOG glasses for activity recognition. In Proc. ACM Ubicomp (pp. 239-242).

[18]

V. Rantanen, P. H. Niemenlehto, J. Verho, and J. Lekkala, (2010). Capacitive facial movement detection for humancomputer interaction to click by frowning and lifting eyebrows. In Springer Medical and biological engineering and computing, Vol. 48, No. 1 (pp. 39-47).

[19]

V. Rantanen, H. Venesvirta, O. Spakov, J. Verho, A. Vetek, V. Surakka, and J. Lekkala, (2013). Capacitive measurement of facial activity intensity. In IEEE Sensors Journal, Vol. 13, No. 11 (pp. 4329-4338).

[20]

M. J. Fagan, S. R. Ell, J. M. Gilbert, E. Sarrazin, and P. M. Chapman, (2008). Development of a (silent) speech recognition system for patients following laryngectomy. In Medical engineering and physics, Vol. 30, No. 4 (pp. 419-425).

[21]

S. C. Jou, T. Schultz, and A. Waibel, (2004). Adaptation for soft whisper recognition using a throat microphone. In Proc. INTERSPEECH.

[22]

P. Heracleous, T. Kaino, H. Saruwatari, and K. Shikano, (2006). Unvoiced speech recognition using tissue-conductive acoustic sensor. EURASIP Journal on Advances in Signal Processing, Vol. 2007, No. 1 (pp. 094068).

[23]

C library for Broadcom BCM 2835 as used in Raspberry Pi https://www.airspayce.com/mikem/bcm2835/

[24]

Gartner Says Worldwide Wearable Device Sales to Grow 26 Percent in 2019 https://www.gartner.com/en/newsroom/press-releases/2018-11-29-gartner-says-worldwide-wearable-device-sales-to-grow

[25]

Raspberry Pi Analog to Digital Converters https://learn.adafruit.com/raspberry-pi-analog-to-digital-converters/mcp3008

[26]

L. Haskins, (1999). The Acoustic Theory of Speech Production: the source-filter model. http://www.haskins.yale.edu/featured/heads/mmsp/acoustic.html

[27]

F. Zheng, G. Zhang, and Z. Song, (2001). Comparison of different implementations of MFCC. Journal of Computer science and Technology, Vol. 16, No. 6 (pp.582-589).

Digital Library

[28]

H. Ocak, and K. A. Loparo, (2001). A new bearing fault detection and diagnosis scheme based on hidden Markov modeling of vibration signals. In Proc. IEEE ICASSP, Vol. 5 (pp. 3141-3144).

[29]

K. Grobel, and M. Assan, (1997, October). Isolated sign language recognition using hidden Markov models. In Proc. IEEE Computational Cybernetics and Simulation, Vol. 1 (pp. 162-167).

[30]

Sound Meter PRO https://play.google.com/store/apps/details?id=com.soundmeter.app

[31]

D. Cutting, J. Kupiec, J. Pedersen, and P. Sibun, (1992, March). A practical part-of-speech tagger. In Proc. Applied natural language processing (pp. 133-140).

[32]

G. D. Forney, (1973). The viterbi algorithm. In Proc. IEEE, Vol. 61, No. 3 (pp. 268-278).

Cited By

Chen WCheng JWang LZhao WMatusik W(2024)Sensor2Text: Enabling Natural Language Interactions for Daily Activity Tracking Using Wearable SensorsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997478:4(1-26)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699747
Chen WLin SPeng ZParizi FHeo SPatel SMatusik WZhao WStankovic J(2024)ViObjectProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435478:1(1-26)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643547
Chen WHu YSong WLiu YTorralba AMatusik W(2024)CAvatarProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314247:4(1-24)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631424
Show More Cited By

Index Terms

FaceInput: A Hand-Free and Secure Text Entry System through Facial Vibration
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens
2. Security and privacy
  1. Security services

Index terms have been assigned to the content through auto-classification.

Recommendations

Automatic face analysis system based on face recognition and facial physiognomy
ICHIT'06: Proceedings of the 1st international conference on Advances in hybrid information technology

An automatic face analysis system is proposed which uses face recognition and facial physiognomy. It first detects human's face, extracts its features, and classifies the shape of facial features. It will analyze the person's facial physiognomy and then ...
Facial Expression Mimicking System
ICPR '10: Proceedings of the 2010 20th International Conference on Pattern Recognition

We propose a facial expression mimicking system that copies the facial expression of one person on the image of another. The system uses the active appearance model (AAM), a commonly used model in the field of facial expression processing. AAM ...
Recognizing Action Units for Facial Expression Analysis

Most automatic expression analysis systems attempt to recognize a small set of prototypic expressions, such as happiness, anger, surprise, and fear. Such prototypic expressions, however, occur rather infrequently. Human emotions and intentions are more ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

2019 16th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON)

Jun 2019

651 pages

Copyright © 2019.

Publisher

IEEE Press

Publication History

Published: 10 June 2019

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen WCheng JWang LZhao WMatusik W(2024)Sensor2Text: Enabling Natural Language Interactions for Daily Activity Tracking Using Wearable SensorsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997478:4(1-26)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699747
Chen WLin SPeng ZParizi FHeo SPatel SMatusik WZhao WStankovic J(2024)ViObjectProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435478:1(1-26)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643547
Chen WHu YSong WLiu YTorralba AMatusik W(2024)CAvatarProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314247:4(1-24)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631424
Chen WWang ZQuan PPeng ZLin SSrivastava MMatusik WStankovic J(2023)Robust Finger Interactions with COTS Smartwatches via Unsupervised Siamese AdaptationProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606794(1-14)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606794

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents