More Web Proxy on the site http://driver.im/

research-article

FingerPing: Recognizing Fine-grained Hand Poses using Active Acoustic On-body Sensing

Authors:

Anandghan Waghmare,

Kenneth Cunefare,

Gregory D. AbowdAuthors Info & Claims

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Paper No.: 437, Pages 1 - 10

https://doi.org/10.1145/3173574.3174011

Published: 21 April 2018 Publication History

Abstract

FingerPing is a novel sensing technique that can recognize various fine-grained hand poses by analyzing acoustic resonance features. A surface-transducer mounted on a thumb ring injects acoustic chirps (20Hz to 6,000Hz) to the body. Four receivers distributed on the wrist and thumb collect the chirps. Different hand poses of the hand create distinct paths for the acoustic chirps to travel, creating unique frequency responses at the four receivers. We demonstrate how FingerPing can differentiate up to 22 hand poses, including the thumb touching each of the 12 phalanges on the hand as well as 10 American sign language poses. A user study with 16 participants showed that our system can recognize these two sets of poses with an accuracy of 93.77% and 95.64%, respectively. We discuss the opportunities and remaining challenges for the widespread use of this input technique.

Supplementary Material

suppl.mov (pn3647-file5.mp4)

Supplemental video

Download
14.49 MB

suppl.mov (pn3647.mp4)

Supplemental video

Download
52.33 MB

References

[1]

Brian Amento, Will Hill, and Loren Terveen. 2002. The Sound of One Hand: A Wrist-mounted Bio-acoustic Fingertip Gesture Interface. In CHI '02 Extended Abstracts on Human Factors in Computing Systems (CHI EA '02). ACM, New York, NY, USA, 724--725.

Digital Library

[2]

Liwei Chan, Yi-Ling Chen, Chi-Hao Hsieh, Rong-Hao Liang, and Bing-Yu Chen. 2015. CyclopsRing: Enabling Whole-Hand and Context-Aware Interactions Through a Fisheye Ring. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST '15). ACM, New York, NY, USA, 549--556.

Digital Library

[3]

Ke-Yu Chen, Kent Lyons, Sean White, and Shwetak Patel. 2013. uTrack: 3D Input Using Two Magnetic Sensors. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). ACM, New York, NY, USA, 237--244.

Digital Library

[4]

Gabe Cohn, Sidhant Gupta, Tien-Jui Lee, Dan Morris, Joshua R Smith, Matthew S Reynolds, Desney S Tan, and Shwetak N Patel. 2012. An ultra-low-power human body motion sensor using static electric field sensing. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing. ACM, 99--102.

Digital Library

[5]

Artem Dementyev and Joseph A. Paradiso. 2014. WristFlex: Low-power Gesture Input with Wrist-worn Pressure Sensors. In Proceedings of the 27th annual ACM symposium on User interface software and technology UIST '14. Association for Computing Machinery (ACM).

Digital Library

[6]

Mingming Fan and Khai N. Truong. 2015. SoQr: Sonically Quantifying the Content Level Inside Containers. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '15). ACM, New York, NY, USA, 3--14.

Digital Library

[7]

Theodoros Giannakopoulos and Aggelos Pikrakis. 2014. Introduction to Audio Analysis: A MATLAB Approach (1st ed.). Academic Press.

Digital Library

[8]

Jun Gong, Xing-Dong Yang, and Pourang Irani. 2016. WristWhirl: One-handed Continuous Smartwatch Input Using Wrist Gestures. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, New York, NY, USA, 861--872.

Digital Library

[9]

Chris Harrison, Desney Tan, and Dan Morris. 2010. Skinput: Appropriating the Body As an Input Surface. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 453--462.

Digital Library

[10]

Da-Yuan Huang, Liwei Chan, Shuo Yang, Fan Wang, Rong-Hao Liang, De-Nian Yang, Yi-Ping Hung, and Bing-Yu Chen. 2016. DigitSpace: Designing Thumb-to-Fingers Touch Interfaces for One-Handed and Eyes-Free Interactions. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 1526--1537.

Digital Library

[11]

Gierad Laput, Xiang 'Anthony' Chen, and Chris Harrison. 2016a. SweepSense: Ad Hoc Configuration Sensing Using Reflected Swept-Frequency Ultrasonics. In Proceedings of the 21st International Conference on Intelligent User Interfaces (IUI '16). ACM, New York, NY, USA, 332--335.

Digital Library

[12]

Gierad Laput, Robert Xiao, and Chris Harrison. 2016b. ViBand: High-Fidelity Bio-Acoustic Sensing Using Commodity Smartwatch Accelerometers. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, New York, NY, USA, 321--333.

Digital Library

[13]

Christian Loclair, Sean Gustafson, and Patrick Baudisch. 2010. PinchWatch: a wearable device for one-handed microinteractions. In Proc. MobileHCI, Vol. 10.

[14]

Adiyan Mujibiya, Xiang Cao, Desney S. Tan, Dan Morris, Shwetak N. Patel, and Jun Rekimoto. 2013. The Sound of Touch: On-body Touch and Gesture Sensing Based on Transdermal Ultrasound Propagation. In Proceedings of the 2013 ACM International Conference on Interactive Tabletops and Surfaces (ITS '13). ACM, New York, NY, USA, 189--198.

Digital Library

[15]

Rajalakshmi Nandakumar, Vikram Iyer, Desney Tan, and Shyamnath Gollakota. 2016. FingerIO: Using Active Sonar for Fine-Grained Finger Tracking. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 1515--1525.

Digital Library

[16]

Tauhidur Rahman, Alexander T. Adams, Mi Zhang, Erin Cherry, Bobby Zhou, Huaishu Peng, and Tanzeem Choudhury. 2014. BodyBeat: A Mobile System for Sensing Non-speech Body Sounds. In Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys '14). ACM, New York, NY, USA, 2--13.

Digital Library

[17]

T. Scott Saponas, Desney S. Tan, Dan Morris, Ravin Balakrishnan, Jim Turner, and James A. Landay. 2009. Enabling Always-available Input with Muscle-computer Interfaces. In Proceedings of the 22Nd Annual ACM Symposium on User Interface Software and Technology (UIST '09). ACM, New York, NY, USA, 167--176.

Digital Library

[18]

Ian H Witten, Eibe Frank, Mark A Hall, and Christopher J Pal. 2016. Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann.

Digital Library

[19]

Cheng Zhang, AbdelKareem Bedri, Gabriel Reyes, Bailey Bercik, Omer T. Inan, Thad E. Starner, and Gregory D. Abowd. 2016. TapSkin: Recognizing On-Skin Input for Smartwatches. In Proceedings of the 2016 ACM on Interactive Surfaces and Spaces (ISS '16). ACM, New York, NY, USA, 13--22.

Digital Library

[20]

Cheng Zhang, Sinan Hersek, Yiming Pu, Danrui Sun, Qiuyue Xue, Thad E. Starner, Gregory D. Abowd, and Omer T. Inan. 2017a. Bioacoustics-Based Human-Body-Mediated Communication. Computer 50, 2 (Feb. 2017), 36--46.

Digital Library

[21]

Cheng Zhang, Anandghan Waghmare, Pranav Kundra, Yiming Pu, Scott Gilliland, Thomas Ploetz, Thad E. Starner, Omer T. Inan, and Gregory D. Abowd. 2017b. FingerSound: Recognizing Unistroke Thumb Gestures Using a Ring. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 3, Article 120 (Sept. 2017), 19 pages.

Digital Library

[22]

Cheng Zhang, Xiaoxuan Wang, Anandghan Waghmare, Sumeet Jain, Thomas Ploetz, Omer T. Inan, Thad E. Starner, and Gregory D. Abowd. 2017c. FingOrbits: Interaction with Wearables Using Synchronized Thumb Movements. In Proceedings of the 2017 ACM International Symposium on Wearable Computers (ISWC '17). ACM, New York, NY, USA, 62--65.

Digital Library

[23]

Cheng Zhang, Junrui Yang, Caleb Southern, Thad E. Starner, and Gregory D. Abowd. 2016a. WatchOut: Extending Interactions on a Smartwatch with Inertial Sensing. In Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC '16). ACM, New York, NY, USA, 136--143.

Digital Library

[24]

Yang Zhang, Junhan Zhou, Gierad Laput, and Chris Harrison. 2016b. SkinTrack: Using the Body As an Electrical Waveguide for Continuous Finger Tracking on the Skin. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 1491--1503.

Digital Library

Cited By

Konishi TChoi MShizuki B(2024)Active Acoustic Sensing Based Authentication System Using a Door HandleProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701587(324-330)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1145/3701571.3701587
Mahmud SParikh VLiang QLi KZhang RAjit AGunda VAgarwal DGuimbretiere FZhang C(2024)ActSonic: Recognizing Everyday Activities from Inaudible Acoustic Wave Around the BodyProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997528:4(1-32)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699752
Yu THu GZhang RLim HMahmud SLee CLi KAgarwal DNie SOh JGuimbretière FZhang C(2024)Ring-a-Pose: A Ring for Continuous Hand Pose TrackingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997418:4(1-30)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699741
Show More Cited By

Index Terms

FingerPing: Recognizing Fine-grained Hand Poses using Active Acoustic On-body Sensing
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

EyeGesener: Eye Gesture Listener for Smart Glasses Interaction Using Acoustic Sensing

The smart glasses market has witnessed significant growth in recent years. The interaction of commercial smart glasses mostly relies on the hand, which is unsuitable for scenarios where both hands are occupied. In this paper, we propose EyeGesener, an ...
A Method to Recognize Eyeball Movement Gesture using Infrared Distance Sensor Array on Eyewear
iiWAS2021: The 23rd International Conference on Information Integration and Web Intelligence

Sensing technology for eyeball movement (i.e., gaze movement) has enabled various applications, e.g., hands-free input interfaces and provided information that helps us understand human beings in various fields. However, the existing method, which ...
Multi-scenario gesture recognition using Kinect
CGAMES '12: Proceedings of the 2012 17th International Conference on Computer Games: AI, Animation, Mobile, Interactive Multimedia, Educational & Serious Games (CGAMES)

Hand gesture recognition (HGR) is an important research topic because some situations require silent communication with sign languages. Computational HGR systems assist silent communication, and help people learn a sign language. In this article, a ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

April 2018

8489 pages

ISBN:9781450356206

DOI:10.1145/3173574

General Chairs:
Regan Mandryk
University of Saskatchewan, Canada
,
Mark Hancock
University of Waterloo, Canada
,
Program Chairs:
Mark Perry
Brunel University London, UK
,
Anna Cox
University College London, UK

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '18

Sponsor:

SIGCHI

CHI '18: CHI Conference on Human Factors in Computing Systems

April 21 - 26, 2018

Montreal QC, Canada

Acceptance Rates

CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

75
Total Citations
View Citations
1,946
Total Downloads

Downloads (Last 12 months)121
Downloads (Last 6 weeks)9

Reflects downloads up to 19 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Konishi TChoi MShizuki B(2024)Active Acoustic Sensing Based Authentication System Using a Door HandleProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701587(324-330)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1145/3701571.3701587
Mahmud SParikh VLiang QLi KZhang RAjit AGunda VAgarwal DGuimbretiere FZhang C(2024)ActSonic: Recognizing Everyday Activities from Inaudible Acoustic Wave Around the BodyProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997528:4(1-32)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699752
Yu THu GZhang RLim HMahmud SLee CLi KAgarwal DNie SOh JGuimbretière FZhang C(2024)Ring-a-Pose: A Ring for Continuous Hand Pose TrackingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997418:4(1-30)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699741
Chen WLin SPeng ZParizi FHeo SPatel SMatusik WZhao WStankovic J(2024)ViObjectProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435478:1(1-26)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643547
Wang XZhang Y(2024)TextureSightProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314137:4(1-27)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631413
Lee CZhang RAgarwal DYu TGunda VLopez OKim JYin SDong BLi KSakashita MGuimbretiere FZhang C(2024)EchoWrist: Continuous Hand Pose Tracking and Hand-Object Interaction Recognition Using Low-Power Active Acoustic Sensing On a WristbandProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642910(1-21)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642910
Li KZhang RChen SChen BSakashita MGuimbretiere FZhang C(2024)EyeEcho: Continuous and Low-power Facial Expression Tracking on GlassesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642613(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642613
Shin HHuh JKwon BKim ICheon EKim HLee COakley I(2024)SkullID: Through-Skull Sound Conduction based Authentication for SmartglassesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642506(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642506
Yang YChen THuang YGuo XShangguan L(2024)MAF: Exploring Mobile Acoustic Field for Hand-to-Face Gesture InteractionsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642437(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642437
Vurgun YKiran M(2024)Recognizing salat activity using deep learning models via smartwatch sensorsNeural Computing and Applications10.1007/s00521-024-10195-136:31(19449-19467)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s00521-024-10195-1
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents