More Web Proxy on the site http://driver.im/

research-article

TapSense: enhancing finger interaction on touch surfaces

Authors:

Chris Harrison,

Scott E. HudsonAuthors Info & Claims

UIST '11: Proceedings of the 24th annual ACM symposium on User interface software and technology

Pages 627 - 636

https://doi.org/10.1145/2047196.2047279

Published: 16 October 2011 Publication History

Abstract

We present TapSense, an enhancement to touch interaction that allows conventional surfaces to identify the type of object being used for input. This is achieved by segmenting and classifying sounds resulting from an object's impact. For example, the diverse anatomy of a human finger allows different parts to be recognized including the tip, pad, nail and knuckle - without having to instrument the user. This opens several new and powerful interaction opportunities for touch input, especially in mobile devices, where input is extremely constrained. Our system can also identify different sets of passive tools. We conclude with a comprehensive investigation of classification accuracy and training implications. Results show our proof-of-concept system can support sets with four input types at around 95% accuracy. Small, but useful input sets of two (e.g., pen and finger discrimination) can operate in excess of 99% accuracy.

Supplementary Material

JPG File (fp122.jpg)

Download
12.45 KB

MOV File (fp122.mov)

Download
26.29 MB

References

[1]

Asano, F., Goto, M., Itou, K. and Asoh, H. Real-time sound source localization and separation system and its application to automatic speech recognition. In Proc. Eurospeech '01. 1013--1016.

[2]

Benko, H., Wilson, A., and R. Balakrishnan, Sphere: multitouch interactions on a spherical display. In Proc. UIST '08. 77--86.

Digital Library

[3]

Burges, C.J. A Tutorial on Support Vector Machines for Pattern Recognition. Data Mining and Know. Disc., 2.2. June 1998, 121--167.

Digital Library

[4]

Cao, X., Wilson, A., Balakrishnan, R., Hinckley, K., and Hudson, S.E. ShapeTouch: Leveraging contact shape on interactive surfaces. In Proc. TABLETOP '08. 129--136.

[5]

Deyle, T., Palinko, S., Poole, E.S., and Starner, T. Hambone. A Bio-Acoustic Gesture Interface. In Proc. ISW '07. 1--8.

Digital Library

[6]

Dietz, P.H., Harsham, B., Forlines, C., Leigh, D., Yerazunis, W., Shipman, S., Schmidt-Nielsen, B., and Ryall, K. DT controls: adding identity to physical interfaces. In Proc. UIST '05. 245--252.

Digital Library

[7]

Dietz, P. and Leigh, D. DiamondTouch: a multi-user touch technology. In Proc. UIST '01. 219--226.

Digital Library

[8]

Gutwin, C., Greenberg, S., Blum, R. and Dyck, J. Supporting Informal Collaboration in Shared Workspace Groupware. HCI Report 2005-01, U. Saskatchewan, Canada. 2005.

[9]

Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., and Witten, I.H. The WEKA Data Mining Software. An Update. SIGKDD Explor., 11,1. 2009.

Digital Library

[10]

Harrison, C., Tan, D., and Morris, D. Skinput: appropriating the bod.

[11]

Harrison, C. and Hudson, S.E. Scratch Input: Creating Large, Inexpensive, Unpowered and Mobile finger Input Surfaces. In Proc. UIST '08. 205--208.

Digital Library

[12]

Hartmann, B., Morris, M.R., Benko, H., and Wilson, A.D. Augmenting interactive tables with mice & keyboards. In Proc. UIST '09. 149--152.

Digital Library

[13]

Hinckley K. and Song, H. Sensor synaesthesia: touch in motion, and motion in touch. In Proc. CHI '11. 801--810.

Digital Library

[14]

Hinckley, K., Yatani, K., Pahud, M., Coddington, N. Rodenhouse, J., Wilson, A., Benko, H., and Buxton, B. Pe. + touch = new tools. In Proc. UIST '10. 27--36.

Digital Library

[15]

Hinckley, K., Yatani, K., Pahud, M., Coddington, N. Rodenhouse, J., Wilson, A., Benko, H., and Buxton, B. Manual deskterity: an exploration of simultaneous pen-touch direct input. In CHI '10 Ext. Abst. 2793--2802.

Digital Library

[16]

Holz, C. and Baudisch, P. The generalized perceived input point model and how to double touch accuracy by extracting fingerprints. In Proc. CHI '10. 581--590.

Digital Library

[17]

Kaltenbrunner, M. and Bencina, R. ReacTIVision: a computer-vision framework for table-based tangible interaction. In Proc. TEI '07. 69--74.

Digital Library

[18]

Matsushita, N. and Rekimoto, J. HoloWall: designing finger, hand, body, and object sensitive wall. In Proc. UIST '97. 209--210.

Digital Library

[19]

Mimio: http://www.mimio.com.

[20]

Olwal, A. and Wilson, A.D. SurfaceFusion: unobtrusive tracking of everyday objects in tangible user interfaces. In Proc. GI '08. 235--242.

Digital Library

[21]

Paradiso, J.A. and Leo, C.K. Tracking and Characterizing Knocks Atop Large Interactive Displays. Sensor Review. 25(2), 2005, 134--143.

[22]

Paradiso, J., Hsiao, K., Strickon, J., Lifton, J., and Adler, A. Sensor Systems for Interactive Surfaces. IBM Sys. Journal. 39(3/4), Oct '00, 892--914.

Digital Library

[23]

Patten, J., Ishii, H., Hines, J., and Pangaro, G. Sensetable: A wireless object tracking platform for tangible user interfaces. In Proc. CHI '01. 253--260.

Digital Library

[24]

Rekimoto, J. and Saitoh, M. Augmented surfaces: a spatially continuous work space for hybrid computing environments. In Proc. CHI '99. 378--385.

Digital Library

[25]

Rekimoto, J. and Sciammarella, E. ToolStone: effective use of the physical manipulation vocabularies of input devices. In Proc. UIST '00. 109--117.

Digital Library

[26]

Rekimoto, J. SmartSkin: an infrastructure for freehand manipulation on interactive surfaces. In Proc. CHI '02. 113--120.

Digital Library

[27]

Vandoren, P., Van Laerhoven, T., Claesen, L., Taelman, J. Di Fiore, F., Van Reeth, F., and Flerackers, E. Dip - its digital infrared painting on an interactive table. In CHI '07 Ext. Abst. 2901--2906.

Digital Library

[28]

Wang, F. and Ren, X. Empirical evaluation for finger input properties in multi-touch interaction. In Proc. CHI '09. 1063--1072.

Digital Library

Cited By

Zhang PYing WRiggs SHeo S(2024)MoiréTag: A Low-Cost Tag for High-Precision Tangible Interactions without Active ComponentsProceedings of the ACM on Human-Computer Interaction10.1145/36981138:ISS(1-19)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3698113
Li CYu JHe KFeng JZhou J(2024)SwivelTouch: Boosting Touchscreen Input with 3D Finger Rotation GestureProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36595848:2(1-30)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659584
Palma GPourjafarian NSteimle JCignoni P(2024)Capacitive Touch Sensing on General 3D SurfacesACM Transactions on Graphics10.1145/365818543:4(1-20)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658185
Show More Cited By

Index Terms

TapSense: enhancing finger interaction on touch surfaces
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

The performance of touch screen soft buttons
CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

The introduction of a new generation of attractive touch screen-based devices raises many basic usability questions whose answers may influence future design and market direction. With a set of current mobile devices, we conducted three experiments ...
Effect of elastic touchscreen and input devices with different softness on user task performance and subjective satisfaction

This study explored the relationship between the softness of two output displays (glass and elastic surfaces) and three direct input devices (fingertip and two types of styli, with acrylic and rubber tips) in terms of user task performance and ...
Imprint-Based Input Techniques for Touch-Based Mobile Devices
MUM '20: Proceedings of the 19th International Conference on Mobile and Ubiquitous Multimedia

Touchscreens translate touches of all kinds into 2D coordinates. This limits the input vocabulary and constrains effective interaction to touches by the fingertip. Previous tabletop research extended the input vocabulary with a myriad of promising ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '11: Proceedings of the 24th annual ACM symposium on User interface software and technology

October 2011

654 pages

ISBN:9781450307161

DOI:10.1145/2047196

General Chair:
Jeff Pierce
IBM Research, USA
,
Program Chairs:
Maneesh Agrawala
University of California, Berkeley, USA
,
Scott Klemmer
Stanford University, USA

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 October 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UIST '11

Sponsor:

UIST '11: The 24th Annual ACM Symposium on User Interface Software and Technology

October 16 - 19, 2011

California, Santa Barbara, USA

Acceptance Rates

UIST '11 Paper Acceptance Rate 67 of 262 submissions, 26%;

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

184
Total Citations
View Citations
3,094
Total Downloads

Downloads (Last 12 months)184
Downloads (Last 6 weeks)22

Reflects downloads up to 17 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang PYing WRiggs SHeo S(2024)MoiréTag: A Low-Cost Tag for High-Precision Tangible Interactions without Active ComponentsProceedings of the ACM on Human-Computer Interaction10.1145/36981138:ISS(1-19)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3698113
Li CYu JHe KFeng JZhou J(2024)SwivelTouch: Boosting Touchscreen Input with 3D Finger Rotation GestureProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36595848:2(1-30)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659584
Palma GPourjafarian NSteimle JCignoni P(2024)Capacitive Touch Sensing on General 3D SurfacesACM Transactions on Graphics10.1145/365818543:4(1-20)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658185
Wattanaparinton RKitada KTakemura K(2024)RFTIRTouch: Touch Sensing Device for Dual-sided Transparent Plane Based on Repropagated Frustrated Total Internal ReflectionProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676428(1-10)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676428
Lertvittayakumjorn PCai SDou BHo CZhai S(2024)Can Capacitive Touch Images Enhance Mobile Keyboard Decoding?Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676420(1-17)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676420
He KLi CDuan YFeng JZhou J(2024)TrackPoseProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314597:4(1-22)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631459
Yang YChen THuang YGuo XShangguan L(2024)MAF: Exploring Mobile Acoustic Field for Hand-to-Face Gesture InteractionsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642437(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642437
Coutrix CProst C(2024)Impact of Fingernails Length on Mobile Tactile InteractionProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642037(1-21)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642037
Jeong JKim DJeong J(2024)EchoTap: Non-Verbal Sound Interaction with Knock and Tap GesturesInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2348837(1-22)Online publication date: 3-Jun-2024
https://doi.org/10.1080/10447318.2024.2348837
Shi YYi XLiang CQin YWang YYan YCheng ZZhu PZhang SLi YLiu YWang XChen JZhou WWang YZhao DDu F(2024)HCI Research and Innovation in China: A 10-Year PerspectiveInternational Journal of Human–Computer Interaction10.1080/10447318.2024.232385840:8(1799-1831)Online publication date: 22-Mar-2024
https://doi.org/10.1080/10447318.2024.2323858
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents