More Web Proxy on the site http://driver.im/

research-article

A Survey on Multi-touch Gesture Recognition and Multi-touch Frameworks

Authors:

Mauricio Cirelli,

Ricardo NakamuraAuthors Info & Claims

ITS '14: Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces

Pages 35 - 44

https://doi.org/10.1145/2669485.2669509

Published: 16 November 2014 Publication History

Abstract

The multi-touch gesture recognition problem has drawn great attention from the human-computer interaction (HCI) community, mainly since multi-touch surfaces and other touch-capable devices reached the mainstream market. In the past decade, several multi-touch gesture recognition techniques and multi-touch frameworks were proposed. When we started our research on touch-based gestures recognition, we identified some surveys focused on computer vision or accelerometers. However, in several multi-touch surface devices, the multi-touch sensor is the only input method available. We here present a survey on touch-based gestures recognition techniques and frameworks, and propose an extended set of requirements such techniques and frameworks should meet in order to provide better support to multi-touch surface applications.

References

[1]

Anthony, L., and Wobbrock, J. O. A lightweight multistroke recognizer for user interface prototypes. In Proceedings of Graphics Interface 2010, GI '10, Canadian Information Processing Society (Toronto, Ont., Canada, Canada, 2010), 245--252.

Digital Library

[2]

Anthony, L., and Wobbrock, J. O. $n-protractor: a fast and accurate multistroke recognizer. In Proceedings of Graphics Interface 2012, GI '12, Canadian Information Processing Society (Toronto, Ont., Canada, Canada, 2012), 117--120.

Digital Library

[3]

Chen, F.-S., Fu, C.-M., and Huang, C.-L. Hand gesture recognition using a real-time tracking method and hidden markov models. Image and Vision Computing 21, 8 (2003), 745--758.

[4]

Chen, L., Wang, F., Deng, H., and Ji, K. A survey on hand gesture recognition. In Proceedings of the 2013 International Conference on Computer Sciences and Applications, CSA '13, IEEE Computer Society (Washington, DC, USA, 2013), 313--316.

Digital Library

[5]

Clayphan, A., Collins, A., Ackad, C., Kummerfeld, B., and Kay, J. Firestorm: a brainstorming application for collaborative group work at tabletops. ITS '11, ACM (New York, NY, USA, 2011), 162--171.

Digital Library

[6]

Codeplex. MIRIA SDK, 2011.

[7]

Dietz, P., and Leigh, D. DiamondTouch: a multi-user touch technology. In Proceedings of the 14th annual ACM symposium on User interface software and technology, UIST '01, ACM (New York, NY, USA, 2001), 219--226.

Digital Library

[8]

Echtler, F., and Butz, A. GISpL: gestures made easy. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction, TEI '12, ACM (New York, NY, USA, 2012), 233--240.

Digital Library

[9]

Echtler, F., and Klinker, G. A multitouch software architecture. In Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges, NordiCHI '08, ACM (New York, NY, USA, 2008), 463--466.

Digital Library

[10]

Echtler, F., Klinker, G., and Butz, A. Towards a unified gesture description language. In Proceedings of the 13th International Conference on Humans and Computers, HC '10, University of Aizu Press (Fukushima-ken, Japan, Japan, 2010), 177--182.

Digital Library

[11]

Govoni, D. Java Application Frameworks, 1 ed. Wiley, June 1999.

Digital Library

[12]

Hinrichs, U., and Carpendale, S. Gestures in the wild: studying multi-touch gesture sequences on interactive tabletop exhibits. In Proceedings of the 2011 annual conference on Human factors in computing systems, CHI '11, ACM (New York, NY, USA, 2011), 3023--3032.

Digital Library

[13]

Hoste, L. Software engineering abstractions for the multi-touch revolution. In 2010 ACM/IEEE 32nd International Conference on Software Engineering, vol. 2, IEEE (May 2010), 509--510.

Digital Library

[14]

Hoste, L., De Rooms, B., and Signer, B. Declarative gesture spotting using inferred and refined control points. In Proceedings of the 2nd International Conference on Pattern Recognition Applications and Methods (ICPRAM 2013) (Barcelona, Spain, 2013), 1--6.

[15]

Hoste, L., Dumas, B., and Signer, B. Mudra: A unified multimodal interaction framework. In Proceedings of the 13th International Conference on Multimodal Interfaces, ICMI '11, ACM (New York, NY, USA, 2011), 97--104.

Digital Library

[16]

Institute, F. MT4j - multitouch for java, 2011.

[17]

Jord, S., Geiger, G., Alonso, M., and Kaltenbrunner, M. The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces. In Proceedings of the 1st international conference on Tangible and embedded interaction, TEI '07, ACM (New York, NY, USA, 2007), 139--146.

Digital Library

[18]

Kaltenbrunner, M. reacTIVision and TUIO: a tangible tabletop toolkit. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, ITS '09, ACM (New York, NY, USA, 2009), 9--16.

Digital Library

[19]

Kaltenbrunner, M., Bovermann, T., Bencina, R., and Costanza, E. TUIO: A protocol for table-top tangible user interfaces. In 6th International Gesture Workshop (2005).

[20]

Kammer, D., Henkens, D., and Groh, R. GeForMTjs: A javascript library based on a domain specific language for multi-touch gestures. In Proceedings of the 12th International Conference on Web Engineering, ICWE'12, Springer-Verlag (Berlin, Heidelberg, 2012), 444--447.

Digital Library

[21]

Kammer, D., Keck, M., Freitag, G., and Wacker, M. Taxonomy and overview of multi-touch frameworks: Architecture, scope and features. In Workshop on Engineering Patterns for Multitouch Interfaces (2010).

[22]

Kammer, D., Wojdziak, J., Keck, M., Groh, R., and Taranko, S. Towards a formalization of multi-touch gestures. ITS '10, ACM (New York, NY, USA, 2010), 49--58.

Digital Library

[23]

Khandkar, S. H., and Maurer, F. A domain specific language to define gestures for multi-touch applications. In Proceedings of the 10th Workshop on Domain-Specific Modeling, DSM '10, ACM (New York, NY, USA, 2010), 2:1--2:6.

Digital Library

[24]

Kin, K., Hartmann, B., DeRose, T., and Agrawala, M. Proton++: a customizable declarative multitouch framework. In Proceedings of the 25th annual ACM symposium on User interface software and technology, UIST '12, ACM (New York, NY, USA, 2012), 477--486.

Digital Library

[25]

Kin, K., Hartmann, B., DeRose, T., and Agrawala, M. Proton: multitouch gestures as regular expressions. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems, CHI '12, ACM (New York, NY, USA, 2012), 2885--2894.

Digital Library

[26]

Kruger, R., Carpendale, S., Scott, S. D., and Greenberg, S. How people use orientation on tables: comprehension, coordination and communication. In Proceedings of the 2003 international ACM SIGGROUP conference on Supporting group work, GROUP '03, ACM (New York, NY, USA, 2003), 369--378.

Digital Library

[27]

L, H., and Li, Y. Gesture coder: a tool for programming multi-touch gestures by demonstration. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems, CHI '12, ACM (New York, NY, USA, 2012), 2875--2884.

Digital Library

[28]

LaViola, Jr., J. J. A survey of hand posture and gesture recognition techniques and technology. Tech. rep., Brown University, Providence, RI, USA, 1999.

Digital Library

[29]

Li, Y. Protractor: a fast and accurate gesture recognizer. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '10, ACM (New York, NY, USA, 2010), 2169--2172.

Digital Library

[30]

Mindstorm. Breezemultitouch, 2010.

[31]

Mitra, S., and Acharya, T. Gesture recognition: A survey. Trans. Sys. Man Cyber Part C 37, 3 (May 2007), 311--324.

Digital Library

[32]

Morris, M. R., Huang, A., Paepcke, A., and Winograd, T. Cooperative gestures: multi-user gestural interactions for co-located groupware. In Proceedings of the SIGCHI conference on Human Factors in computing systems, CHI '06, ACM (New York, NY, USA, 2006), 1201--1210.

Digital Library

[33]

NUIGroup, T. Gesture definition markup language (GDML), 2009.

[34]

Nygard, E. S., and Thomassen, A. Multi-touch Interaction with Gesture Recognition. PhD thesis, 2010.

[35]

Renaux, T., Hoste, L., Marr, S., and De Meuter, W. Parallel gesture recognition with soft real-time guarantees. In Proceedings of the 2Nd Edition on Programming Systems, Languages and Applications Based on Actors, Agents, and Decentralized Control Abstractions, AGERE! '12, ACM (New York, NY, USA, 2012), 35--46.

Digital Library

[36]

Rhyne, J. R., and Wolf, C. G. Gestural interfaces for information processing applications. Tech. Rep. RC12179, IBM Corporation, Yorktown Heights, NY, 1986.

[37]

Rubine, D. Specifying gestures by example. SIGGRAPH Comput. Graph. 25, 4 (July 1991), 329--337.

Digital Library

[38]

Scholliers, C., Hoste, L., Signer, B., and De Meuter, W. Midas: a declarative multi-touch interaction framework. In Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction, TEI '11, ACM (New York, NY, USA, 2011), 49--56.

Digital Library

[39]

Scott, S. D., Sheelagh, M., Carpendale, T., and Inkpen, K. M. Territoriality in collaborative tabletop workspaces. In Proceedings of the 2004 ACM conference on Computer supported cooperative work, CSCW '04, ACM (New York, NY, USA, 2004), 294--303.

Digital Library

[40]

Shen, C., Vernier, F. D., Forlines, C., and Ringel, M. DiamondSpin: an extensible toolkit for around-the-table interaction. In Proceedings of the SIGCHI conference on Human factors in computing systems, CHI '04, ACM (New York, NY, USA, 2004), 167--174.

Digital Library

[41]

Spano, L. D., Cisternino, A., and Patern, F. A compositional model for gesture definition. In Proceedings of the 4th International Conference on Human-Centered Software Engineering, HCSE'12, Springer-Verlag (Berlin, Heidelberg, 2012), 34--52.

Digital Library

[42]

Strothoff, S., Valkov, D., and Hinrichs, K. Triangle cursor: interactions with objects above the tabletop. ITS '11, ACM (New York, NY, USA, 2011), 111--119.

Digital Library

[43]

Vatavu, R.-D., Anthony, L., and Wobbrock, J. O. Gestures as point clouds: a $p recognizer for user interface prototypes. In Proceedings of the 14th ACM international conference on Multimodal interaction, ICMI '12, ACM (New York, NY, USA, 2012), 273--280.

Digital Library

[44]

Wobbrock, J. O., Morris, M. R., and Wilson, A. D. User-defined gestures for surface computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '09, ACM (New York, NY, USA, 2009), 1083--1092.

Digital Library

[45]

Wobbrock, J. O., Wilson, A. D., and Li, Y. Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In Proceedings of the 20th annual ACM symposium on User interface software and technology, UIST '07, ACM (New York, NY, USA, 2007), 159--168.

Digital Library

[46]

Yoon, H.-S., Soh, J., Bae, Y. J., and Seung Yang, H. Hand gesture recognition using combined features of location, angle and velocity. Pattern Recognition 34, 7 (2001), 1491--1501.

Cited By

Vanderdonckt JVatavu RSluÿters A(2024)Engineering Touchscreen Input for 3-Way Displays: Taxonomy, Datasets, and ClassificationCompanion Proceedings of the 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems10.1145/3660515.3661331(57-65)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3660515.3661331
Andrei ABilius LVatavu R(2024)Take a Seat, Make a Gesture: Charting User Preferences for On-Chair and From-Chair Gesture InputProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642028(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642028
Terenti MPamparău CVatavu R(2024)The user experience of distal arm-level vibrotactile feedback for interactions with virtual versus physical displaysVirtual Reality10.1007/s10055-024-00977-228:2Online publication date: 22-Mar-2024
https://dl.acm.org/doi/10.1007/s10055-024-00977-2
Show More Cited By

Index Terms

A Survey on Multi-touch Gesture Recognition and Multi-touch Frameworks
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens
    2. Interaction paradigms

Recommendations

The MiTable Multi-Touch Gestures Engine
ITS '15: Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces

From smartphones to large multi-touch surfaces, different devices and applications need a technique to match the input from each user to a known action. The multi-touch gestures recognition problem has been studied for a long time and has obtained its ...
Single capacitive touch sensor that detects multi-touch gestures
ISWC '14: Proceedings of the 2014 ACM International Symposium on Wearable Computers

A technique that allows a single capacitive touch sensor to recognize multi-touch gestures is proposed. Touch, multi-finger swipe and swipe direction are recognized. It does not need a multiplexer or complicated wiring and well suits wearable devices. ...
Direct and indirect multi-touch interaction on a wall display
IHM '14: Proceedings of the 26th Conference on l'Interaction Homme-Machine

Multi-touch wall displays allow to take advantage of co-located interaction (direct interaction) on very large surfaces. However interacting with content beyond arms' reach requires body movements, introducing fatigue and impacting performance. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ITS '14: Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces

November 2014

524 pages

ISBN:9781450325875

DOI:10.1145/2669485

General Chairs:
Raimund Dachselt
Technische Universität Dresden, Germany
,
Nicholas Graham
Queen's University, Canada
,
Program Chairs:
Kasper Hornbæk
University of Copenhagen, Denmark
,
Miguel Nacenta
University of St. Andrews, UK

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 November 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ITS '14

Sponsor:

SIGCHI

ITS '14: Interactive Tabletops and Surfaces

November 16 - 19, 2014

Dresden, Germany

Acceptance Rates

ITS '14 Paper Acceptance Rate 31 of 112 submissions, 28%;

Overall Acceptance Rate 119 of 418 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
810
Total Downloads

Downloads (Last 12 months)55
Downloads (Last 6 weeks)8

Reflects downloads up to 10 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Vanderdonckt JVatavu RSluÿters A(2024)Engineering Touchscreen Input for 3-Way Displays: Taxonomy, Datasets, and ClassificationCompanion Proceedings of the 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems10.1145/3660515.3661331(57-65)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3660515.3661331
Andrei ABilius LVatavu R(2024)Take a Seat, Make a Gesture: Charting User Preferences for On-Chair and From-Chair Gesture InputProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642028(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642028
Terenti MPamparău CVatavu R(2024)The user experience of distal arm-level vibrotactile feedback for interactions with virtual versus physical displaysVirtual Reality10.1007/s10055-024-00977-228:2Online publication date: 22-Mar-2024
https://dl.acm.org/doi/10.1007/s10055-024-00977-2
Stefanidi EBentvelzen MWoźniak PKosch TWoźniak MMildner TSchneegass SMüller HNiess J(2023)Literature Reviews in HCI: A Review of ReviewsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581332(1-24)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581332
Şiean APamparău CSluÿters AVatavu RVanderdonckt J(2023)Flexible gesture input with radars: systematic literature review and taxonomy of radar sensing integration in ambient intelligence environmentsJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-023-04606-914:6(7967-7981)Online publication date: 10-Apr-2023
https://doi.org/10.1007/s12652-023-04606-9
Prange ASonntag D(2022)Modeling Users' Cognitive Performance Using Digital Pen FeaturesFrontiers in Artificial Intelligence10.3389/frai.2022.7871795Online publication date: 3-May-2022
https://doi.org/10.3389/frai.2022.787179
Hall BBartram LBrehmer M(2022)Augmented Chironomia for Presenting Data to Remote AudiencesProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545614(1-14)Online publication date: 29-Oct-2022
https://dl.acm.org/doi/10.1145/3526113.3545614
Prange ASonntag D(2021)Assessing Cognitive Test Performance Using Automatic Digital Pen Features AnalysisProceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization10.1145/3450613.3456812(33-43)Online publication date: 21-Jun-2021
https://dl.acm.org/doi/10.1145/3450613.3456812
Castells-Rufas DBorrego-Carazo JCarrabina JNaqui JBiempica E(2020)Continuous touch gesture recognition based on RNNs for capacitive proximity sensorsPersonal and Ubiquitous Computing10.1007/s00779-020-01472-626:6(1355-1372)Online publication date: 9-Nov-2020
https://doi.org/10.1007/s00779-020-01472-6
Hesenius MGruhn V(2019)GestureCardsProceedings of the ACM on Human-Computer Interaction10.1145/33311643:EICS(1-35)Online publication date: 13-Jun-2019
https://dl.acm.org/doi/10.1145/3331164
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents