More Web Proxy on the site http://driver.im/

research-article

Public Access

$Q: a super-quick, articulation-invariant stroke-gesture recognizer for low-resource devices

Authors:

Radu-Daniel Vatavu,

Jacob O. WobbrockAuthors Info & Claims

MobileHCI '18: Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services

Article No.: 23, Pages 1 - 12

https://doi.org/10.1145/3229434.3229465

Published: 03 September 2018 Publication History

Abstract

We introduce $Q, a super-quick, articulation-invariant point-cloud stroke-gesture recognizer for mobile, wearable, and embedded devices with low computing resources. $Q ran up to 142X faster than its predecessor $P in our benchmark evaluations on several mobile CPUs, and executed in less than 3% of $P's computations without any accuracy loss. In our most extreme evaluation demanding over 99% user-independent recognition accuracy, $P required 9.4s to run a single classification, while $Q completed in just 191ms (a 49X speed-up) on a Cortex-A7, one of the most widespread CPUs on the mobile market. $Q was even faster on a low-end 600-MHz processor, on which it executed in only 0.7% of $P's computations (a 142X speed-up), reducing classification time from two minutes to less than one second. $Q is the next major step for the "$-family" of gesture recognizers: articulation-invariant, extremely fast, accurate, and implementable on top of $P with just 30 extra lines of code.

References

[1]

L. Anthony, Q. Brown, J. Nias, and B. Tate. 2015. Children (and Adults) Benefit from Visual Feedback During Gesture Interaction on Mobile Touchscreen Devices. Int. J. Child-Comp. Interact. 6 (2015), 17--27.

Digital Library

[2]

Lisa Anthony and Jacob O. Wobbrock. 2010. A Lightweight Multistroke Recognizer for User Interface Prototypes. In Proc. of GI '10. 245--252. http://dl.acm.org/citation.cfm?id=1839214.1839258

Digital Library

[3]

Lisa Anthony and Jacob O. Wobbrock. 2012. $N-protractor: A Fast and Accurate Multistroke Recognizer. In Proc. of GI '12. 117--120. http://dl.acm.org/citation.cfm?id=2305276.2305296

Digital Library

[4]

ARM. 2016. Humanizing the Digital World: The world's leading semiconductor intellectual property (IP) supplier. (2016). http://www.arm.com/about/company-profile/

[5]

ARM. 2017a. ARM1136 processor. (2017). https://www.arm.com/products/processors/classic/arm11/arm1136.php

[6]

ARM. 2017b. Cortex-A5 processor. (2017). http://www.arm.com/products/processors/cortex-a/cortex-a5.php

[7]

ARM. 2017c. Cortex-A53 processor. (2017). http://www.arm.com/products/processors/cortex-a/cortex-a53-processor.php

[8]

ARM. 2017d. Cortex-A7 processor. (2017). https://www.arm.com/products/processors/cortex-a/cortex-a7.php

[9]

ARM. 2017e. Cortex-A9 processor. (2017). http://www.arm.com/products/processors/cortex-a/cortex-a9.php

[10]

R. Balcazar, F.R. Ortega, K. Tarre, A. Barreto, M. Weiss, and N.D. Rishe. 2017. CircGR: Interactive Multi-Touch Gesture Recognition Using Circular Measurements. In Proc. of ISS '17. ACM, New York, NY, USA, 12--21.

Digital Library

[11]

Rainer Burkard, Mauro Dell'Amico, and Silvano Martello. 2009. Assignment Problems. Society for Industrial and Applied Mathematics, Philadelphia, PA.

Digital Library

[12]

G. Casiez, N. Roussel, and D. Vogel. 2012. 1€Filter: A Simple Speed-based Low-pass Filter for Noisy Input in Interactive Systems. In Proc. of CHI '12. 2527--2530.

Digital Library

[13]

D.C. Cireşan, U. Meier, L.M. Gambardella, and J. Schmidhuber. 2010. Deep, Big, Simple Neural Nets for Handwritten Digit Recognition. Neural Computation 22, 12 (Dec. 2010), 3207--3220.

Digital Library

[14]

Tim Cross. 2016. After Moore's Law. The Economist. (March 2016). http://www.economist.com/technology-quarterly/2016-03-12/after-moores-law

[15]

Jack Edmonds. 1965. Paths, trees, and flowers. Canadian Journal of Mathematics 17 (1965), 449--467.

[16]

T.A. Hammond and R. Davis. 2009. Recognizing Interspersed Sketches Quickly. In GI '09. 157--166. http://dl.acm.org/citation.cfm?id=1555880.1555917

Digital Library

[17]

Tracy Hammond and Brandon Paulson. 2011. Recognizing Sketched Multistroke Primitives. ACM TiiS 1, 1, Article 4 (Oct. 2011), 34 pages.

Digital Library

[18]

Chris Harrison, Desney Tan, and Dan Morris. 2011. Skinput: Appropriating the Skin As an Interactive Canvas. Commun. ACM 54, 8 (Aug. 2011), 111--118.

Digital Library

[19]

J. Herold and T. F. Stahovich. 2012. The 1¢Recognizer: A Fast, Accurate, and Easy-to-implement Handwritten Gesture Recognition Technique. In Proc. of SBIM '12. http://dl.acm.org/citation.cfm?id=2331067.2331074

Digital Library

[20]

L.B. Kara and T.F. Stahovich. 2004. Hierarchical Parsing and Recognition of Hand-sketched Diagrams. In Proc. of UIST '04. ACM, 13--22.

Digital Library

[21]

L.B. Kara and T.F. Stahovich. 2005. An Image-based, Trainable Symbol Recognizer for Hand-drawn Sketches. Comput. Graph. 29, 4 (2005), 501--517.

Digital Library

[22]

Kris Kaspersky. 2003. Code Optimization: Effective Memory Usage. A-List Publishing.

Digital Library

[23]

Eamonn Keogh. 2002. Exact Indexing of Dynamic Time Warping. In Proc. of VLDB '02. 406--417. http://dl.acm.org/citation.cfm?id=1287369.1287405

Digital Library

[24]

Sven Kratz and Michael Rohs. 2010. A $3 Gesture Recognizer: Simple Gesture Recognition for Devices Equipped with 3D Acceleration Sensors. In Proc. of IUI '10. ACM, New York, NY, USA, 341--344.

Digital Library

[25]

Per-Ola Kristensson and Shumin Zhai. 2004. SHARK2: A Large Vocabulary Shorthand Writing System for Pen-based Computers. In Proc. of UIST '04. ACM, 43--52.

Digital Library

[26]

Mary LaLomia. 1994. User Acceptance of Handwritten Recognition Accuracy. In Companion CHI '94. 107--108.

Digital Library

[27]

Yang Li. 2010. Protractor: A Fast and Accurate Gesture Recognizer. In Proc. of CHI '10. 2169--2172.

Digital Library

[28]

Hao Lü, James A. Fogarty, and Yang Li. 2014. Gesture Script: Recognizing Gestures and Their Structure Using Rendering Scripts and Interactively Trained Parts. In Proc. of CHI '14. ACM, 1685--1694.

Digital Library

[29]

Steve McConnell. 2004. Code Complete: A Practical Handbook of Software Construction, 2nd Ed. Microsoft Press. http://www.cc2e.com/Default.aspx

[30]

David McNeill. 1992. Hand and Mind: What Gestures Reveal about Thought. University of Chicago Press.

[31]

Martez E. Mott, Radu-Daniel Vatavu, Shaun K. Kane, and Jacob O. Wobbrock. 2016. Smart Touch: Improving Touch Accuracy for People with Motor Impairments with Template Matching. In Proc. of CHI '16. 1934--1946.

Digital Library

[32]

C.S. Myers and L.R. Rabiner. 1981. A comparative study of several dynamic time-warping algorithms for connected-word recognition. The Bell System Technical Journal 60, 7 (1981), 1389--1409.

[33]

Corey Pittman, Eugene M. Taranta II, and Joseph J. LaViola, Jr. 2016. A $-Family Friendly Approach to Prototype Selection. In Proc. of IUI '16. ACM, 370--374.

Digital Library

[34]

William H. Press, Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery. 1992. Numerical Recipes in C (2nd Ed.): The Art of Scientific Computing. Cambridge University Press, New York, NY, USA.

Digital Library

[35]

T. Rakthanmanon, B. Campana, A. Mueen, G. Batista, B. Westover, Q. Zhu, J. Zakaria, and E. Keogh. 2012. Searching and Mining Trillions of Time Series Subsequences Under Dynamic Time Warping. In Proc. of KDD '12. 262--270.

Digital Library

[36]

T. Rakthanmanon, B. Campana, A. Mueen, G. Batista, B. Westover, Q. Zhu, J. Zakaria, and E. Keogh. 2013. Addressing Big Data Time Series: Mining Trillions of Time Series Subsequences Under Dynamic Time Warping. ACM Trans. Knowl. Discov. Data 7, 3 (2013), 10:1--10:31.

Digital Library

[37]

Chotirat Ann Ratanamahatana and Eamonn Keogh. 2005. Three Myths about Dynamic Time Warping. In Proc. of SDM '05. 506--510.

[38]

J. Reaver, T. F. Stahovich, and J. Herold. 2011. How to Make a Quick$: Using Hierarchical Clustering to Improve the Efficiency of the Dollar Recognizer. In Proc. of SBIM '11. ACM, New York, NY, USA, 103--108.

Digital Library

[39]

Dean Rubine. 1991. Specifying Gestures by Example. In Proc. of SIGGRAPH '91. ACM, New York, NY, USA, 329--337.

Digital Library

[40]

T. Scott Saponas, Chris Harrison, and Hrvoje Benko. 2011. PocketTouch: Through-fabric Capacitive Touch Input. In Proc. of UIST '11. ACM, 303--308.

Digital Library

[41]

Stefan Schneegass and Alexandra Voit. 2016. GestureSleeve: Using Touch Sensitive Fabrics for Gestural Input on the Forearm for Controlling Smartwatches. In Proc. of ISWC '16. ACM, 108--115.

Digital Library

[42]

Tevfik Metin Sezgin and Randall Davis. 2005. HMM-based Efficient Sketch Recognition. In Proc. of IUI '05. ACM, New York, NY, USA, 281--283. DO

Digital Library

[43]

Eugene M. Taranta, II and Joseph J. LaViola, Jr. 2015. Penny Pincher: A Blazing Fast, Highly Accurate $-family Recognizer. In Proc. of GI '15. 195--202. http://dl.acm.org/citation.cfm?id=2788890.2788925

Digital Library

[44]

Eugene M. Taranta, II, Andrés N. Vargas, and Joseph J. LaViola. 2016. Streamlined and Accurate Gesture Recognition with Penny Pincher. Comput. Graph. 55, C (April 2016), 130--142.

Digital Library

[45]

Eugene M. Taranta II, Amirreza Samiei, Mehran Maghoumi, Pooya Khaloo, Corey R. Pittman, and Joseph J. LaViola Jr. 2017. Jackknife: A Reliable Recognizer with Few Samples and Many Modalities. In Proc. of CHI '17. ACM, 5850--5861.

Digital Library

[46]

Radu-Daniel Vatavu. 2011. The Effect of Sampling Rate on the Performance of Template-based Gesture Recognizers. In Proc. of ICMI '11. ACM, 271--278.

Digital Library

[47]

Radu-Daniel Vatavu. 2012a. 1F: One Accessory Feature Design for Gesture Recognizers. In Proc. of IUI '12. ACM, New York, NY, USA, 297--300.

Digital Library

[48]

Radu-Daniel Vatavu. 2012b. Small Gestures Go a Long Way: How Many Bits Per Gesture Do Recognizers Actually Need?. In Proc. of DIS '12. ACM, 328--337.

Digital Library

[49]

Radu-Daniel Vatavu. 2017. Improving Gesture Recognition Accuracy on Touch Screens for Users with Low Vision. In Proc. of CHI '17. 4667--4679.

Digital Library

[50]

Radu-Daniel Vatavu, Lisa Anthony, and Jacob O. Wobbrock. 2012. Gestures As Point Clouds: A $P Recognizer for User Interface Prototypes. In Proc. of ICMI '12. ACM, New York, NY, USA, 273--280.

Digital Library

[51]

Andrew Webb. 2002. Statistical Pattern Recognition, 2nd Ed. John Wiley & Sons, West Sussex, England.

[52]

Jacob O. Wobbrock. 2009. TapSongs: Tapping Rhythm-based Passwords on a Single Binary Sensor. In Proc. of UIST '09. ACM, New York, NY, USA, 93--96.

Digital Library

[53]

Jacob O. Wobbrock, Andrew D. Wilson, and Yang Li. 2007. Gestures Without Libraries, Toolkits or Training: A $1 Recognizer for User Interface Prototypes. In Proc. of UIST '07. ACM, 159--168.

Digital Library

[54]

Chao Xu, Parth H. Pathak, and Prasant Mohapatra. 2015. Finger-writing with Smartwatch: A Case for Finger and Hand Gesture Recognition Using Smartwatch. In Proc. of HotMobile '15. ACM, 9--14.

Digital Library

[55]

Shumin Zhai, Per Kristensson, Caroline Appert, Tue Andersen, and Xiang Cao. 2012. Foundational Issues in Touch-Surface Stroke Gesture Design: An Integrative Review. Found. Trends HCI 5, 2 (Feb. 2012), 97--205.

Digital Library

[56]

Shumin Zhai and Per-Ola Kristensson. 2003. Shorthand Writing on Stylus Keyboard. In Proc. of CHI '03. ACM, New York, NY, USA, 97--104.

Digital Library

Cited By

Zhao JPreechayasomboon PChristensen TMemar AShen ZColonnese NKhbeis MZhu M(2024)TouchpadAnyWear: Textile-Integrated Tactile Sensors for Multimodal High Spatial-Resolution Touch Inputs with Motion Artifacts ToleranceProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676344(1-14)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676344
Pascher MSaad ALiebers JHeger RGerken JSchneegass SGruenefeld UGrollman DBroadbent EJu WSoh HWilliams T(2024)Hands-On Robotics: Enabling Communication Through Direct Gesture ControlCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640635(822-827)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610978.3640635
Sluÿters ALambot SVanderdonckt JVatavu R(2023)RadarSense: Accurate Recognition of Mid-air Hand Gestures with Radar Sensing and Few Training ExamplesACM Transactions on Interactive Intelligent Systems10.1145/358964513:3(1-45)Online publication date: 11-Sep-2023
https://dl.acm.org/doi/10.1145/3589645
Show More Cited By

Recommendations

Gestures as point clouds: a $P recognizer for user interface prototypes
ICMI '12: Proceedings of the 14th ACM international conference on Multimodal interaction

Rapid prototyping of gesture interaction for emerging touch platforms requires that developers have access to fast, simple, and accurate gesture recognition approaches. The $-family of recognizers ($1, $N) addresses this need, but the current most ...
Penny pincher: a blazing fast, highly accurate $-family recognizer
GI '15: Proceedings of the 41st Graphics Interface Conference

The $-family of recognizers ($1, Protractor $N, $P, 1¢, and variants) are an easy to understand, easy to implement, accurate set of gesture recognizers designed for non-experts and rapid prototyping. They use template matching to classify candidate ...
Streamlined and accurate gesture recognition with Penny Pincher

Penny Pincher is a recently introduced template matching $-family gesture recognizer that exhibits competitive accuracy with even just one template. However, our recognizer is also able to rapidly compare a candidate gesture against numerous templates ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MobileHCI '18: Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services

September 2018

552 pages

ISBN:9781450358989

DOI:10.1145/3229434

General Chairs:
Lynne Bailie,
Nuria Oliver

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 September 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

MobileHCI '18

Sponsor:

SIGCHI

MobileHCI '18: 20th International Conference on Human-Computer Interaction with Mobile Devices and Services

September 3 - 6, 2018

Barcelona, Spain

Acceptance Rates

Overall Acceptance Rate 202 of 906 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
528
Total Downloads

Downloads (Last 12 months)122
Downloads (Last 6 weeks)12

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhao JPreechayasomboon PChristensen TMemar AShen ZColonnese NKhbeis MZhu M(2024)TouchpadAnyWear: Textile-Integrated Tactile Sensors for Multimodal High Spatial-Resolution Touch Inputs with Motion Artifacts ToleranceProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676344(1-14)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676344
Pascher MSaad ALiebers JHeger RGerken JSchneegass SGruenefeld UGrollman DBroadbent EJu WSoh HWilliams T(2024)Hands-On Robotics: Enabling Communication Through Direct Gesture ControlCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640635(822-827)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610978.3640635
Sluÿters ALambot SVanderdonckt JVatavu R(2023)RadarSense: Accurate Recognition of Mid-air Hand Gestures with Radar Sensing and Few Training ExamplesACM Transactions on Interactive Intelligent Systems10.1145/358964513:3(1-45)Online publication date: 11-Sep-2023
https://dl.acm.org/doi/10.1145/3589645
Maslych MTaranta EAldilati MLaviola J(2023)Effective 2D Stroke-based Gesture Augmentation for RNNsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581358(1-13)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581358
Vatavu R(2023)iFAD Gestures: Understanding Users’ Gesture Input Performance with Index-Finger Augmentation DevicesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580928(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580928
Yamanaka SUsuba H(2023)Tuning Endpoint-variability Parameters by Observed Error Rates to Obtain Better Prediction Accuracy of Pointing MissesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580746(1-18)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580746
Goguey AOrtega M(2023)Studies and guidelines for two concurrent stroke gesturesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2022.102942170:COnline publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1016/j.ijhcs.2022.102942
Vatavu R(2023)Gesture-Based InteractionHandbook of Human Computer Interaction10.1007/978-3-319-27648-9_20-1(1-47)Online publication date: 9-Feb-2023
https://doi.org/10.1007/978-3-319-27648-9_20-1
Costagliola GDe Rosa MFuccella V(2023)Gesture‐Based ComputingHandbook of Human‐Machine Systems10.1002/9781119863663.ch32(397-408)Online publication date: 7-Jul-2023
https://doi.org/10.1002/9781119863663.ch32
Campos CSandak JKljun MČopič Pucihar K(2022)The Hybrid Stylus: A Multi-Surface Active Stylus for Interacting with and Handwriting on Paper, Tabletop Display or BothSensors10.3390/s2218705822:18(7058)Online publication date: 18-Sep-2022
https://doi.org/10.3390/s22187058
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents