More Web Proxy on the site http://driver.im/

research-article

Multiray: Multi-Finger Raycasting for Large Displays

Authors:

Fabrice Matulic,

Daniel VogelAuthors Info & Claims

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Paper No.: 245, Pages 1 - 13

https://doi.org/10.1145/3173574.3173819

Published: 21 April 2018 Publication History

Abstract

We explore and evaluate a multi-finger raycasting design space that we call "multiray". Each finger projects a ray on to the display, so the user is interacting from a distance using a form of direct input. Specifically, we propose techniques, where patterns of ray intersections created by hand postures form 2D geometric shapes to trigger actions and perform direct manipulations that go beyond single-point selections. Two formative studies examine characteristics of multi-finger raycasting for different projection methods, shapes, and tasks. Based on the results of those investigations, we demonstrate a number of dynamic UI controls and operations that utilise multiray points and shapes.

Supplementary Material

suppl.mov (pn2516-file3.mp4)

Supplemental video

Download
63.86 MB

suppl.mov (pn2516-file5.mp4)

Supplemental video

Download
11.27 MB

References

[1]

https://glofx.com/product/glofx-lzr-laser-gloves-green/

[2]

F. Argelaguet, C. Andujar and R. Trueba. 2008. Overcoming eye-hand visibility mismatch in 3D pointing selection. In Proceedings of the 2008 ACM symposium on Virtual reality software and technology, ACM, 43--46.

Digital Library

[3]

G. Bailly, R. Walter, J. Müller, T. Ning and E. Lecolinet. 2011. Comparing Free Hand Menu Techniques for Distant Displays Using Linear, Marking and FingerCount Menus. In Proceedings of the 13th IFIP TC 13 international conference on Human-computer interaction - Volume Part II, Springer Berlin Heidelberg, 248--262.

Digital Library

[4]

A. Banerjee, J. Burstyn, A. Girouard and R. Vertegaal. 2012. MultiPoint: Comparing laser and manual pointing as remote input in large display interactions. International Journal of Human-Computer Studies, 70, 10 (2012/10/01/ 2012), 690--702.

Digital Library

[5]

A. Banerjee, J. Burstyn, A. Girouard and R. Vertegaal. 2011. Pointable: an in-air pointing technique to manipulate out-of-reach targets on tabletops. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, ACM, 11--20.

Digital Library

[6]

S. Bateman, R. L. Mandryk, C. Gutwin and R. Xiao. 2013. Analysis and comparison of target assistance techniques for relative ray-cast pointing. International Journal of Human-Computer Studies, 71, 5 (2013/05/01/ 2013), 511--532.

Digital Library

[7]

M. A. Brown and W. Stuerzlinger. 2016. Exploring the Throughput Potential of In-Air Pointing. In Proceedings, Part II, of the 18th International Conference on Human-Computer Interaction. Interaction Platforms and Techniques - Volume 9732, Springer-Verlag New York, Inc., 13--24.

Digital Library

[8]

G. Casiez, N. Roussel and D. Vogel. 2012. 1 - filter: a simple speed-based low-pass filter for noisy input in interactive systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 2527--2530.

Digital Library

[9]

A. Cockburn, P. Quinn, C. Gutwin, G. Ramos and J. Looser. 2011. Air pointing: Design and evaluation of spatial target acquisition with and without visual feedback. International Journal of Human-Computer Studies, 69, 6 (2011/06/01/ 2011), 401--414.

Digital Library

[10]

W. Fikkert, P. van der Vet and A. Nijholt. 2010. Gestures in an Intelligent User Interface. In Multimedia Interaction and Intelligent User Interfaces: Principles, Methods and Applications, (2010), 215--242.

[11]

M. Frisch, S. Kleinau, R. Langner and R. Dachselt. 2011. Grids & guides: multi-touch layout and alignment tools. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 16151618.

Digital Library

[12]

E. Ghomi, S. Huot, O. Bau, M. Beaudouin-Lafon and W. E. Mackay. 2013. Arpège: learning multitouch chord gestures vocabularies. In Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces, ACM, 209--218.

Digital Library

[13]

A. Goguey, M. Nancel, G. Casiez and D. Vogel. 2016. The Performance and Preference of Different Fingers and Chords for Pointing, Dragging, and Object Transformation. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, ACM, 4250--4261.

Digital Library

[14]

V. Gracia-Ibáñez, M. Vergara and J. L. Sancho-Bru. 2016. Interdependency of the maximum range of flexion--extension of hand metacarpophalangeal joints. Computer Methods in Biomechanics and Biomedical Engineering, 19, 16 (2016/12/09 2016), 1800--1807.

[15]

C. Häger-Ross and M. H. Schieber. 2000. Quantifying the Independence of Human Finger Movements: Comparisons of Digits, Hands, and Movement Frequencies. The Journal of Neuroscience, 20, 22 (2000), 8542--8550.

[16]

F. Haque, M. Nancel and D. Vogel. 2015. Myopoint: Pointing and Clicking Using Forearm Mounted Electromyography and Inertial Motion Sensors. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM, 36533656.

Digital Library

[17]

C. Harrison, R. Xiao, J. Schwarz and S. E. Hudson. 2014. TouchTools: leveraging familiarity and skill with physical tools to augment touch interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 2913--2916.

Digital Library

[18]

R. Jota, M. A. Nacenta, J. A. Jorge, S. Carpendale and S. Greenberg. 2010. A comparison of ray pointing techniques for very large displays. In Proceedings of Graphics Interface 2010, Canadian Information Processing Society, 269--276.

Digital Library

[19]

W. A. König, J. Gerken, S. Dierdorf and H. Reiterer. 2009. Adaptive pointing: implicit gain adaptation for absolute pointing devices. In CHI '09 Extended Abstracts on Human Factors in Computing Systems, ACM, 4171--4176.

Digital Library

[20]

R. Kopper, D. A. Bowman, M. G. Silva and R. P. McMahan. 2010. A human motor behavior model for distal pointing tasks. Int. J. Hum.-Comput. Stud., 68, 10 (2010), 603--615.

Digital Library

[21]

R. Kopper, M. G. Silva, R. P. McMahan and D. A. Bowman. 2008. Increasing the Precision of Distant Pointing for Large High-Resolution Displays. Technical Report,

[22]

Y. Kou, Y. M. Kow and K. Cheng. 2015. Developing Intuitive Gestures for Spatial Interaction with Large Public Displays. In Proceedings of the Third International Conference on Distributed, Ambient, and Pervasive Interactions - Volume 9189, Springer-Verlag New York, Inc., 174--181.

Digital Library

[23]

G. J. Lepinski, T. Grossman and G. Fitzmaurice. 2010. The design and evaluation of multitouch marking menus. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 22332242.

Digital Library

[24]

Z. M. Li, S. Dun, D. A. Harkness and T. L. Brininger. 2004. Motion enslaving among multiple fingers of the human hand. Motor control, 8, 1 (Jan 2004), 1--15.

[25]

M. Liu, M. Nancel and D. Vogel. 2015. Gunslinger: Subtle Arms-down Mid-air Interaction. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, ACM, 63--71.

Digital Library

[26]

X. Lou, A. X. Li, R. Peng and P. Hansen. 2016. Optimising Free Hand Selection in Large Displays by Adapting to User's Physical Movements. In Proceedings of the 2016 Symposium on Spatial User Interaction, ACM, 23--31.

Digital Library

[27]

S. Mayer, K. Wolf, S. Schneegass and N. Henze. 2015. Modeling Distant Pointing for Compensating Systematic Displacements. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM, 4165--4168.

Digital Library

[28]

V. McArthur, S. J. Castellucci and I. S. MacKenzie. 2009. An empirical comparison of "wiimote" gun attachments for pointing tasks. In Proceedings of the 1st ACM SIGCHI symposium on Engineering interactive computing systems, ACM, 203--208.

Digital Library

[29]

T. Moscovich and J. F. Hughes. 2006. Multi-finger cursor techniques. In Proceedings of Graphics Interface 2006, Canadian Information Processing Society, 1--7.

Digital Library

[30]

B. A. Myers, R. Bhatnagar, J. Nichols, C. H. Peck, D. Kong, R. Miller and A. C. Long. 2002. Interacting at a distance: measuring the performance of laser pointers and other devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 33--40.

Digital Library

[31]

M. Nancel, J. Wagner, E. Pietriga, O. Chapuis and W. Mackay. 2011. Mid-air pan-and-zoom on wall-sized displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 177--186.

Digital Library

[32]

T. Ni, R. P. McMahan and D. A. Bowman. 2008. Technote: rapMenu: Remote Menu Selection Using Freehand Gestural Input. In Proceedings of the 2008 IEEE Symposium on 3D User Interfaces, IEEE Computer Society, 55--58.

Digital Library

[33]

M. Ortega and L. Nigay. 2009. AirMouse: Finger Gesture for 2D and 3D Interaction. In Proceedings of the 12th IFIP TC 13 International Conference on HumanComputer Interaction: Part II, Springer-Verlag, 214227.

Digital Library

[34]

K. Pietroszek, A. Kuzminykh, J. R. Wallace and E. Lank. 2014. Smartcasting: a discount 3D interaction technique for public displays. In Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: the Future of Design, ACM, 119128.

Digital Library

[35]

O. Polacek, M. Klima, A. J. Sporka, P. Zak, M. Hradis, P. Zemcik and V. Prochazka. 2012. A comparative study on distant free-hand pointing. In Proceedings of the 10th European Conference on Interactive TV and Video, ACM, 139--142.

Digital Library

[36]

I. Poupyrev, T. Ichikawa, S. Weghorst and M. Billinghurst. 1998. Egocentric Object Manipulation in Virtual Environments: Empirical Evaluation of Interaction Techniques. Computer Graphics Forum, 17, 3 (1998), 41--52.

[37]

M. Schwaller and D. Lalanne. 2013. Pointing in the Air: Measuring the Effect of Hand Selection Strategies on Performance and Effort. In Human Factors in Computing and Informatics: First International Conference, SouthCHI 2013, Maribor, Slovenia, July 13, 2013. Proceedings, (2013), 732--747.

[38]

P. Song, W. B. Goh, W. Hutama, C.-W. Fu and X. Liu. 2012. A handle bar metaphor for virtual object manipulation with mid-air interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 1297--1306.

Digital Library

[39]

S. Sridhar, A. M. Feit, C. Theobalt and A. Oulasvirta. 2015. Investigating the Dexterity of Multi-Finger Input for Mid-Air Text Entry. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM, 3643--3652.

Digital Library

[40]

J. Taylor, L. Bordeaux, T. Cashman, B. Corish, C. Keskin, T. Sharp, E. Soto, D. Sweeney, J. Valentin, B. Luff, A. Topalian, E. Wood, S. Khamis, P. Kohli, S. Izadi, R. Banks, A. Fitzgibbon and J. Shotton. 2016. Efficient and precise interactive hand tracking through joint, continuous optimization of pose and correspondences. ACM Trans. Graph., 35, 4 (2016), 112.

Digital Library

[41]

D. Vogel and R. Balakrishnan. 2005. Distant freehand pointing and clicking on very large, high resolution displays. In Proceedings of the 18th annual ACM symposium on User interface software and technology, ACM, 33--42.

Digital Library

[42]

R. Walter, G. Bailly, N. Valkanova, Jörg Müller. 2014. Cuenesics: using mid-air gestures to select items on interactive public displays. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services, ACM, 299308.

Digital Library

[43]

W. Westerman. 1999. Hand tracking, finger identification, and chordic manipulation on a multitouch surface. PhD Thesis, University of Delaware

[44]

D. Wigdor, H. Benko, J. Pella, J. Lombardo and S. Williams. 2011. Rock & rails: extending multi-touch interactions with shape gestures to enable precise spatial manipulations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 1581--1590.

Digital Library

[45]

M. L. Wittorf and M. R. Jakobsen. 2016. Eliciting MidAir Gestures for Wall-Display Interaction. In Proceedings of the 9th Nordic Conference on HumanComputer Interaction, ACM, 1--4.

Digital Library

[46]

M. Wu and R. Balakrishnan. 2003. Multi-finger and whole hand gestural interaction techniques for multiuser tabletop displays. In Proceedings of the 16th annual ACM symposium on User interface software and technology, ACM, 193--202.

Digital Library

[47]

M. Wu, S. Chia, K. Ryall, C. Forlines and R. Balakrishnan. 2006. Gesture registration, relaxation, and reuse for multi-point direct-touch surfaces. In First IEEE International Workshop on Horizontal Interactive Human-Computer Systems, 185--192.

Digital Library

[48]

H. P. Wyss, R. Blach and M. Bues. 2006. iSith Intersection-based Spatial Interaction for Two Hands. In Proceedings of the IEEE conference on Virtual Reality, IEEE Computer Society, 123.

[49]

C. Xiang, A. D. Wilson, R. Balakrishnan, K. Hinckley and S. E. Hudson. 2008. ShapeTouch: Leveraging contact shape on interactive surfaces. In Horizontal Interactive Human Computer Systems, 2008. TABLETOP 2008. 3rd IEEE International Workshop on, 129--136.

[50]

S. Zhai. 1998. User performance in relation to 3D input device design. SIGGRAPH Comput. Graph., 32, 4 (1998), 50--54.

Digital Library

Cited By

Matulic FKashima TBeker DSuzuo DFujiwara HVogel D(2024)Above-Screen Fingertip Tracking and Hand Representation for Precise Touch Input with a Phone in Virtual RealityProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670961(1-15)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670961
Eady AGuerrero ACheung VThue DGirouard A(2024)BendAide: A Deformable Interface to Augment Touchscreen Mobile DevicesProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670960(1-10)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670960
Jeong JJeong J(2024)Effect of Onset Position of Ray Casting in Virtual RealityExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650905(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650905
Show More Cited By

Index Terms

Multiray: Multi-Finger Raycasting for Large Displays
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Touchless circular menus: toward an intuitive UI for touchless interactions with large displays
AVI '14: Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces

Researchers are exploring touchless interactions in diverse usage contexts. These include interacting with public displays, where mouse and keyboards are inconvenient, activating kitchen devices without touching them with dirty hands, or supporting ...
Investigating Cross-Device Interaction between a Handheld Device and a Large Display
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

There is a growing interest in HCI research to explore cross-device interaction, giving rise to an interest in different approaches facilitating interaction between handheld devices and large displays. Contributing to this, we have investigated the use ...
Distant freehand pointing and clicking on very large, high resolution displays
UIST '05: Proceedings of the 18th annual ACM symposium on User interface software and technology

We explore the design space of freehand pointing and clicking interaction with very large high resolution displays from a distance. Three techniques for gestural pointing and two for clicking are developed and evaluated. In addition, we present subtle ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

April 2018

8489 pages

ISBN:9781450356206

DOI:10.1145/3173574

General Chairs:
Regan Mandryk
University of Saskatchewan, Canada
,
Mark Hancock
University of Waterloo, Canada
,
Program Chairs:
Mark Perry
Brunel University London, UK
,
Anna Cox
University College London, UK

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article

Funding Sources

NSERC

Conference

CHI '18

Sponsor:

SIGCHI

CHI '18: CHI Conference on Human Factors in Computing Systems

April 21 - 26, 2018

Montreal QC, Canada

Acceptance Rates

CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
886
Total Downloads

Downloads (Last 12 months)48
Downloads (Last 6 weeks)4

Reflects downloads up to 19 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Matulic FKashima TBeker DSuzuo DFujiwara HVogel D(2024)Above-Screen Fingertip Tracking and Hand Representation for Precise Touch Input with a Phone in Virtual RealityProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670961(1-15)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670961
Eady AGuerrero ACheung VThue DGirouard A(2024)BendAide: A Deformable Interface to Augment Touchscreen Mobile DevicesProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670960(1-10)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670960
Jeong JJeong J(2024)Effect of Onset Position of Ray Casting in Virtual RealityExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650905(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650905
Belkacem ITominski CMédoc NKnudsen SDachselt RGhoniem M(2024)Interactive Visualization on Large High‐Resolution Displays: A SurveyComputer Graphics Forum10.1111/cgf.1500143:6Online publication date: 30-Apr-2024
https://doi.org/10.1111/cgf.15001
Henderson JNeshati AMizobuchi SZhou WVogel DLank E(2023)Interaction Region Characteristics for Midair Barehand Targeting on a TelevisionExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585877(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585877
Matulic FKashima TBeker DSuzuo DFujiwara HVogel D(2023)Above-Screen Fingertip Tracking with a Phone in Virtual RealityExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585728(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585728
Courtoux EAppert CChapuis O(2023)SurfAirs: Surface + Mid-air Input for Large Vertical DisplaysProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580877(1-15)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580877
Lee HKim BGim SPark Y(2022)Hand-in-O: Exploring Possibilities of Sensing and Constraining the Gestures with its Frame to Provide Light and Sound Feedback[ ] With Design: Reinventing Design Modes10.1007/978-981-19-4472-7_93(1443-1452)Online publication date: 6-Nov-2022
https://doi.org/10.1007/978-981-19-4472-7_93
Seo HKim JSeo KKim BNoh J(2021)OverthereProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34781285:3(1-24)Online publication date: 14-Sep-2021
https://dl.acm.org/doi/10.1145/3478128
Xu CLi HLi ZZhang HRathore AChen XWang KHuang MXu W(2021)CardiacWaveProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34781275:3(1-26)Online publication date: 14-Sep-2021
https://dl.acm.org/doi/10.1145/3478127
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents