research-article

Your phone or mine?: fusing body, touch and device sensing for multi-user device-display interaction

Authors:

Mahsan Rofouei,

Andrew Wilson,

A.J. Brush,

Stewart TansleyAuthors Info & Claims

CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 1915 - 1918

https://doi.org/10.1145/2207676.2208332

Published: 05 May 2012 Publication History

Get Access

Abstract

Determining who is interacting with a multi-user interactive touch display is challenging. We describe a technique for associating multi-touch interactions to individual users and their accelerometer-equipped mobile devices. Real-time device accelerometer data and depth camera-based body tracking are compared to associate each phone with a particular user, while body tracking and touch contacts positions are compared to associate a touch contact with a specific user. It is then possible to associate touch contacts with devices, allowing for more seamless device-display multi-user interactions. We detail the technique and present a user study to validate and demonstrate a content exchange application using this approach.

References

[1]

Maunder, A. J., Marsden, G., and Harper, R. SnapAndGrab: accessing and sharing contextual multimedia content using Bluetooth enabled camera phones and large situated displays. In CHI 2008 Extended Abstracts, 2319--2324.

Digital Library

Google Scholar

[2]

Hutama, H., Song, P., Fu, C. and Goh, W. B. Distinguishing multiple smart-phone interactions on a multi-touch wall display using tilt correlation. Proc. CHI 2011, 3315--3318.

Digital Library

Google Scholar

[3]

Morris, M. R., Fisher, D., and Wigdor, D. Search on surfaces: Exploring the potential of interactive tabletops for collaborative search tasks. Inf. Processing & Management, 2009.

Digital Library

Google Scholar

[4]

Schmidt, D., Chehimi, F., Rukzio, E., and Gellersen, H., PhoneTouch: a technique for direct phone interaction on surfaces, Proc. UIST2010.

Digital Library

Google Scholar

[5]

Dohse, K. C., Dohse, T., Still, J., and Parkhurst, D. J. Enhancing multi-user interaction with multi-touch tabletop displays using hand tracking. Proc. Advances in Computer-Human Interaction 2008, 297--302.

Digital Library

Google Scholar

[6]

Dietz, P. H., and Leigh, D. DiamondTouch: A multi-user touch technology. Proc. UIST 2001, 219--226.

Digital Library

Google Scholar

[7]

Scott, S. D., Carpendale, M. S. T., and Inkpen, K. M. Territoriality in collaborative tabletop workspaces. Proc. CSCW 2004, 294--303.

Digital Library

Google Scholar

[8]

Rukzio, E., Schmidt, A., and Hussmann, H. An analysis of the usage of mobile phones for personalized interactions with ubiquitous public displays. Workshop on Ubiquitous Display Environments in conjunction with UbiComp 2004, Nottingham, UK, 2004.

Google Scholar

[9]

Grimes, A., Tarasewich, P., and Campbell, C. Keeping information private in the mobile environment. Position paper presented at First Intl. Workshop on Social Implications of Ubiquitous Computing at CHI 2005.

Google Scholar

[10]

Schöning, J., Rohs, M., Krüger A. Using Mobile Phones to Spontaneously Authenticate and Interact with MultiTouch Surfaces. AVI: Workshop on designing multitouch interaction techniques for coupled private and public displays PPD, 2008.

Google Scholar

[11]

Seewoonauth, K., Rukzio, E., Hardy, R., Holleis, P. Touch & connect and touch & select: interacting with a computer by touching it with a mobile phone, Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services, September 15--18, 2009.

Digital Library

Google Scholar

[12]

Michelle Annett, Tovi Grossman, Daniel Wigdor, and George Fitzmaurice. 2011. Medusa: a proximity-aware multi-touch tabletop. Proc. UIST 2011 337--346.

Digital Library

Google Scholar

Cited By

View all

Masullo ABurghardt TDamen DPerrett TMirmehdi M(2020)Person Re-ID by Fusion of Video Silhouettes and Wearable Signals for Home Monitoring ApplicationsSensors10.3390/s2009257620:9(2576)Online publication date: 1-May-2020
https://doi.org/10.3390/s20092576
Clarke CEhrich PGellersen H(2020)Motion Coupling of Earable Devices in Camera ViewProceedings of the 19th International Conference on Mobile and Ubiquitous Multimedia10.1145/3428361.3428470(13-17)Online publication date: 22-Nov-2020
https://dl.acm.org/doi/10.1145/3428361.3428470
Ishihara HKumano STruong KHeylen DCzerwinski MBerthouze NChetouani MNakano M(2020)Gravity-Direction-Aware Joint Inter-Device Matching and Temporal Alignment between Camera and Wearable SensorsCompanion Publication of the 2020 International Conference on Multimodal Interaction10.1145/3395035.3425968(433-441)Online publication date: 25-Oct-2020
https://dl.acm.org/doi/10.1145/3395035.3425968
Show More Cited By

Index Terms

Your phone or mine?: fusing body, touch and device sensing for multi-user device-display interaction
1. Human-centered computing

Recommendations

Design and evaluation of interaction models for multi-touch mice
GI '10: Proceedings of Graphics Interface 2010

Adding multi-touch sensing to the surface of a mouse has the potential to substantially increase the number of interactions available to the user. However, harnessing this increased bandwidth is challenging, since the user must perform multi-touch ...
Direct and indirect multi-touch interaction on a wall display
IHM '14: Proceedings of the 26th Conference on l'Interaction Homme-Machine

Multi-touch wall displays allow to take advantage of co-located interaction (direct interaction) on very large surfaces. However interacting with content beyond arms' reach requires body movements, introducing fatigue and impacting performance. ...
Thumb widgets: apply thumb-tracking to enhance capabilities of multi-touch on mobile devices
CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing Systems

While multi-touch design allows natural and flexible figure gestures in interacting with mobile devices, the palm-size screen often limits its potential in application and arouses issues such as inefficient mode switch and incomplete gesture. To enrich ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

May 2012

3276 pages

ISBN:9781450310154

DOI:10.1145/2207676

General Chair:
Joseph A. Konstan
University of Minnesota
,
Program Chairs:
Ed H. Chi
Google
,
Kristina Höök
Mobile Life at KTH

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 May 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '12

Sponsor:

SIGCHI

CHI '12: CHI Conference on Human Factors in Computing Systems

May 5 - 10, 2012

Texas, Austin, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
724
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)1

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Masullo ABurghardt TDamen DPerrett TMirmehdi M(2020)Person Re-ID by Fusion of Video Silhouettes and Wearable Signals for Home Monitoring ApplicationsSensors10.3390/s2009257620:9(2576)Online publication date: 1-May-2020
https://doi.org/10.3390/s20092576
Clarke CEhrich PGellersen H(2020)Motion Coupling of Earable Devices in Camera ViewProceedings of the 19th International Conference on Mobile and Ubiquitous Multimedia10.1145/3428361.3428470(13-17)Online publication date: 22-Nov-2020
https://dl.acm.org/doi/10.1145/3428361.3428470
Ishihara HKumano STruong KHeylen DCzerwinski MBerthouze NChetouani MNakano M(2020)Gravity-Direction-Aware Joint Inter-Device Matching and Temporal Alignment between Camera and Wearable SensorsCompanion Publication of the 2020 International Conference on Multimodal Interaction10.1145/3395035.3425968(433-441)Online publication date: 25-Oct-2020
https://dl.acm.org/doi/10.1145/3395035.3425968
Mäkelä VLinna JKeskinen THakulinen JTurunen MKhamis MSorce SCauchard JGentile V(2019)Acceptance and perceptions of interactive location-tracking displaysProceedings of the 8th ACM International Symposium on Pervasive Displays10.1145/3321335.3324931(1-7)Online publication date: 12-Jun-2019
https://dl.acm.org/doi/10.1145/3321335.3324931
Masullo ABurghardt TDamen DPerrett TMirmehdi M(2019)Who Goes There? Exploiting Silhouettes and Wearable Signals for Subject Identification in Multi-Person Environments2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)10.1109/ICCVW.2019.00199(1599-1607)Online publication date: Oct-2019
https://doi.org/10.1109/ICCVW.2019.00199
Nagatomo MAburada KYamaba HOkazaki NPark M(2019)Proposal of Ad-Hoc Secure Device Pairing Method Using Similarity Between Marker Movement and AccelerationWeb, Artificial Intelligence and Network Applications10.1007/978-3-030-15035-8_66(683-692)Online publication date: 15-Mar-2019
https://doi.org/10.1007/978-3-030-15035-8_66
Leung HFu CHeng P(2018)TwistInProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/32142752:2(1-24)Online publication date: 5-Jul-2018
https://dl.acm.org/doi/10.1145/3214275
Kharrufa APloetz TOlivier P(2017)A Unified Model for User Identification on Multi-Touch Surfaces: A Survey and Meta-AnalysisACM Transactions on Computer-Human Interaction10.1145/314456924:6(1-39)Online publication date: 19-Dec-2017
https://dl.acm.org/doi/10.1145/3144569
Clarke CBellino AEsteves AGellersen H(2017)Remote Control by Body Movement in Synchrony with Orbiting WidgetsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/31309101:3(1-22)Online publication date: 11-Sep-2017
https://dl.acm.org/doi/10.1145/3130910
Velloso ECarter MNewn JEsteves AClarke CGellersen H(2017)Motion CorrelationACM Transactions on Computer-Human Interaction10.1145/306493724:3(1-35)Online publication date: 28-Apr-2017
https://dl.acm.org/doi/10.1145/3064937
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Design and evaluation of interaction models for multi-touch mice

Direct and indirect multi-touch interaction on a wall display

Thumb widgets: apply thumb-tracking to enhance capabilities of multi-touch on mobile devices