More Web Proxy on the site http://driver.im/

research-article

dSensingNI: a framework for advanced tangible interaction using a depth camera

Authors:

Florian Klompmaker,

Alex FastAuthors Info & Claims

TEI '12: Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction

Pages 217 - 224

https://doi.org/10.1145/2148131.2148179

Published: 19 February 2012 Publication History

Abstract

This work presents an approach to overcome the disadvantages of existing interaction frameworks and technologies for touch detection and object interaction. The robust and easy to use framework dSensingNI (Depth Sensing Natural Interaction) is described, which supports multitouch and tangible interaction with arbitrary objects. It uses images from a depth-sensing camera and provides tracking of users fingers of palm of hands and combines this with object interaction, such as grasping, grouping and stacking, which can be used for advanced interaction techniques.

Supplementary Material

WMV File (p217-klompmaker.wmv)

Download
73.78 MB

References

[1]

Aish, R. 3D input for CAAD systems. Computer Aided Design, 11(2) (1979), 66--70

[2]

Ishii, H. and Ullmer, B. Tangible Bits. Towards seamless interfaces between people, bits and atoms. In Proc.: CHI'97, ACM Press (1997), 234--241

Digital Library

[3]

Benko, H. Beyond flat surface computing: challenges of depth-aware and curved interfaces. In Proc.: 17th ACM international conference on Multimedia, ACM Press (2009), 935--944

Digital Library

[4]

Wilson, A. D. Using a depth camera as a touch sensor. In Proc.: ACM International Conference on Interactive Tabletops and Surfaces, ACM Press (2010), 69--72

Digital Library

[5]

Appenrodt, J., Al-Hamadi, A. and Michaelis, B. Data Gathering for Gesture Recognition Systems Based on Single Color-, Stereo Color- and Thermal Cameras. International Journal of Signal Processing, Image Processing and Pattern Recognition, (2010), 37--50

[6]

Hilliges, O., Izadi, S., Wilson, A. D., Hodges, S., Garcia-Mendoza, A. and Butz, A. Interactions in the air: adding further depth to interactive tabletops. In Proc.: Annual ACM symposium on User interface software and technology, ACM Press (2009), 139--148

Digital Library

[7]

Wilson, A. D. and Benko, H. Combining multiple depth cameras and projectors for interactions on, above and between surfaces. In Proc.: 23rd ACM symposium on User interface software and technology, ACM Press (2010), 273--282

Digital Library

[8]

Harrison, C., Benko, H. and Wilson, A. D. OmniTouch: wearable multitouch interaction everywhere. In Proc.: 24th annual ACM symposium on User interface software and technology, ACM Press (2011), 441--450

Digital Library

[9]

Klompmaker, F., Fischer, H. and Jung, H. Authenticated Tangible Interaction using RFID and Depth-Sensing Cameras. In Proc.: Fifth International Conference on Advances in Computer-Human Interactions. ISI Thompson Conference Proceedings (2012)

[10]

Hamer, H., Schindler, K., Koller-Meier, E. and Van Gool, L. Tracking a hand manipulating an object. In Proc.: 12th International Conference on Computer Vision, IEEE (2009), 1475--1482

[11]

Jordà, S., Geiger, G., Alonso, M. and Kaltenbrunner, M. The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces. In Proc.: 1st international conference on Tangible and embedded interaction, ACM Press (2007), 139--146

Digital Library

[12]

Weiss, M., Jennings, R., Khosabeh, R., Borchers, J., Wagner, J., Jansen, Y. and Hollan, J. D. SLAP widgets: bridging the gap between virtual and physical controls on tabletops. In Proc.: of CHI 2009, ACM Press (2009), 3229--3234.

Digital Library

[13]

Leitner, J. and Haller, M. Geckos: Combining Magnets and Pressure Images to Enable New Tangible-object Design and Interaction. In Proc.: CHI 11, ACM Press (2011), 2985--2994

Digital Library

[14]

Klompmaker, F., Nebe, K. and Jung, H. Smart Fiducials: Advanced Tangible Interaction Techniques through Dynamic Visual Patterns. IUI Workshop on Interacting with Smart Objects (2011)

Digital Library

[15]

Bartindale, T. and Harrison, C. Stacks on the surface: resolving physical order using fiducial markers with structured transparency. In Proc.: ACM International Conference on Interactive Tabletops and Surfaces, ACM Press (2009), 57--60

Digital Library

[16]

Baudisch, P., Becker, T. and Rudeck, F. Lumino: tangible blocks for tabletop computers based on glass fiber bundles. In Proc.: 28th international conference on Human factors in computing systems, ACM Press (2010), 1165--1174

Digital Library

[17]

Nebe, K., Müller, T. and Klompmaker, F. An Investigation on Requirements for Co-located Group-Work using Multitouch-, Pen-based- and Tangible-Interaction. In Proc.: HCI International, LNCS 6776, 16 (2011)

Digital Library

[18]

Ha, V., Inkpen, K. M., Whalen, T. and Mandryk, R. L. Direct Intentions: The Effects of Input Devices on Collaboration around a Tabletop Display. In Proc.: IEEE International Workshop on Horizontal Interactive Human-Computer Systems, IEEE (2006), 177--184

Digital Library

[19]

Hornecker, E. Tangible User Interfaces als kooperationsunterstüzendes Medium. PhD Thesis. published electronically at Elektronische Bibliothek, Staats und Universitätsbibliothek Bremen (2004)

[20]

Seifried, T., Haller, M., Scott, S. D., Perteneder, F., Rendl, C., Sakamoto, D. and Inami, M. CRISTAL: Design and Implementation of a Remote Control System Based on a Multi-touch Display. In Proc.: International Conference on Interactive Tabletops and Surfaces, ACM Press (2009), 37--44

Digital Library

[21]

Wilson, A. D. PlayAnywhere: a compact interactive tabletop projection-vision system. In Proc.: 18th annual ACM symposium on User interface software and technology, ACM Press (2005), 83--92

Digital Library

[22]

Wilson, A. D. and Robbins, D. C. Playtogether: Playing games across multiple interactive tabletops. IUI Workshop on Tangible Play: Research and Design for Tangible and Tabletop Games, (2007), 53--56

[23]

Jones, B., Sodhi, R., Campbell, R., Garnett, G. and Bailey, B. P. Build your world and play in it: Interacting with surface particles on complex objects. In Proc.: IEEE International Symposium on Mixed and Augmented, IEEE (2010), 165--174

[24]

Ishii, H., Ratti, C., Piper, B., Wang, Y., Bidermann, A. and Ben-Joseph, E. Bringing clay and sand into digital design - continuous tangible user interfaces. BT Technology. 22, 4 (2004), 287--299

Digital Library

[25]

Kaltenbrunner, M., Bovermann, T., Bencina, R. and Costanza, E. TUIO - A Protocol for Table-Top Tangible User Interfaces. In Proc.: 6th International Workshop on Gesture in HCI and Simulation, (2005)

Cited By

Asgarov AParsayan A(2023)3D-CNNs-Based Touchless Human-Machine InterfaceInteractive Collaborative Robotics10.1007/978-3-031-43111-1_7(70-80)Online publication date: 25-Oct-2023
https://dl.acm.org/doi/10.1007/978-3-031-43111-1_7
Günther SNelles FHorn FMühlhäuser M(2022)Demonstrating MagneTisch: Tangibles in Motion on an Interactive SurfaceProceedings of Mensch und Computer 202210.1145/3543758.3547520(590-593)Online publication date: 4-Sep-2022
https://dl.acm.org/doi/10.1145/3543758.3547520
Ntoa SMargetis GAntona MStephanidis C(2022)Digital Accessibility in Intelligent EnvironmentsHuman-Automation Interaction10.1007/978-3-031-10780-1_25(453-475)Online publication date: 1-Oct-2022
https://doi.org/10.1007/978-3-031-10780-1_25
Show More Cited By

Index Terms

dSensingNI: a framework for advanced tangible interaction using a depth camera
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Extended multitouch: recovering touch posture and differentiating users using a depth camera
UIST '12: Proceedings of the 25th annual ACM symposium on User interface software and technology

Multitouch surfaces are becoming prevalent, but most existing technologies are only capable of detecting the user's actual points of contact on the surface and not the identity, posture, and handedness of the user. In this paper, we define the concept ...
Smart Objects and Replicas: A Survey of Tangible and Embodied Interactions in Museums and Cultural Heritage Sites
Tangible and embodied interactions are areas of research within HCI and interaction design. They refer to a way in which interacting with computer systems is closer to the way we interact with the real world. Instead of using devices such as a computer or ...
Exploring how tangible tools enable collaboration in a multi-touch tabletop game
INTERACT'11: Proceedings of the 13th IFIP TC 13 international conference on Human-computer interaction - Volume Part II

Digital tabletop surfaces afford multiple user interaction and collaboration. Hybrid tabletops that include both tangible and multitouch elements are increasingly being deployed in public settings (e.g. Microsoft Surface, reacTable). Designers need to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

TEI '12: Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction

February 2012

413 pages

ISBN:9781450311748

DOI:10.1145/2148131

Conference Chair:
Roel Vertegaal
Queen's University, Canada
,
Editor:
Stephen N. Spencer
University of Washington
,
Program Chairs:
Ylva Fernaeus
Swedish Institute of Computer Science, Sweden
,
Audrey Girouard
Carleton University, Canada
,
Sergi Jordà
Universitat Pompeu Fabra, Spain

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 February 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

TEI'12

Sponsor:

SIGCHI

TEI'12: Sixth International Conference on Tangible, Embedded, and Embodied Interaction

February 19 - 22, 2012

Ontario, Kingston, Canada

Acceptance Rates

Overall Acceptance Rate 393 of 1,367 submissions, 29%

Upcoming Conference

TEI '25

Sponsor:
sigchi

Nineteenth International Conference on Tangible, Embedded, and Embodied Interaction

March 4 - 7, 2025

Bordeaux / Talence , France

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

37
Total Citations
View Citations
1,124
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)5

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Asgarov AParsayan A(2023)3D-CNNs-Based Touchless Human-Machine InterfaceInteractive Collaborative Robotics10.1007/978-3-031-43111-1_7(70-80)Online publication date: 25-Oct-2023
https://dl.acm.org/doi/10.1007/978-3-031-43111-1_7
Günther SNelles FHorn FMühlhäuser M(2022)Demonstrating MagneTisch: Tangibles in Motion on an Interactive SurfaceProceedings of Mensch und Computer 202210.1145/3543758.3547520(590-593)Online publication date: 4-Sep-2022
https://dl.acm.org/doi/10.1145/3543758.3547520
Ntoa SMargetis GAntona MStephanidis C(2022)Digital Accessibility in Intelligent EnvironmentsHuman-Automation Interaction10.1007/978-3-031-10780-1_25(453-475)Online publication date: 1-Oct-2022
https://doi.org/10.1007/978-3-031-10780-1_25
Wang JWang ZFu SHuang D(2021)Research on Hand Detection in Complex Scenes Based on RGB-D SensorHuman-Computer Interaction. Interaction Techniques and Novel Applications10.1007/978-3-030-78465-2_12(147-158)Online publication date: 3-Jul-2021
https://doi.org/10.1007/978-3-030-78465-2_12
Riemann JSchmitz MHendrich AMühlhäuser M(2018)FlowPutProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/31917632:1(1-23)Online publication date: 26-Mar-2018
https://dl.acm.org/doi/10.1145/3191763
(2017)Pattern mining and process modelling of collaborative interaction data in an online multi-tabletop learning environmentInternational Journal of Knowledge Engineering and Data Mining10.1504/IJKEDM.2017.0862264:2(114-144)Online publication date: 1-Jan-2017
https://dl.acm.org/doi/10.1504/IJKEDM.2017.086226
Tazhigaliyeva NKalidolda NImashev AIslam SAitpayev KParisi GSandygulova A(2017)Cyrillic manual alphabet recognition in RGB and RGB-D data for sign language interpreting robotic system (SLIRS)2017 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA.2017.7989526(4531-4536)Online publication date: May-2017
https://doi.org/10.1109/ICRA.2017.7989526
Ntelidakis AZabulis XGrammenos DKoutlemanis P(2017)Touch detection for planar interactive displays based on lateral depth viewsMultimedia Tools and Applications10.1007/s11042-016-3695-576:10(12683-12707)Online publication date: 1-May-2017
https://dl.acm.org/doi/10.1007/s11042-016-3695-5
Brancati NCaggianese GFrucci MGallo LNeroni P(2017)Experiencing touchless interaction with augmented content on wearable head-mounted displays in cultural heritage applicationsPersonal and Ubiquitous Computing10.1007/s00779-016-0987-821:2(203-217)Online publication date: 1-Apr-2017
https://dl.acm.org/doi/10.1007/s00779-016-0987-8
Funk MLischke LMayer SShirazi ASchmidt A(2017)Teach Me How! Interactive Assembly Instructions Using Demonstration and In-Situ ProjectionAssistive Augmentation10.1007/978-981-10-6404-3_4(49-73)Online publication date: 12-Nov-2017
https://doi.org/10.1007/978-981-10-6404-3_4
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents