research-article

Real-time 3D markerless multiple hand detection and tracking for human computer interaction applications

Authors:

Yiyu CaiAuthors Info & Claims

VRCAI '10: Proceedings of the 9th ACM SIGGRAPH Conference on Virtual-Reality Continuum and its Applications in Industry

Pages 323 - 330

https://doi.org/10.1145/1900179.1900247

Published: 12 December 2010 Publication History

Get Access

Abstract

In this paper we present a purely vision based implementation of markerless hand detection and tracking system which effectively detects and tracks the hand positions irrespective of hand orientation. A shape based detection algorithm using a new line approximation technique followed by an adaptive minimum distance classifier based tracking technique is implemented. The technique is very generic and can be practically used in all types of immersive and semi-immersive environments like simulations, three-dimensional (3D) games, visually assisted medical surgery and other human-computer interactive applications.

The proposed technique overcomes certain limitations of glove based, handheld object based and marker based techniques for a truly immersive experience. The technique also overcomes the main limitation of previous vision based hand tracking systems by providing much larger degrees of freedom essential for effective virtual reality systems. The novelty of the technique proposed is that it separates the detection and tracking parts and therefore allows tracking of multiple hands in a 3D space for real-time interaction.

References

[1]

Douglas D. H., Peucker T. K. 1973. Algorithms for the reduction of the number of points required to represent a line or its caricature The Canadian Cartographer, vol. 10 issue 2, December 1973, pp 112--122.

Google Scholar

[2]

Manders Corey, Farbiz Farzam, Yin Ka Ting, Miao-Long Yuan, Chong Bryan, Guan Gim Chua 2008. Interacting with 3D Objects in a Virtual Environment Using an Intuitive Gesture System, VRCAI 2008, ACM.

Digital Library

Google Scholar

[3]

Kolsch M., Turk M. 2004. Fast 2D Hand Tracking with Flocks of Features and Multi-Cue Integration, In IEEE Workshop on Real-Time Vision for Human-Computer Interaction CVPR, pp 158.

Digital Library

Google Scholar

[4]

Byungsung Lee, Junchul Chun 2009. Manipulation of Virtual Objects in Marker- less AR System by Fingertip Tracking and Hand Gesture Recognition, ACM International Conference Proceeding Series; Vol. 403, Proceedings of the 2nd International Conference on Interaction Sciences: Information Technology, Culture and Human, pp 1110--1115.

Digital Library

Google Scholar

[5]

Pierce S. Jeffery, Steams C. Brian, Pausch Randy 1999. Voodoo Dolls: Seamless Interaction at Multiple Scales in Virtual Environments, Symposium on Interactive 3D Graphics, Proceedings of the 1999 symposium on Interactive 3D graphics, pp 141--145.

Digital Library

Google Scholar

[6]

Zhigeng Pan, Yang Li, Mingmin Zhang, Chao Sun, Kangde Guo, Xing Tang, Steven Zhiying Zhou 2010. A Real-time Multi-cue Hand Tracking Algorithm Based on Computer Vision, Virtual Reality Conference (VR), 2010 IEEE, pp 219--222.

Digital Library

Google Scholar

[7]

Stoakley Richard, Conway J. Matthew, Paush Randy 1995. Virtual Reality on a WIM: Interactive Worlds in Miniature, Conference on Human Factors on Computing Systems, Proceedings of the SIGCHI conference on Human factors in computing systems, 1995, pp 265 272.

Digital Library

Google Scholar

[8]

Bowman A. Doug, Hodges F. Larry 1997. An Evaluation of Techniques for Grabbing and Manipulating Remote Objects in Immersive Virtual Environments, Symposium on Interactive 3D Graphics, Proceedings of the 1997 symposium on Interactive 3D graphics, pp 35 ff.

Digital Library

Google Scholar

[9]

Gallo Luigi, Pietro De Giuseppe, Marra Ivana 2008. 3D Interaction with Volumetric Medical Data: experiencing the Wiimote, Proceedings of the 1st international conference on Ambient media and systems, SIGCHI (2008).

Digital Library

Google Scholar

[10]

Nickel Kai, Seemann Edgar, Steifelhagen Rainer 2004. 3D-Tracking of Heads and Hands for Pointing Gesture Recognition in a Human-Robot Interaction Scenario, Sixth IEEE International Conference on Automatic Face and Gesture Recognition (FG 04).

Digital Library

Google Scholar

[11]

Frediksson Jonas, Ryen Berg Sven, Fjeld Morten 2008. Real-Time 3D Hand-Computer Interaction: Optimization and Complexity Reduction, ACM International Conference Proceeding Series, Vol. 358, Proceedings of the 5th Nordic conference on Human- computer interaction: building bridges, 2008, pp 133 141.

Digital Library

Google Scholar

[12]

Grossman Tovi, Wigdor Daniel, Balakrishnan Ravin 2004. Multi-Finger Gestural Interaction with 3D Volumetric Displays, Symposium on User Interface Software and Technology, Proceedings of the 17th annual ACM symposium on User interface software and technology, 2004, pp 61 70.

Digital Library

Google Scholar

[13]

Perez-Quninones A. Manuel A. Perez-Quninones, Sibert L. John 1996. A Collaborative Model of Feedback in Human-Computer Interaction, CHI 96, (1996), ACM Press.

Digital Library

Google Scholar

Cited By

View all

Seib VSchmidt GKusenbach MPaulus D(2015)Fourier Features For Person Detection in Depth DataComputer Analysis of Images and Patterns10.1007/978-3-319-23192-1_69(824-836)Online publication date: 25-Aug-2015
https://doi.org/10.1007/978-3-319-23192-1_69
Chen KWong SBednarz THuang WNguyen QWu Y(2014)Interactive Sand Art Drawing Using KinectProceedings of the 7th International Symposium on Visual Information Communication and Interaction10.1145/2636240.2636846(78-87)Online publication date: 5-Aug-2014
https://dl.acm.org/doi/10.1145/2636240.2636846
Ono AKakusho KFunatomi TIiyama M(2013)Replacing a Human Hand by a Virtual Hand and Adjusting Its Posture to a Virtual Object for Its Manipulation in an AR EnvironmentProceedings of the 2013 International Conference on Cyberworlds10.1109/CW.2013.90(116-119)Online publication date: 21-Oct-2013
https://dl.acm.org/doi/10.1109/CW.2013.90

Recommendations

3D hand tracking for human computer interaction

We propose a real-time model-based 3D hand tracker that combines image regions and the signal from an off-the-shelf 3-axis accelerometer placed on the user's hand. The visual regions allow the tracker to cope with occlusions, motion blur and background ...
A real time vision-based hand gestures recognition system
ISICA'10: Proceedings of the 5th international conference on Advances in computation and intelligence

Hand gesture recognition is an important aspect in Human-Computer interaction, and can be used in various applications, such as virtual reality and computer games. In this paper, we propose a real time hand gesture recognition system. It includes three ...
Robust Hand Detection and Tracking Based on Monocular Vision
IHMSC '14: Proceedings of the 2014 Sixth International Conference on Intelligent Human-Machine Systems and Cybernetics - Volume 02

Vision based hand tracking is an important area of human machine interaction (HCI) and virtual reality techniques. However, it is still a great challenge. Due to the complicated environment and various hand appearances, it is difficult to realize stable ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

VRCAI '10: Proceedings of the 9th ACM SIGGRAPH Conference on Virtual-Reality Continuum and its Applications in Industry

December 2010

399 pages

ISBN:9781450304597

DOI:10.1145/1900179

General Chairs:
Hyun Seung Yang
KAIST, Korea
,
Kazumasa Enami
National Institute of Information and Communications Technology, Japan
,
Nadia Magnenat-Thalmann
MIRALab, University of Geneva, Switzerland
,
Jim Chen
George Mason University
,
Program Chairs:
Seiki Inoue
NHK Science and Technical Research Labs, Japan
,
Zhigeng Pan
State Key Lab of CAD&CG, Zhejiang University, China
,
Jong -IL Park
Hanyang University, Korea

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 December 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Australian Research Council

Conference

VRCAI '10

Sponsor:

SIGGRAPH

VRCAI '10: Virtual Reality Continuum and its Applications in Industry

December 12 - 13, 2010

Seoul, South Korea

Acceptance Rates

Overall Acceptance Rate 51 of 107 submissions, 48%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
671
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Seib VSchmidt GKusenbach MPaulus D(2015)Fourier Features For Person Detection in Depth DataComputer Analysis of Images and Patterns10.1007/978-3-319-23192-1_69(824-836)Online publication date: 25-Aug-2015
https://doi.org/10.1007/978-3-319-23192-1_69
Chen KWong SBednarz THuang WNguyen QWu Y(2014)Interactive Sand Art Drawing Using KinectProceedings of the 7th International Symposium on Visual Information Communication and Interaction10.1145/2636240.2636846(78-87)Online publication date: 5-Aug-2014
https://dl.acm.org/doi/10.1145/2636240.2636846
Ono AKakusho KFunatomi TIiyama M(2013)Replacing a Human Hand by a Virtual Hand and Adjusting Its Posture to a Virtual Object for Its Manipulation in an AR EnvironmentProceedings of the 2013 International Conference on Cyberworlds10.1109/CW.2013.90(116-119)Online publication date: 21-Oct-2013
https://dl.acm.org/doi/10.1109/CW.2013.90

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

3D hand tracking for human computer interaction

A real time vision-based hand gestures recognition system

Robust Hand Detection and Tracking Based on Monocular Vision