Abstract
Hand pose tracking and gesture recognition are useful techniques in human–computer interaction (HCI) scenarios, while previous work in this field suffers from the lack of discriminative features to differentiate and track hand parts. In this chapter, we present a robust hand parsing scheme to obtain a high-level and discriminative representation of the hand from raw depth image. A novel distance-adaptive feature selection method is proposed to generate more discriminative depth-context features for hand parsing. The random decision forest is adopted for per-pixel labeling, and it is combined with the temporal prior to form an ensemble of classifiers for enhanced performance. To enforce the spatial smoothness and remove the misclassified isolated regions, we further build a superpixel Markov random field, which is capable to handle the per-pixel labeling error at variable scales. To demonstrate the effectiveness of our proposed method, we have compared it to the benchmark methods. The results show it produces 17.2 % higher accuracy on the synthesized datasets for single-frame parsing. The tests on the real-world sequences show our method is more robust against complex hand poses. In addition, we develop a hand gesture recognition algorithm with the hand parsing results. The experiments show our method achieves good performance compared to state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aristidou A, Lasenby J (2010) Motion capture with constrained inverse kinematics for real-time hand tracking. In: ISCCSP, pp 1–5
Arun KS, Huang TS, Blostein SD (1987) Least-squares fitting of two 3D point sets. IEEE Trans PAMI
Baak A, Muller M, Bharaj G, Seidel HP, Theobalt C (2011) A data-driven approach for real-time full body pose reconstruction from a depth camera. In: Proceedings of ICCV
Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans PAMI
Besag J (1986) On the statistical analysis of dirty pictures. J Roy Stat Soc B 48(3):259–302
Bo L, Ren X, Fox D (2011) Depth kernel descriptors for object recognition. In: Proceedings of IROS
Breiman L (2001) Random forests. Mach Learn 45(1):5–32
CyberGlove. www.cyberglovesystems.com
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of CVPR
Erol A, Bebis G, Nicolescu M, Boyle RD, Twombly X (2005) A review on vision-based full DOF hand motion estimation. In: Proceedings of CVPR
Fauvel M, Chanussot J, Benediktsson JA (2010) SVM- and MRF-based method for accurate classification of hyperspectral images. IEEE Geosci Remote Sens Lett 7(4):736–740
Felzenszwalb PF, Huttenlocher DP (2005) Pictorial structures for object recognition. Int J Comput Vis 61(1):55–79
Heikkila J (1997) A four-step camera calibration procedure with implicit image correction. In: Proceedings of CVPR, pp 1106–1112
Kerdvibulvech C, Saito H (2009) Model-based hand tracking by chamfer distance and adaptive color learning using particle filter. EURASIP J Image Video Process
Keskin C, Kirac F, Kara YE, Akarun L (2011) Real-time hand pose estimation using depth sensors. In: Proceedings of ICCV
Keskin C, Kirac F, Kara YE, Akarun L (2012) Hand pose estimation and hand shape classification using multi-layered randomized decision forests. In: ECCV 2012
Kindermann R, Snell JL (1980) Markov random fields and their applications. The American Mathematical Society
Lewis JP, Cordner M, Fong N (2000) Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation. In: SIGGRAPH
Liang H, Yuan J, Thalmann D, Zhang Z (2013) Model-based hand pose estimation via spatial-temporal hand parsing and 3D fingertip localization, in the visual. Comput J 29(6–8):837–848
Liang H, Yuan J, Thalmann D (2014) Parsing the hand in depth images. IEEE Trans Multimedia, in Press
Li Z, Jarvis R (2009) Real time hand gesture recognition using a range camera. In: Proceedings of ACRA, pp 529–534
Lin LJ, Ying W, Huang TS (2000) Modeling the constraints of human hand motion. In: Proceedings of the workshop on human motion, pp 121–126
Lu S, Metaxas D, Samaras D, Oliensis J (2003) Using multiple cues for hand tracking and model refinement. In: Proceedings of CVPR, vol 2, pp 443–450
Pishchulin L, Andriluka M, Gehler P, Schiele B (2013) Strong appearance and expressive spatial models for human pose estimation. In: Proceedings of ICCV
Ramanan D (2007) Learning to parse images of articulated bodies. In: Proceedings of NIPS
Ren Z, Yuan J, Meng J, Zhang Z (2013) Robust part-based hand gesture recognition using kinect sensor. IEEE Trans Multimedia 15(5):1110–1120
Ren Z, Yuan J, Li C, Liu W (2011) Minimum near-convex decomposition for robust shape representation. In: Proceedings of ICCV
Rusu RB, Blodow N, Beetz M (2009) Fast point feature histograms (FPFH) for 3D registration. In: ICRA, Kobe, Japan
Shaji A, Smith K, Lucchi A, Fua P, Susstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans PAMI 34(11):2274–2282
Shen X, Hua G, Williams L, Wu Y (2011) Motion divergence fields for dynamic hand gesture recognition. In: Proceedings of FGR
Shotton J, Fitzgibbon A, Cook M, Sharp T, Finocchio M (2011) Real-time human pose recognition in parts from single depth images. In: Proceedings of CVPR
Theobalt C, Albrecht I, Haber J, Magnor M, Seidel H (2004) Pitching a baseball—tracking high-speed motion with multi-exposure images. In: SIGGRAPH
Tighe J, Lazebnik S (2010) SuperParsing: scalable nonparametric image parsing with superpixels. In: Proceedings of ECCV
Van den Bergh M, Koller-Meier E, Bosche F, Van Gool L (2009) Haarlet-based hand gesture recognition for 3D interaction. In: Proceedings of WACV
Wachs JP, Kolsch M, Stern H, Edan Y (2011) Vision-based hand-gesture applications. Commun ACM
Wang RY, Popovic J (2009) Real-time hand-tracking with a color glove. In: ACM transaction on graphics
Wang X (2009) A new localized superpixel Markov random field for image segmentation. In: Proceedings of ICME 2009
Wang H, Leu MC, Oz C (2006) American sign language recognition using multi-dimensional hidden Markov models. J Inf Sci Eng, vol 22, no. 5, pp 1109–1123
Yao Y, Fu Y (2012) Real-time hand pose estimation from RGB-D sensor. In: Proceedings of ICME
Zhao W, Chai J, Xu Y (2012) Combining marker-based mocap and RGB-D camera for acquiring high-fidelity hand motion data. In: Proceedings of the ACM SIGGRAPH/eurographics symposium on computer animation
Acknowledgments
This work is supported in part by the NTU IMI seed grant M4081078.B40.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Liang, H., Yuan, J. (2014). Hand Parsing and Gesture Recognition with a Commodity Depth Camera. In: Shao, L., Han, J., Kohli, P., Zhang, Z. (eds) Computer Vision and Machine Learning with RGB-D Sensors. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-319-08651-4_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-08651-4_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-08650-7
Online ISBN: 978-3-319-08651-4
eBook Packages: Computer ScienceComputer Science (R0)