More Web Proxy on the site http://driver.im/

research-article

Leveraging depth cameras and wearable pressure sensors for full-body kinematics and dynamics capture

Authors:

Jinxiang ChaiAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 33, Issue 6

Article No.: 221, Pages 1 - 14

https://doi.org/10.1145/2661229.2661286

Published: 19 November 2014 Publication History

Abstract

We present a new method for full-body motion capture that uses input data captured by three depth cameras and a pair of pressure-sensing shoes. Our system is appealing because it is low-cost, non-intrusive and fully automatic, and can accurately reconstruct both full-body kinematics and dynamics data. We first introduce a novel tracking process that automatically reconstructs 3D skeletal poses using input data captured by three Kinect cameras and wearable pressure sensors. We formulate the problem in an optimization framework and incrementally update 3D skeletal poses with observed depth data and pressure data via iterative linear solvers. The system is highly accurate because we integrate depth data from multiple depth cameras, foot pressure data, detailed full-body geometry, and environmental contact constraints into a unified framework. In addition, we develop an efficient physics-based motion reconstruction algorithm for solving internal joint torques and contact forces in the quadratic programming framework. During reconstruction, we leverage Newtonian physics, friction cone constraints, contact pressure information, and 3D kinematic poses obtained from the kinematic tracking process to reconstruct full-body dynamics data. We demonstrate the power of our approach by capturing a wide range of human movements and achieve state-of-the-art accuracy in our comparison against alternative systems.

Supplementary Material

ZIP File (a221.zip)

Supplemental material.

Download
178.58 MB

References

[1]

Adelsberger, R., and Tröster, G. 2013. Pimu: A wireless pressure-sensing imu. In ISSNIP, IEEE Proceedings of the 8th International Conference on Intelligent Sensors, Sensor Networks and Information Processing.

[2]

Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: Reconstruction and parameterization from range scans. ACM Trans. Graph. 22, 3 (July), 587--594.

Digital Library

[3]

Ascension, 2014. http://www.ascension-tech.com/.

[4]

Baak, A., Müller, M., Bharaj, G., Seidel, H.-P., and Theobalt, C. 2011. A data-driven approach for real-time full body pose reconstruction from a depth camera. In IEEE 13th International Conference on Computer Vision (ICCV), 1092--1099.

Digital Library

[5]

Bregler, C., Malik, J., and k Pullen. 2004. Twist based acquisition and tracking of animal and human kinematics. International Journal of Computer Vision. 56(3):179--194.

Digital Library

[6]

Bridson, R., Marino, S., and Fedkiw, R. 2003. Simulation of clothing with folds and wrinkles. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA '03, 28--36.

Digital Library

[7]

Brubaker, M. A., and Fleet, D. J. 2008. The Kneed Walker for human pose tracking. In Proceedings of IEEE CVPR.

[8]

Curless, B., and Levoy, M. 1996. A volumetric method for building complex models from range images. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, ACM, New York, NY, USA, SIGGRAPH '96, 303--312.

Digital Library

[9]

de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Trans. Graph. 27, 3 (Aug.), 98:1--98:10.

Digital Library

[10]

Elgammal, A., and Lee, C. 2004. Inferring 3D body pose from silhouettes using activity manifold learning. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2: 681--688.

Digital Library

[11]

Fischler, M. A., and Bolles, R. C. 1981. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Comm. of the ACM 24, 6, 381--395.

Digital Library

[12]

Fisher, S., and Lin, M. C. 2001. Deformed distance fields for simulation of non-penetrating flexible bodies. In Proceedings of the Eurographic Workshop on Computer Animation and Simulation, Springer-Verlag New York, Inc., New York, NY, USA, 99--111.

Digital Library

[13]

Grest, D., Kruger, V., and Koch, R. 2007. Single view motion tracking by depth and silhouette information. In Proceedings of the 15th Scandinavian Conference on Image Analysis (SCIA), 719--729.

Digital Library

[14]

Guendelman, E., Bridson, R., and Fedkiw, R. 2003. Non-convex rigid bodies with stacking. ACM Trans. Graph. 22, 3 (July), 871--878.

Digital Library

[15]

Ha, S., Bai, Y., and Liu, C. K. 2011. Human motion reconstruction from force sensors. In Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ACM, New York, NY, USA, SCA '11, 129--138.

Digital Library

[16]

Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., and Seidel, H.-P. 2009. A statistical model of human pose and body shape. In Computer Graphics Forum (Proc. Eurographics 2008), P. Dutr'e and M. Stamminger, Eds., vol. 2.

[17]

Knoop, S., Vacek, S., and Dillmann, R. 2006. Sensor fusion for 3d human body tracking with an articulated 3d body model. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 1686--1691.

[18]

Kovar, L., Schreiner, J., and Gleicher, M. 2002. Footskate cleanup for motion capture editing. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ACM, New York, NY, USA, SCA '02, 97--104.

Digital Library

[19]

Microsoft Kinect API for Windows, 2014. http://www.microsoft.com/en-us/kinectforwindows/.

[20]

Moeslund, T. B., Hilton, A., and Krüger, V. 2006. A survey of advances in vision-based human motion capture and analysis. Journal of Computer Vision and Image Understanding. 104:90--126.

Digital Library

[21]

Pavlović, V., Rehg, J. M., and MacCormick, J. 2000. Learning Switching Linear Models of Human Motion. In Advances in Neural Information Processing Systems 13, 981--987.

[22]

Plagemann, C., Ganapathi, V., Koller, D., and Thrun, S. 2010. Realtime identification and localization of body parts from depth images. In Proceedings of International Conferences on Robotics and Automation (ICRA 2010), 3108--3113.

[23]

Rosales, R., and Sclaroff, S. 2000. Specialized mappings and the estimation of human body pose from a single image. In Proceedings of the Workshop on Human Motion. 19--24.

Digital Library

[24]

Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., and Blake, A. 2011. Real-time human pose recognition in parts from a single depth image. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1297--1304.

Digital Library

[25]

Tekscan, 2014. http://www.tekscan.com/.

[26]

Urtasun, R., Fleet, D. J., Hertzmann, A., and Fua., P. 2005. Priors for people tracking from small training sets. In IEEE International Conference on Computer Vision, 403--C410.

Digital Library

[27]

Vicon Systems, 2014. http://www.vicon.com.

[28]

Vlasic, D., Baran, I., Matusik, W., and Popović, J. 2008. Articulated mesh animation from multi-view silhouettes. ACM Trans. Graph. 27, 3 (Aug.), 97:1--97:9.

Digital Library

[29]

Vondrak, M., Sigal, L., and Jenkins, O. C. 2008. Physical simulation for probabilistic motion tracking. In IEEE Conference on Computer Vision and Pattern Recognition, 1--8.

[30]

Vondrak, M., Sigal, L., Hodgins, J., and Jenkins, O. 2012. Video-based 3d motion capture through biped control. ACM Trans. Graph. 31, 4 (July), 27:1--27:12.

Digital Library

[31]

Wei, X. K., and Chai, J. 2010. Videomocap: Modeling physically realistic human motion from monocular video sequences. ACM Transactions on Graphics. 29(4): Article No. 42.

Digital Library

[32]

Wei, X., Zhang, P., and Chai, J. 2012. Accurate realtime full-body motion capture using a single depth camera. ACM Trans. Graph. 31, 6 (Nov.), 188:1--188:12.

Digital Library

[33]

Xsens, 2014. http://www.xsens.com.

[34]

Ye, M., Wang, X., Yang, R., Ren, L., and Pollefeys, M. 2011. Accurate 3d pose estimation from a single depth image. In Proceedings of IEEE 13th International Conference on Computer Vision, 731--738.

Digital Library

[35]

Yin, K., and Pai, D. K. 2003. Footsee: An interactive animation system. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA '03, 329--338.

Digital Library

Cited By

Choi JOh JKo TChung BChoi YLee SOh H(2025)Intelligent metallic loose part monitoring in three-dimensional structures using convolutional neural networks and the position-invariant loss functionNuclear Engineering and Technology10.1016/j.net.2025.103474(103474)Online publication date: Jan-2025
https://doi.org/10.1016/j.net.2025.103474
Wang SDeng YDeng MYu HZhou JChen DKomura TWu JZhu B(2024)An Eulerian Vortex Method on Flow MapsACM Transactions on Graphics10.1145/368799643:6(1-14)Online publication date: 19-Dec-2024
https://dl.acm.org/doi/10.1145/3687996
Smith IScheme EBateman S(2024)Mapping Real World Locomotion Speed to the Virtual World in Large Field of View Virtual EnvironmentsProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682077(1-12)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682077
Show More Cited By

Index Terms

Leveraging depth cameras and wearable pressure sensors for full-body kinematics and dynamics capture
1. Computing methodologies
  1. Computer graphics
    1. Animation

Recommendations

Accurate realtime full-body motion capture using a single depth camera

We present a fast, automatic method for accurately capturing full-body motion data using a single depth camera. At the core of our system lies a realtime registration process that accurately reconstructs 3D human poses from single monocular depth images,...
Motion capture from body-mounted cameras
SIGGRAPH '11: ACM SIGGRAPH 2011 papers

Motion capture technology generally requires that recordings be performed in a laboratory or closed stage setting with controlled lighting. This restriction precludes the capture of motions that require an outdoor setting or the traversal of large areas. ...
Motion capture from body-mounted cameras

Motion capture technology generally requires that recordings be performed in a laboratory or closed stage setting with controlled lighting. This restriction precludes the capture of motions that require an outdoor setting or the traversal of large areas. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 33, Issue 6

November 2014

704 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2661229

Issue’s Table of Contents

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 November 2014

Published in TOG Volume 33, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

74
Total Citations
View Citations
756
Total Downloads

Downloads (Last 12 months)44
Downloads (Last 6 weeks)3

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Choi JOh JKo TChung BChoi YLee SOh H(2025)Intelligent metallic loose part monitoring in three-dimensional structures using convolutional neural networks and the position-invariant loss functionNuclear Engineering and Technology10.1016/j.net.2025.103474(103474)Online publication date: Jan-2025
https://doi.org/10.1016/j.net.2025.103474
Wang SDeng YDeng MYu HZhou JChen DKomura TWu JZhu B(2024)An Eulerian Vortex Method on Flow MapsACM Transactions on Graphics10.1145/368799643:6(1-14)Online publication date: 19-Dec-2024
https://dl.acm.org/doi/10.1145/3687996
Smith IScheme EBateman S(2024)Mapping Real World Locomotion Speed to the Virtual World in Large Field of View Virtual EnvironmentsProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682077(1-12)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682077
Zhang HRen SYuan HZhao JLi FSun SLiang ZYu TShen QCao X(2024)MMVP: A Multimodal MoCap Dataset with Vision and Pressure Sensors2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.02063(21842-21852)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.02063
Tao RZhang XRen HDong YYang X(2024)A unified particle method for fluid simulation in ship fire scenarioOcean Engineering10.1016/j.oceaneng.2024.119266312(119266)Online publication date: Nov-2024
https://doi.org/10.1016/j.oceaneng.2024.119266
Liu JZhang TSun S(2024)Review of deep learning algorithms in molecular simulations and perspective applications on petroleum engineeringGeoscience Frontiers10.1016/j.gsf.2023.10173515:2(101735)Online publication date: Mar-2024
https://doi.org/10.1016/j.gsf.2023.101735
Tao RZhang XRen HYang XZhou Y(2024)Wall-bounded flow simulation on vortex dynamicsComputers & Graphics10.1016/j.cag.2024.103990122(103990)Online publication date: Aug-2024
https://doi.org/10.1016/j.cag.2024.103990
YANG WLI YXING SCAI JWANG X(2023)Lightweight multi-person motion capture system in the wildSCIENTIA SINICA Informationis10.1360/SSI-2022-039753:11(2230)Online publication date: 31-Oct-2023
https://doi.org/10.1360/SSI-2022-0397
Mourot LHoyet LClerc FHellier P(2023)UnderPressure: Deep Learning for Foot Contact Detection, Ground Reaction Force Estimation and Footskate CleanupComputer Graphics Forum10.1111/cgf.1463541:8(195-206)Online publication date: 20-Mar-2023
https://doi.org/10.1111/cgf.14635
Keyvanian SJohnson MFigueroa N(2023)Learning Realistic Joint Space Boundaries for Range of Motion Analysis of Healthy and Impaired Human Arms2023 IEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids)10.1109/Humanoids57100.2023.10375147(1-8)Online publication date: 12-Dec-2023
https://doi.org/10.1109/Humanoids57100.2023.10375147
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents