Quantifying Arm and Leg Movements in 3-Month-Old Infants Using Pose Estimation: Proof of Concept
<p>Sample image of all 33 anatomical landmarks on a child. The added number indicates the following virtual markers used for the study: (1) right shoulder, (2) left shoulder, (3) right wrist, (4) left wrist, (5) right ankle, and (6) left ankle.</p> "> Figure 2
<p>Example of movement count for a single limb. In this case, we display the movement of the left ankle. Triangles represent each occurrence of a movement.</p> "> Figure 3
<p>Box plots for leg (<b>A</b>) and arm (<b>B</b>) movement rate between infants with (CCHD) and without (TD) complex congenital heart disease. Dots on each plot represent average individual movement rate data. Edges of the box represent the 25th and 75th percentile, the middle line in the box is the median, and whiskers represent maximum and minimum values.</p> "> Figure 4
<p>Box plots of sample entropy for leg and arm movements for infants with (CCHD) and without (TD) complex congenital heart disease. The side and limb for each plot is as follows: (<b>A</b>) left leg, (<b>B</b>) right leg, (<b>C</b>) left arm, (<b>D</b>) right arm. Dots on each plot represent individual data. Edges of the box represent the 25th and 75th percentile, the middle line in the box is the median, and whiskers represent maximum and minimum values. A plus represents a potential outlier according to the MATLAB function.</p> ">
Abstract
:1. Introduction
2. Materials and Methods
2.1. Participants
2.2. Procedures
2.3. Data Processing
2.4. Behavioral Coding of Leg and Arm Movements
2.5. Measurements
2.6. Statistics
3. Results
3.1. Participants
3.2. Model’s Accuracy to Detect Leg and Arm Movement in TD and CCHD
(a) | ||
Behavioral coded movement | No behavioral coded movement | |
Pose estimation movement | 499 | 57 |
No pose estimation movement | 30 | |
(b) | ||
Behavioral coded movement | No behavioral coded movement | |
Pose estimation movement | 345 | 37 |
No pose estimation movement | 27 | |
(c) | ||
Behavioral coded movement | No behavioral coded movement | |
Pose estimation movement | 154 | 20 |
No pose estimation movement | 3 |
3.3. Differences in Average Limb Movement Rate
3.4. Sample Entropy of the Acceleration Signal for Arm and Leg Movements
3.5. Associations with Movement Rate and Developmental Assessments
4. Discussion
4.1. Frequency
4.2. Two-Dimensional vs. Three-Dimensional Motion Analysis
4.3. Preliminary Findings Associating Movement Rate and Clinical Assessments
4.4. Limitations
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Rosales, M.R.; Rohloff, P.; Vanderbilt, D.L.; Tripathi, T.; Valentini, N.C.; Dusing, S.C.; Smith, B.A. Collecting infant environmental and experiential data using smartphone surveys. Pediatr. Phys. Ther. 2021, 33, 47–49. [Google Scholar] [CrossRef]
- Smith, B.A.; Trujillo-Priego, I.A.; Lane, C.J.; Finley, J.M.; Horak, F.B. Daily quantity of infant leg movement: Wearable sensor algorithm and relationship to walking onset. Sensors 2015, 15, 19006–19020. [Google Scholar] [CrossRef]
- Deng, W.; Nishiyori, R.; Vanderbilt, D.L.; Smith, B.A. How Many Days are Necessary to Represent Typical Daily Leg Movement Behavior for Infants at Risk of Developmental Disabilities? Sensors 2020, 20, 5344. [Google Scholar] [CrossRef] [PubMed]
- Krasowicz, K.; Michoński, J.; Liberadzki, P.; Sitnik, R. Monitoring Improvement in Infantile Cerebral Palsy Patients Using the 4DBODY System—A Preliminary Study. Sensors 2020, 20, 3232. [Google Scholar] [CrossRef] [PubMed]
- Schroeder, A.S.; Hesse, N.; Weinberger, R.; Tacke, U.; Gerstl, L.; Hilgendorff, A.; Heinen, F.; Arens, M.; Dijkstra, L.J.; Pujades Rocamora, S.; et al. General Movement Assessment from videos of computed 3D infant body models is equally effective compared to conventional RGB video rating. Early Hum. Dev. 2020, 144, 104967. [Google Scholar] [CrossRef]
- Lin, S.C.; Chandra, E.; Tsao, P.N.; Liao, W.C.; Chen, W.J.; Yen, T.A.; Hsu, J.Y.J.; Jeng, S.F. Application of Artificial Intelligence in Infant Movement Classification: A Reliability and Validity Study in Infants Who Were Full-Term and Preterm. Phys. Ther. 2024, 104, pzad176. [Google Scholar] [CrossRef] [PubMed]
- Cherry-Allen, K.M.; French, M.A.; Stenum, J.; Xu, J.; Roemmich, R.T. Opportunities for Improving Motor Assessment and Rehabilitation after Stroke by Leveraging Video-Based Pose Estimation. Am. J. Phys. Med. Rehabil. 2023, 102, S68–S74. [Google Scholar] [CrossRef]
- Chambers, C.; Seethapathi, N.; Saluja, R.; Loeb, H.; Pierce, S.R.; Bogen, D.K.; Prosser, L.; Johnson, M.J.; Kording, K.P. Computer Vision to Automatically Assess Infant Neuromotor Risk. IEEE Trans. Neural Syst. Rehabil. Eng. 2020, 28, 2431–2442. [Google Scholar] [CrossRef]
- Lam, W.W.T.; Tang, Y.M.; Fong, K.N.K. A systematic review of the applications of markerless motion capture (MMC) technology for clinical measurement in rehabilitation. J. Neuroeng. Rehabil. 2023, 20, 57. [Google Scholar] [CrossRef] [PubMed]
- Baumgartner, T.; Paassen, B.; Klatt, S. Extracting spatial knowledge from track and field broadcasts for monocular 3D human pose estimation. Sci. Rep. 2023, 13, 14031. [Google Scholar] [CrossRef] [PubMed]
- John, K.; Stenum, J.; Chiang, C.C.; French, M.A.; Kim, C.; Manor, J.; Statton, M.A.; Cherry-Allen, K.M.; Roemmich, R.T. Accuracy of Video-Based Gait Analysis Using Pose Estimation During Treadmill Walking Versus Overground Walking in Persons After Stroke. Phys. Ther. 2024, 104, pzad121. [Google Scholar] [CrossRef]
- Aravamuthan, B.R.; Pearson, T.S.; Ueda, K.; Miao, H.; Zerafati-Jahromi, G.; Gilbert, L.; Comella, C.; Perlmutter, J.S. Determinants of gait dystonia severity in cerebral palsy. Dev. Med. Child Neurol. 2023, 65, 968–977. [Google Scholar] [CrossRef] [PubMed]
- Spann, J.; Chen, S.A.; Ashizawa, T.; Hoque, E. Getting on the Right Foot: Using Observational and Quantitative Methods to Evaluate Movement Disorders. In Proceedings of the 29th International Conference on Intelligent User Interfaces, Greenville, SC, USA, 18–21 March 2024; pp. 742–749. [Google Scholar] [CrossRef]
- Dressler, V. Databrary. Ref. Rev. 2015, 29, 32–39. [Google Scholar] [CrossRef]
- Nieuwenhuys, A.; Õunpuu, S.; Van Campenhout, A.; Theologis, T.; De Cat, J.; Stout, J.; Molenaers, G.; De Laet, T.; Desloovere, K. Identification of joint patterns during gait in children with cerebral palsy: A Delphi consensus study. Dev. Med. Child Neurol. 2016, 58, 306–313. [Google Scholar] [CrossRef] [PubMed]
- Soska, K.C.; Seisler, A.R.; Gilmore, R.O. Databrary & Datavyu Help Videos 2018. Available online: https://nyu.databrary.org/volume/686 (accessed on 25 November 2024).
- Steffan, A.; Zimmer, L.; Arias-Trejo, N.; Bohn, M.; Dal Ben, R.; Flores-Coronado, M.A.; Franchin, L.; Garbisch, I.; Grosse Wiesmann, C.; Hamlin, J.K.; et al. Validation of an open source, remote web-based eye-tracking method (WebGazer) for research in early childhood. Infancy 2024, 29, 31–55. [Google Scholar] [CrossRef] [PubMed]
- Dickie, D.A.; Shenkin, S.D.; Anblagan, D.; Lee, J.; Cabez, M.B.; Rodriguez, D.; Boardman, J.P.; Waldman, A.; Job, D.E.; Wardlaw, J.M. Whole brain magnetic resonance image atlases: A systematic review of existing atlases and caveats for use in population imaging. Front. Neuroinform. 2017, 11, 1. [Google Scholar] [CrossRef]
- Easterby, R.; Kroemer, K.H.E.; Chaffin, D.B. (Eds.) Anthropometry and Biomechanics; Springer: Boston, MA, USA, 1982; ISBN 978-1-4684-1100-3. [Google Scholar]
- Sermpon, N.; Gima, H. Correlation between pose estimation features regarding movements towards the midline in early infancy. PLoS ONE 2024, 19, e0299758. [Google Scholar] [CrossRef] [PubMed]
- O’Brien, M.K.; Hohl, K.; Lieber, R.L.; Jayaraman, A. Automate, Illuminate, Predict: A Universal Framework for Integrating Wearable Sensors in Healthcare. Digit. Biomarkers 2024, 8, 149–158. [Google Scholar] [CrossRef]
- Deng, W.; Sargent, B.; Havens, K.; Vanderbilt, D.; Rosales, M.; Pulido, J.C.; Matarić, M.J.; Smith, B.A. Correlation between performance and quantity/variability of leg exploration in a contingency learning task during infancy. Infant Behav. Dev. 2023, 70, 101788. [Google Scholar] [CrossRef]
- Shida-Tokeshi, J.; Lane, C.J.; Trujillo-Priego, I.A.; Deng, W.; Vanderbilt, D.L.; Loeb, G.E.; Smith, B.A. Relationships between full-day arm movement characteristics and developmental status in infants with typical development as they learn to reach: An observational study. Gates Open Res. 2018, 2, 17. [Google Scholar] [CrossRef]
- Smith, B.; Vanderbilt, D.; Applequist, B.; Kyvelidou, A. Sample Entropy Identifies Differences in Spontaneous Leg Movement Behavior between Infants with Typical Development and Infants at Risk of Developmental Delay. Technologies 2017, 5, 55. [Google Scholar] [CrossRef] [PubMed]
- Rosales, M.R.; Deng, W.; Nishiyori, R.; Vanderbilt, D.L.; Smith, B.A. Leg Movement Rate before and after a Caregiver-Provided Intervention for Infants at Risk of Developmental Disability: A Pilot Study. Phys. Occup. Ther. Pediatr. 2022, 42, 259–274. [Google Scholar] [CrossRef]
- Khasgiwale, R.N.; Smith, B.A.; Looper, J. Leg Movement Rate Pre- and Post-Kicking Intervention in Infants with Down Syndrome. Phys. Occup. Ther. Pediatr. 2021, 41, 590–600. [Google Scholar] [CrossRef]
- Chen, C.Y.; Harrison, T.; Heathcock, J. Infants with complex congenital heart diseases show poor short-term memory in the mobile paradigm at 3 months of age. Infant Behav. Dev. 2015, 40, 12–19. [Google Scholar] [CrossRef]
- Campbell, S.K.; Infant Motor Performance Scales, LLC. The Test of Infant Motor Performance: Test User’s Manual Version 3.0 for the TIMP Version 5; Infant Motor Performance Scales, LLC: Glenview, IL, USA, 2012. [Google Scholar]
- Bayley, N. Bayley Scale of Infant and Toddler Development, Third Edition (Bayley-3), 3rd ed.; NCS Pearson: Bloomington, MN, USA, 2006. [Google Scholar]
- Lugaresi, C.; Tang, J.; Nash, H.; McClanahan, C.; Uboweja, E.; Hays, M.; Zhang, F.; Chang, C.-L.; Yong, M.G.; Lee, J.; et al. MediaPipe: A Framework for Building Perception Pipelines. arXiv 2019, arXiv:1906.08172. [Google Scholar]
- Knotions. Children’s Body Measurements. Available online: https://knotions.com/resources/standard-measurements/childrens-body-measurements/ (accessed on 1 May 2024).
- Fitter, N.T.; Funke, R.; Pulido, J.C.; Eisenman, L.E.; Deng, W.; Rosales, M.R.; Bradley, N.S.; Sargent, B.; Smith, B.A.; Mataric, M.J. Socially Assistive Infant-Robot Interaction: Using Robots to Encourage Infant Leg-Motion Training. IEEE Robot. Autom. Mag. 2019, 26, 12–23. [Google Scholar] [CrossRef]
- Richman, J.S.; Moorman, J.R. Physiological time-series analysis using approximate entropy and sample entropy maturity in premature infants Physiological time-series analysis using approximate entropy and sample entropy. Am. J. Physiol. Heart Circ. Physiol. 2000, 278, H2039–H2049. [Google Scholar] [CrossRef]
- Cohen, J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed.; L. Erlbaum Associates: Hillsdale, NJ, USA, 1988; ISBN 9781134742707. [Google Scholar]
- Smith, B.A. Full-day arm movement data across infancy. In Journal of Sport and Exercise Psychology, Proceedings of the North American Society for the Psychology of Sport and Physical Activity, Montreal, QC, Canada, 15–18 June 2016; Human Kinetics: Champaign, IL, USA, 2016; p. s141. [Google Scholar]
- Abrishami, M.S.; Nocera, L.; Mert, M.; Trujillo-Priego, I.A.; Purushotham, S.; Shahabi, C.; Smith, B.A. Identification of developmental delay in infants using wearable sensors: Full-day leg movement statistical feature analysis. IEEE J. Transl. Eng. Health Med. 2019, 7, 2800207. [Google Scholar] [CrossRef]
- Goodfellow, D.; Zhi, R.; Funke, R.; Pulido, J.C.; Mataric, M.; Smith, B.A. Predicting Infant Motor Development Status using Day Long Movement Data from Wearable Sensors. arXiv 2018, arXiv:1807.02617. [Google Scholar]
- Deng, W.; Vanderbilt, D.L.; Smith, B.A. Differences in spontaneous leg movement patterns between infants with typical development and infants at risk for developmental delay: Cross-sectional observation prior to sitting onset. J. Mot. Learn. Dev. 2018, 6, 101–113. [Google Scholar] [CrossRef]
- Ulrich, B.D. Opportunities for Early Intervention Based on Theory, Basic Neuroscience, and Clinical Science. Phys. Ther. 2010, 90, 1868–1880. [Google Scholar] [CrossRef]
- Lobo, M.A.; Harbourne, R.T.; Dusing, S.C.; McCoy, S.W. Grounding early intervention: Physical therapy cannot just be about motor skills anymore. Phys. Ther. 2013, 93, 94–103. [Google Scholar] [CrossRef]
- Trujillo-Priego, I.A.; Smith, B.A. Kinematic characteristics of infant leg movements produced across a full day. J. Rehabil. Assist. Technol. Eng. 2017, 4, 205566831771746. [Google Scholar] [CrossRef] [PubMed]
- Harbourne, R.T.; Lobo, M.A.; Karst, G.M.; Galloway, J.C. Sit happens: Does sitting development perturb reaching development, or vice versa? Infant Behav. Dev. 2013, 36, 438–450. [Google Scholar] [CrossRef]
- Lobo, M.A.; Galloway, J.C.; Heathcock, J.C. Characterization and intervention for upper extremity exploration and reaching behaviors in infancy. J. Hand Ther. 2015, 28, 114–125. [Google Scholar] [CrossRef] [PubMed]
- Oh, J.; Ordoñez, E.L.T.; Velasquez, E.; Mejía, M.; Grazioso, M.P.; Rohloff, P.; Smith, B.A. Early full-day leg movement kinematics and swaddling patterns in infants in rural Guatemala: A pilot study. PLoS ONE 2024, 19, e0298652. [Google Scholar] [CrossRef]
- Liao, Y.; Skelton, K.; Dunton, G.; Bruening, M. A systematic review of methods and procedures used in ecological momentary assessments of diet and physical activity research in youth: An adapted STROBE checklist for reporting EMA Studies (CREMAS). J. Med. Internet Res. 2016, 18, e151. [Google Scholar] [CrossRef] [PubMed]
(a) | ||
Behavioral coded movement | No behavioral coded movement | |
Pose estimation movement | 255 | 31 |
No pose estimation movement | 16 | |
(b) | ||
Behavioral coded movement | No behavioral coded movement | |
Pose estimation movement | 162 | 18 |
No pose estimation movement | 15 | |
(c) | ||
Behavioral coded movement | No behavioral coded movement | |
Pose estimation movement | 93 | 13 |
No pose estimation movement | 1 |
(a) | ||
Behavioral coded movement | No behavioral coded movement | |
Pose estimation movement | 244 | 26 |
No pose estimation movement | 14 | |
(b) | ||
Behavioral coded movement | No behavioral coded movement | |
Pose estimation movement | 183 | 19 |
No pose estimation movement | 12 | |
(c) | ||
Behavioral coded movement | No behavioral coded movement | |
Pose estimation movement | 61 | 7 |
No pose estimation movement | 2 |
Movement Variable | Developmental Assessment | r | p |
---|---|---|---|
Leg movement rate | Bayley—Cognitive | r = −0.15 | p = 0.64 |
Bayley—Communication | r = 0.33 | p = 0.29 | |
Bayley—Motor | r = −0.01 | p = 0.97 | |
TIMP | r = 0.47 | p = 0.12 | |
Arm movement rate | Bayley—Cognitive | r = −0.34 | p = 0.28 |
Bayley—Communication | r = 0.60 | p = 0.04 | |
Bayley—Motor | r = 0.10 | p = 0.76 | |
TIMP | r = 0.23 | p = 0.35 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Rosales, M.R.; Simsic, J.; Kneeland, T.; Heathcock, J. Quantifying Arm and Leg Movements in 3-Month-Old Infants Using Pose Estimation: Proof of Concept. Sensors 2024, 24, 7586. https://doi.org/10.3390/s24237586
Rosales MR, Simsic J, Kneeland T, Heathcock J. Quantifying Arm and Leg Movements in 3-Month-Old Infants Using Pose Estimation: Proof of Concept. Sensors. 2024; 24(23):7586. https://doi.org/10.3390/s24237586
Chicago/Turabian StyleRosales, Marcelo R., Janet Simsic, Tondi Kneeland, and Jill Heathcock. 2024. "Quantifying Arm and Leg Movements in 3-Month-Old Infants Using Pose Estimation: Proof of Concept" Sensors 24, no. 23: 7586. https://doi.org/10.3390/s24237586
APA StyleRosales, M. R., Simsic, J., Kneeland, T., & Heathcock, J. (2024). Quantifying Arm and Leg Movements in 3-Month-Old Infants Using Pose Estimation: Proof of Concept. Sensors, 24(23), 7586. https://doi.org/10.3390/s24237586