Abstract
Many existing methods that learn robot motion planning task models or control policies from demonstrations require that the demonstrations be temporally aligned. Temporal registration involves an assignment of individual observations from a demonstration to the ordered steps in some reference model, which facilitates learning features of the motion over time. We introduce probability-weighted temporal registration (PTR), a general form of temporal registration that includes two useful features for motion planning and control policy learning. First, PTR explicitly captures uncertainty in the temporal registration. Second PTR avoids degenerate registrations in which too few observations are aligned to a time step. Our approach is based on the forward-backward algorithm. We show how to apply PTR to two task model learning methods from prior work, one which learns a control policy and another which learns costs for a sampling-based motion planner. We show that incorporating PTR yields higher-quality learned task models that enable faster task executions and higher task success rates.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Amor, H.B., Neumann, G., Kamthe, S., Kroemer, O., Peters, J.: Interaction primitives for human-robot cooperation tasks. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 2831–2837 (2014)
Baum, L.E., Petrie, T., Soules, G., Weiss, N.: A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Stat. 41(1), 164–171 (1970)
Bellman, R.: On a routing problem. Technical report, DTIC Document (1956)
van den Berg, J., Miller, S., Duckworth, D., Hu, H., Wan, A., Goldberg, K., Abbeel, P.: Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 2074–2081 (2010)
Berger, E., Sastuba, M., Vogt, D., Jung, B., Ben Amor, H.: Estimation of perturbations in robotic behavior using dynamic mode decomposition. Adv. Robot. 29(5), 331–343 (2015)
Bodiroža, S., Doisy, G., Hafner, V.V.: Position-invariant, real-time gesture recognition based on dynamic time warping. In: ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 87–88 (2013)
Bowen, C., Ye, G., Alterovitz, R.: Asymptotically-optimal motion planning for learned tasks using time-dependent cost maps. IEEE Trans. Autom. Sci. Eng. 12(1), 171–182 (2015)
Calinon, S.: A tutorial on task-parameterized movement learning and retrieval. Intell. Serv. Robot. 9(1), 1–29 (2016)
Calinon, S., Guenter, F., Billard, A.: On learning, representing, and generalizing a task in a humanoid robot. IEEE Trans. Syst. Man Cybern.–Part B 37(2), 286–298 (2007)
Davis, R.I., Lovell, B.C.: Comparing and evaluating HMM ensemble training algorithms using train and test and condition number criteria. Formal Pattern Anal. Appl. 6(4), 327–335 (2004)
Herstein, I.N.: Topics in Algebra. Wiley, Hoboken (2006)
Itakura, F.: Minimum prediction residual principle applied to speech recognition. IEEE Trans. Acoust. Speech Signal Process. 23(1), 67–72 (1975)
Krishnan, S., Garg, A., Liaw, R., Thananjeyan, B., Miller, L., Pokorny, F.T., Goldberg, K.: SWIRL: a sequential windowed inverse reinforcement learning algorithm for robot tasks with delayed rewards. In: WAFR (2016)
Lee, A.X., Lu, H., Gupta, A., Levine, S., Abbeel, P.: Learning force-based manipulation of deformable objects from multiple demonstrations. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 177–184 (2015)
Lember, J., Koloydenko, A.A.: Bridging Viterbi and posterior decoding: a generalized risk approach to hidden path inference based on hidden Markov models. J. Mach. Learn. Res. 15(1), 1–58 (2014)
Lv, F., Nevatia, R.: Single view human action recognition using key pose matching and Viterbi path searching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)
Maeda, G., Ewerton, M., Lioutikov, R., Amor, H.B., Peters, J., Neumann, G.: Learning interaction for collaborative tasks with probabilistic movement primitives. In: IEEE-RAS International Conference on Humanoid Robots, pp. 527–534 (2014)
Marteau, P.: Times series averaging and denoising from a probabilistic perspective on time-elastic kernels (2016). http://arxiv.org/abs/1611.09194
Müller, M.: Dynamic time warping. In: Information Retrieval for Music and Motion, pp. 69–84 (2007)
Pavlovic, V., Rehg, J.M., MacCormick, J.: Learning switching linear models of human motion. In: Advances in Neural Information Processing Systems, pp. 981–987 (2001)
Tanwani, A.K., Calinon, S.: Learning robot manipulation tasks with task-parameterized semitied hidden semi-Markov model. IEEE Robot. Autom. Lett. 1(1), 235–242 (2016)
Titsias, M.K., Holmes, C.C., Yau, C.: Statistical inference in hidden Markov models using k-segment constraints. J. Am. Stat. Assoc. 111(513), 200–215 (2016)
Vakanski, A., Mantegh, I., Irish, A., Janabi-Sharifi, F.: Trajectory learning for robot programming by demonstration using hidden Markov model and dynamic time warping. IEEE Trans. Syst. Man Cybern. Part B: Cybern. 42(4), 1039–1052 (2012)
Viterbi, A.J.: Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Trans. Inf. Theory 13(2), 260–269 (1967)
Vuković, N., Mitić, M., Miljković, Z.: Trajectory learning and reproduction for differential drive mobile robots based on GMM/HMM and dynamic time warping using learning from demonstration framework. Eng. Appl. Artif. Intell. 45, 388–404 (2015)
Weichsel, P.M.: The Kronecker product of graphs. Proc. Am. Math. Soc. 13(1), 47–52 (1962)
Yu, S.Z., Kobayashi, H.: An efficient forward-backward algorithm for an explicit-duration hidden Markov model. IEEE Signal Process. Lett. 10(1), 11–14 (2003)
Acknowledgments
We thank Armaan Sethi for his assistance evaluating methods. This research was supported in part by the U.S. National Science Foundation (NSF) under Awards IIS-1149965 and CCF-1533844.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Bowen, C., Alterovitz, R. (2020). Probability-Weighted Temporal Registration for Improving Robot Motion Planning and Control Learned from Demonstrations. In: Morales, M., Tapia, L., Sánchez-Ante, G., Hutchinson, S. (eds) Algorithmic Foundations of Robotics XIII. WAFR 2018. Springer Proceedings in Advanced Robotics, vol 14. Springer, Cham. https://doi.org/10.1007/978-3-030-44051-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-44051-0_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-44050-3
Online ISBN: 978-3-030-44051-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)