More Web Proxy on the site http://driver.im/

research-article

Visual closed-loop control for pouring liquids

Authors:

Connor Schenck,

Dieter FoxAuthors Info & Claims

2017 IEEE International Conference on Robotics and Automation (ICRA)

Pages 2629 - 2636

https://doi.org/10.1109/ICRA.2017.7989307

Published: 29 May 2017 Publication History

Abstract

Pouring a specific amount of liquid is a challenging task. In this paper we develop methods for robots to use visual feedback to perform closed-loop control for pouring liquids. We propose both a model-based and a model-free method utilizing deep learning for estimating the volume of liquid in a container. Our results show that the model-free method is better able to estimate the volume. We combine this with a simple PID controller to pour specific amounts of liquid, and show that the robot is able to achieve an average 38ml deviation from the target amount. To our knowledge, this is the first use of raw visual feedback to pour liquids in robotics.

References

[1]

Y. Jiang, M. Lim, C. Zheng, and A. Saxena, “Learning to place new objects in a scene”, The International Journal of Robotics Research, vol. 31, no. 9, pp. 1021–1043, 2012.

Digital Library

[2]

S. Miller, J. Van Den Berg, M. Fritz, T. Darrell, K. Goldberg, and P. Abbeel, “A geometric approach to robotic laundry folding”, The International Journal of Robotics Research, vol. 31, no. 2, pp. 249–267, 2012.

Digital Library

[3]

Y. Li, X. Hu, D. Xu, Y. Yue, E. Grinspun, and P. Allen, “Multi-sensor surface analysis for robotic ironing”, arXiv preprint arXiv:, 2016.

[4]

Z. Xu and M. Cakmak, “Enhanced robotic cleaning with a low-cost tool attachment”. in IROS. 2014.

[5]

A. Yamaguchi and C. G. Atkeson, “Differential dynamic programming with temporally decomposed dynamics”, in Humanoids, 2015, pp. 696–703.

[6]

A. Yamaguchi and C. Atkeson, “Stereo vision of liquid and particle flow for robot pouring”, in Proceedings of the International Conference on Humanoid Robotics (Humanoids), 2016.

[7]

C. Schenck and D. Fox, “Towards learning to perceive and reason about liauids”. in ISER, 2016.

[8]

D. Walker, L. Smarandescu, and B. Wansink, “Half full or empty: Cues that lead wine drinkers to unintentionally overpour”, Substance Use & Misuse, vol. 49, no. 3, 2014.

[9]

L. Kunze and M. Beetz, “Envisioning the qualitative effects of robot manipulation actions using simulation-based projections”, Artificial Intelligence, 2015.

[10]

J. Langsfeld, K. Kaipa, R. Gentili, J. Reggia, and S. Gupta, “Incorpo-rating failure-to-success transitions in imitation learning for a dynamic pouring task”, in IROS Workshop on Compliant Manipulation, 2014.

[11]

K. Okada, M. Kojima, Y. Sagawa, T. Ichino, K. Sato, and M. Inaba, “Vision based behavior verification system of humanoid robot for daily environment tasks”, in Humanoids, 2006, pp. 7–12.

[12]

M. Tamosiunaite, B. Nemec, A. Ude, and F. Wörgötter, “Learning to pour with a robot arm combining goal and shape learning for dynamic movement primitives”, Robotics and Autonomous Systems, vol. 59, no. 11, pp. 910–922, 2011.

Digital Library

[13]

M. Cakmak and A. L. Thomaz, “Designing robot learners that ask good questions”, in HRI. ACM, 2012, pp. 17–24.

[14]

L. Rozo, P. Jimenez, and C. Torras, “Force-based robot learning of pouring skills using parametric hidden markov models”, in RoMoCo, 2013, PP. 227–232.

[15]

C. Schenck and D. Fox, “Guided policy search with delayed senor measurements”. arXiv preprint arXiv:, 2016.

[16]

S. Griffith, V. Sukhoy, T. Wegter, and A. Stoytchev, “Object categorization in the sink: Learning behavior-grounded object categories with water”, in Proceedings of the 2012 ICRA Workshop on Semantic Perception Mapping and Exploration, Citeseer. 2012.

[17]

A. Rankin and L. Matthies, “Daytime water detection based on color variation”, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2010, pp. 215–221.

[18]

A. L. Rankin, L. H. Matthies, and P. Bellutta, “Daytime water detection based on sky reflections”, in ICRA, 2011.

[19]

J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation”, in CVPR, 2015, pp. 3431–3440.

[20]

S. Hochreiter and J. Schmidhuber, “Long short-term memory”, Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997.

Digital Library

[21]

C. Zhang and T. Chen, “Efficient feature extraction for 2d/3d objects in mesh representation”, in Image Processing, 2001. Proceedings. 2001 International Conference on, vol. 3. IEEE, 2001, pp. 935–938.

[22]

Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell, “Caffe: Convolutional architecture for fast feature embedding”, arXiv preprint arXiv:, 2014.

[23]

K. Greff, R. K. Srivastava, J. Koutník, B. R. Steunebrink, and J. Schmidhuber, “Lstm: A search space odyssey”, arXiv preprint arXiv:, 2015.

[24]

R. B. Rusu and S. Cousins, “3d is here: Point cloud library (pcl)” in ICRA, 2011.

[25]

P. Pinggera, T. Breckon, and H. Bischof, “On cross-spectral stereo matching using dense gradient features”, in CVPR, 2012.

[26]

P. Blunsom, “Hidden markov models”, Lecture notes, August, vol. 15, pp. 18–19, 2004.

[27]

D. Kingma and J. Ba, “Adam: A method for stochastic optimization”, arXiv preprint arXiv:, 2014.

Cited By

Prezas LArkouli ZMichalos GAngelakis PGkournelos CMakris S(2024)A multi-purpose robot perception system enabling closed-loop control for zero defect manufacturing in gluing processes of large partsRobotics and Autonomous Systems10.1016/j.robot.2024.104778181:COnline publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1016/j.robot.2024.104778
Huang ZChen FPu YLin CSu HGan COh ANaumann TGloberson ASaenko KHardt MLevine S(2023)DiffVLProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3667423(29875-29900)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3667423
Ren BYe XPan ZZhang T(2022)Versatile Control of Fluid-directed Solid Objects Using Multi-task Reinforcement LearningACM Transactions on Graphics10.1145/355473142:2(1-14)Online publication date: 12-Aug-2022
https://dl.acm.org/doi/10.1145/3554731
Show More Cited By

Index Terms

Visual closed-loop control for pouring liquids

Index terms have been assigned to the content through auto-classification.

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

2017 IEEE International Conference on Robotics and Automation (ICRA)

May 2017

24678 pages

Copyright © 2017.

Publisher

IEEE Press

Publication History

Published: 29 May 2017

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Prezas LArkouli ZMichalos GAngelakis PGkournelos CMakris S(2024)A multi-purpose robot perception system enabling closed-loop control for zero defect manufacturing in gluing processes of large partsRobotics and Autonomous Systems10.1016/j.robot.2024.104778181:COnline publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1016/j.robot.2024.104778
Huang ZChen FPu YLin CSu HGan COh ANaumann TGloberson ASaenko KHardt MLevine S(2023)DiffVLProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3667423(29875-29900)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3667423
Ren BYe XPan ZZhang T(2022)Versatile Control of Fluid-directed Solid Objects Using Multi-task Reinforcement LearningACM Transactions on Graphics10.1145/355473142:2(1-14)Online publication date: 12-Aug-2022
https://dl.acm.org/doi/10.1145/3554731
Burns AXiang SLee DJackel LSong SIsler V(2022)Look and Listen: A Multi-Sensory Pouring Network and Dataset for Granular Media from Human Demonstrations2022 International Conference on Robotics and Automation (ICRA)10.1109/ICRA46639.2022.9812125(2519-2524)Online publication date: 23-May-2022
https://dl.acm.org/doi/10.1109/ICRA46639.2022.9812125
Piacenza PLee DIsler V(2022)Pouring by Feel: An Analysis of Tactile and Proprioceptive Sensing for Accurate Pouring2022 International Conference on Robotics and Automation (ICRA)10.1109/ICRA46639.2022.9811898(10248-10254)Online publication date: 23-May-2022
https://dl.acm.org/doi/10.1109/ICRA46639.2022.9811898

View Options

View options

Media

Figures

Other

Tables

View Table of Contents