More Web Proxy on the site http://driver.im/

research-article

Robot task planning and explanation in open and uncertain worlds

Authors:

Moritz Gbelbecker,

Graham S. Horn,

Andrzej Pronobis,

Patric Jensfelt,

Charles Gretton,

Richard Dearden,

Miroslav Janicek,

Hendrik Zender,

Geert-Jan Kruijff,

Jeremy L. WyattAuthors Info & Claims

Artificial Intelligence, Volume 247, Issue C

Pages 119 - 150

https://doi.org/10.1016/j.artint.2015.08.008

Published: 01 June 2017 Publication History

Abstract

A long-standing goal of AI is to enable robots to plan in the face of uncertain and incomplete information, and to handle task failure intelligently. This paper shows how to achieve this. There are two central ideas. The first idea is to organize the robot's knowledge into three layers: instance knowledge at the bottom, commonsense knowledge above that, and diagnostic knowledge on top. Knowledge in a layer above can be used to modify knowledge in the layer(s) below. The second idea is that the robot should represent not just how its actions change the world, but also what it knows or believes. There are two types of knowledge effects the robot's actions can have: epistemic effects (I believe X because I saw it) and assumptions (I'll assume X to be true). By combining the knowledge layers with the models of knowledge effects, we can simultaneously solve several problems in robotics: (i) task planning and execution under uncertainty; (ii) task planning and execution in open worlds; (iii) explaining task failure; (iv) verifying those explanations. The paper describes how the ideas are implemented in a three-layer architecture on a mobile robot platform. The robot implementation was evaluated in five different experiments on object search, mapping, and room categorization.

References

[1]

A. Aydemir, M. Gbelbecker, A. Pronobis, K. Sj, P. Jensfelt, Plan-based object search and exploration using semantic spatial knowledge in the real world, in: Proceedings of the European Conference on Mobile Robotics, Sep. 2011, pp. 13-18.

[2]

A. Aydemir, A. Pronobis, M. Gobelbecker, P. Jensfelt, Active visual object search in unknown environments using uncertain semantics, IEEE Trans. Robot., 29 (2013) 986-1002.

Digital Library

[3]

A. Aydemir, K. Sj, J. Folkesson, P. Jensfelt, Search in the real world: active visual object search based on spatial relations, in: Proceedings of the 2011 IEEE International Conference on Robotics and Automation, May 2011, pp. 2818-2824.

[4]

C. Bckstrm, B. Nebel, Complexity results for SAS+ planning, Comput. Intell., 11 (1995) 625-655.

[5]

M. Brenner, B. Nebel, Continual planning and acting in dynamic multiagent environments, Auton. Agents Multi-Agent Syst., 19 (2009) 297-331.

Digital Library

[6]

Cognitive Systems, in: Cognitive Systems Monographs, vol. 8, Springer, Berlin, 2010.

Digital Library

[7]

S. Ekvall, D. Kragic, P. Jensfelt, Object detection and mapping for service robot tasks, Robotica, 25 (2007) 175-187.

Digital Library

[8]

C. Galindo, J.-A. Fernndez-Madrigal, J. Gonzlez, A. Saffiotti, Robot task planning using semantic maps, Robot. Auton. Syst., 56 (2008) 955-966.

Digital Library

[9]

P. Gardenfors, Belief revisions and the Ramsey test for conditionals, Philos. Rev., 95 (1986) 81-93.

[10]

A.E. Gerevini, P. Haslum, D. Long, A. Saetti, Y. Dimopoulos, Deterministic planning in the Fifth International Planning Competition: PDDL3 and experimental evaluation of the planners, Artif. Intell., 173 (2009) 619-668.

Digital Library

[11]

M. Gbelbecker, C. Gretton, R. Dearden, A switching planner for combined task and observation planning, in: Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, 2011, pp. 964-970.

Digital Library

[12]

H. Gonzlez-Banos, J. Latombe, A randomized art-gallery algorithm for sensor placement, in: Proceedings of the Seventeenth Annual Symposium on Computational Geometry, ACM, New York, USA, 2001, pp. 232-240.

Digital Library

[13]

M. Hanheide, C. Gretton, R.W. Dearden, N.A. Hawes, J.L. Wyatt, A. Pronobis, A. Aydemir, M. Gbelbecker, H. Zender, Exploiting probabilistic knowledge under uncertain sensing for efficient robot behaviours, in: Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, 2011, pp. 2442-2449.

Digital Library

[14]

M. Hanheide, N. Hawes, J.L. Wyatt, M. Gbelbecker, M. Brenner, K. Sj, A. Aydemir, P. Jensfelt, H. Zender, G.-J.M. Kruijff, A framework for goal generation and management, in: Proceedings of the AAAI Workshop on Goal-Directed Autonomy, 2010.

[15]

N. Hawes, M. Hanheide, J. Hargreaves, B. Page, H. Zender, P. Jensfelt, Home alone: autonomous extension and correction of spatial representations, in: Proceedings of the 2011 IEEE International Conference on Robotics and Automation, 2011, pp. 3907-3914.

[16]

N. Hawes, J.L. Wyatt, Engineering intelligent information-processing systems with CAST, Adv. Eng. Inform., 24 (2010) 27-39.

Digital Library

[17]

N. Hawes, J.L. Wyatt, M. Sridharan, H. Jacobsson, R. Dearden, A. Sloman, G.-J. Kruijff, Architecture and representations, in: Cognitive Systems, Springer, Berlin, 2010, pp. 51-93.

[18]

M. Helmert, The fast downward planning system, J. Artif. Intell. Res., 26 (2006) 191-246.

Digital Library

[19]

H. Jacobsson, N. Hawes, G.-J. Kruijff, J. Wyatt, Crossmodal content binding in information-processing architectures, in: Proceedings of the 3rd ACM/IEEE International Conference on Human Robot Interaction, 2008, pp. 81-88.

Digital Library

[20]

D. Jain, L. Mosenlechner, M. Beetz, Equipping robot control programs with first-order probabilistic reasoning capabilities, in: Proceedings of 2009 IEEE International Conference on Robotics and Automation, IEEE, May 2009, pp. 3626-3631.

Digital Library

[21]

M. Janek, Abductive reasoning for continual dialogue understanding, in: New Directions in Logic, Language, and Computation, Springer, 2012, pp. 16-31.

Digital Library

[22]

R. Kaplow, A. Atrash, J. Pineau, Variable resolution decomposition for robotic navigation under a POMDP framework, in: Proceedings of the 2010 IEEE International Conference on Robotics and Automation, 2010, pp. 369-376.

[23]

M. Klenk, M. Molineaux, D.W. Aha, Goal-driven autonomy for responding to unexpected events in strategy simulations, Comput. Intell., 29 (2013) 187-206.

[24]

T. Kollar, N. Roy, Utilizing objectobject and objectscene context when planning to find things, in: Proceedings of the 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan, 2009, pp. 4116-4121.

Digital Library

[25]

D. Kraft, E. Baeski, M. Popovi, A.M. Batog, A. Kjr-Nielsen, N. Krger, R. Petrick, C. Geib, N. Pugeault, M. Steedman, T. Asfour, R. Dillmann, S. Kalkan, F. Wrgtter, B. Hommel, R. Detry, J. Piater, Exploration and planning in a three-level cognitive architecture, in: International Conference on Cognitive Systems, 2008, pp. 71-78.

[26]

L. Kunze, M. Beetz, M. Saito, H. Azuma, K. Okada, Searching objects in large-scale indoor environments: a decision-theoretic approach, in: Proceedings of the 2012 IEEE International Conference on Robotics and Automation, 2012, pp. 4385-4390.

[27]

D. Lewis, Counterfactuals, Harvard University Press, 1973.

[28]

R. Lienhart, J. Maydt, An extended set of Haar-like features for rapid object detection, in: Proceedings of the 2002 International Conference on Image Processing, vol. 1, Rochester, New York, 2002, pp. 900-903.

[29]

R. Martinez-Cantin, N. de Freitas, E. Brochu, J. Castellanos, A. Doucet, A Bayesian explorationexploitation approach for optimal online sensing and planning with a visually guided mobile robot, Auton. Robots, 27 (2009) 93-103.

[30]

T.M. Mitchell, R.M. Keller, S.T. Kedar-Cabelli, Explanation-based generalization: a unifying view, Mach. Learn., 1 (1986) 47-80.

Digital Library

[31]

J.M. Mooij, libDAI: a free and open source C++ library for discrete approximate inference in graphical models, J. Mach. Learn. Res., 11 (Aug. 2010) 2169-2173.

Digital Library

[32]

R. Peter Bonasso, R. James Firby, E. Gat, D. Kortenkamp, D.P. Miller, M.G. Slack, Experiences with an architecture for intelligent, reactive agents, J. Exp. Theor. Artif. Intell., 9 (Apr. 1997) 237-256.

[33]

R.P.A. Petrick, F. Bacchus, A knowledge-based approach to planning with incomplete information and sensing, in: Proceedings of the Sixth International Conference on Artificial Intelligence Planning Systems 2002, 2002, pp. 212-222.

Digital Library

[34]

J. Pineau, M. Montemerlo, M. Pollack, N. Roy, S. Thrun, Towards robotic assistants in nursing homes: challenges and results, Robot. Auton. Syst., 42 (2003) 271-281.

[35]

A. Pronobis, P. Jensfelt, Large-scale semantic mapping and reasoning with heterogeneous modalities, in: Proceedings of the 2012 IEEE International Conference on Robotics and Automation, 2012, pp. 3515-3522.

[36]

A. Pronobis, O.M. Mozos, B. Caputo, P. Jensfelt, Multi-modal semantic place classification, Int. J. Robot. Res., 29 (February 2010) 298-320.

Digital Library

[37]

A. Pronobis, K. Sj, A. Aydemir, A.N. Bishop, P. Jensfelt, A framework for robust cognitive spatial mapping, in: Proceedings of the 14th International Conference on Advanced Robotics, 2009, pp. 1-8.

[38]

A. Pronobis, K. Sj, A. Aydemir, A.N. Bishop, P. Jensfelt, Representing spatial knowledge in mobile cognitive systems, in: 11th International Conference on Intelligent Autonomous Systems, 2010, pp. 133-142.

[39]

R. Reiter, A theory of diagnosis from first principles, Artif. Intell., 32 (1987) 57-95.

Digital Library

[40]

A. Richtsfeld, T. Mrwald, M. Zillich, M. Vincze, Taking in shape: detection and tracking of basic 3D shapes in a robotics context, in: Proceedings of the 15th Computer Vision Winter Workshop, 2010, pp. 91-98.

[41]

M. Shanahan, Prediction is deduction but explanation is abduction, in: Proceedings of the 11th International Joint Conference on Artificial Intelligence, vol. 2, 1989, pp. 1055-1060.

Digital Library

[42]

M. Shanahan, Perception as abduction: turning sensor data into meaningful representation, Cogn. Sci., 29 (2005) 103-134.

[43]

R. Simmons, R. Goodwin, K.Z. Haigh, S. Koenig, J. O'Sullivan, A layered architecture for office delivery robots, in: Proceedings of the First International Conference on Autonomous Agents, Marina del Rey, USA, 1997, pp. 245-252.

Digital Library

[44]

K. Sj, A. Aydemir, D. Schlyter, P. Jensfelt, Topological spatial relations for active visual search, Centre for Autonomous Systems, KTH, Stockholm, July 2010.

[45]

K. Sj, H. Zender, P. Jensfelt, G.-J.M. Kruijff, A. Pronobis, N. Hawes, M. Brenner, The Explorer system, in: Cognitive Systems, Springer, Berlin, 2010, pp. 395-421.

[46]

D. Skoaj, M. Kristan, A. Vreko, M. Mahni, M. Janek, G.-J.M. Kruijff, M. Hanheide, N. Hawes, T. Keller, M. Zillich, K. Zhou, A system for interactive learning in dialogue with a tutor, in: Proceedings of the 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2011, pp. 3387-3394.

[47]

S. Sohrabi, J.A. Baier, S.A. McIlraith, Preferred explanations: theory and generation via planning, in: Proceedings of the 25th AAAI Conference on Artificial Intelligence, 2011, pp. 261-267.

Digital Library

[48]

I.H. Suh, G.H. Lim, W. Hwang, H. Suh, J.-H. Choi, Y.-T. Park, Ontology-based multi-layered robot knowledge framework (OMRKF) for robot intelligence, in: Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2007, pp. 429-436.

[49]

K. Talamadupula, J. Benton, S. Kambhampati, P. Schermerhorn, M. Scheutz, Planning for humanrobot teaming in open worlds, ACM Trans. Intell. Syst. Technol., 1 (December 2010) 14:1-14:24.

Digital Library

[50]

M. Tenorth, M. Beetz, KNOWROBknowledge processing for autonomous personal robots, in: Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009, pp. 4261-4266.

Digital Library

[51]

J.J. Velez, A.S. Huang, G.A. Hemann, N. Roy, I. Posner, Modelling observation correlations for active exploration and robust object detection, J. Artif. Intell. Res., 44 (2012) 423-453.

[52]

J.L. Wyatt, A. Aydemir, M. Brenner, M. Hanheide, N. Hawes, P. Jensfelt, M. Kristan, G.-J.M. Kruijff, P. Lison, A. Pronobis, K. Sj, D. Skoaj, A. Vreko, H. Zender, M. Zillich, Self-understanding and self-extension: a systems and representational approach, IEEE Trans. Auton. Ment. Dev., 2 (December 2010) 282-303.

Digital Library

[53]

H.L.S. Younes, M. Littman, PPDDL 1.0: an extension to PDDL for expressing planning domains with probabilistic effects, School of Computer Science, Carnegie Mellon University, Pittsburgh, Pennsylvania, 2004.

[54]

H. Zender, G.-J.M. Kruijff, I. Kruijff-Korbayov, Situated resolution and generation of spatial referring expressions for robotic assistants, in: Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence, 2009, pp. 1604-1609.

Digital Library

[55]

H. Zender, O.M. Mozos, P. Jensfelt, G.-J.M. Kruijff, W. Burgard, Conceptual spatial representations for indoor mobile robots, Robot. Auton. Syst., 56 (2008) 493-502.

Digital Library

[56]

S. Zhang, M. Sridharan, J. Wyatt, Integrating probabilistic graphical models and non-monotonic logical inference for robots, IEEE Trans. Robot., 31 (2015) 699-713.

Digital Library

Cited By

Lee CPraveena PMutlu B(2024)REX: Designing User-centered Repair and Explanations to Address Robot FailuresProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661559(2911-2925)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661559
Pan TShome RKavraki L(2024)Task and Motion Planning for Execution in the RealIEEE Transactions on Robotics10.1109/TRO.2024.341855040(3356-3371)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TRO.2024.3418550
Cui CAmiri SDing YZhan XZhang SEvans RShpitser I(2023)Learning to reason about contextual knowledge for planning under uncertaintyProceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence10.5555/3625834.3625878(465-475)Online publication date: 31-Jul-2023
https://dl.acm.org/doi/10.5555/3625834.3625878
Show More Cited By

Robot task planning and explanation in open and uncertain worlds
1. Computing methodologies
  1. Artificial intelligence
    1. Knowledge representation and reasoning
    2. Planning and scheduling

Recommendations

Robot task planning using semantic maps

Task planning for mobile robots usually relies solely on spatial information and on shallow domain knowledge, such as labels attached to objects and places. Although spatial information is necessary for performing basic robot operations (navigation and ...
Multiple Mobile Robot Task and Motion Planning: A Survey
With recent advances in mobile robotics, autonomous systems, and artificial intelligence, there is a growing expectation that robots are able to solve complex problems. Many of these problems require multiple robots working cooperatively in a multi-robot ...
Planning for human-robot teaming in open worlds

As the number of applications for human-robot teaming continue to rise, there is an increasing need for planning technologies that can guide robots in such teaming scenarios. In this article, we focus on adapting planning technology to Urban Search And ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Artificial Intelligence

Artificial Intelligence Volume 247, Issue C

June 2017

176 pages

ISSN:0004-3702

Issue’s Table of Contents

Copyright © Elsevier B.V.

Publisher

Elsevier Science Publishers Ltd.

United Kingdom

Publication History

Published: 01 June 2017

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lee CPraveena PMutlu B(2024)REX: Designing User-centered Repair and Explanations to Address Robot FailuresProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661559(2911-2925)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661559
Pan TShome RKavraki L(2024)Task and Motion Planning for Execution in the RealIEEE Transactions on Robotics10.1109/TRO.2024.341855040(3356-3371)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TRO.2024.3418550
Cui CAmiri SDing YZhan XZhang SEvans RShpitser I(2023)Learning to reason about contextual knowledge for planning under uncertaintyProceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence10.5555/3625834.3625878(465-475)Online publication date: 31-Jul-2023
https://dl.acm.org/doi/10.5555/3625834.3625878
Ding YZhang XAmiri SCao NYang HKaminski AEsselink CZhang S(2023)Integrating action knowledge and LLMs for task planning and situation handling in open worldsAutonomous Robots10.1007/s10514-023-10133-547:8(981-997)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1007/s10514-023-10133-5
Dazeley RVamplew PCruz F(2023)Explainable reinforcement learning for broad-XAI: a conceptual framework and surveyNeural Computing and Applications10.1007/s00521-023-08423-135:23(16893-16916)Online publication date: 6-Mar-2023
https://dl.acm.org/doi/10.1007/s00521-023-08423-1
Mu ZZhao WYin YXi XSong WGu JZhu S(2023)KGGPT: Empowering Robots with OpenAI’s ChatGPT and Knowledge GraphIntelligent Robotics and Applications10.1007/978-981-99-6495-6_29(340-351)Online publication date: 5-Jul-2023
https://dl.acm.org/doi/10.1007/978-981-99-6495-6_29
Bayreuther SJacob FGrotz MKartmann RPeller-Konrad FPaus FHartenstein HAsfour TDietrich SChowdhury OTakabi D(2022)BlueSky: Combining Task Planning and Activity-Centric Access Control for Assistive Humanoid RobotsProceedings of the 27th ACM on Symposium on Access Control Models and Technologies10.1145/3532105.3535018(185-194)Online publication date: 7-Jun-2022
https://dl.acm.org/doi/10.1145/3532105.3535018
Boyd K(2022)Designing Up with Value-Sensitive Design: Building a Field Guide for Ethical ML DevelopmentProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency10.1145/3531146.3534626(2069-2082)Online publication date: 21-Jun-2022
https://dl.acm.org/doi/10.1145/3531146.3534626
Jovan FTomy MHawes NWyatt J(2022)Efficiently exploring for human robot interaction: partially observable Poisson processesAutonomous Robots10.1007/s10514-022-10070-947:1(121-138)Online publication date: 28-Oct-2022
https://dl.acm.org/doi/10.1007/s10514-022-10070-9
Xi XZhu S(2022)A comprehensive review of task understanding of command-triggered execution of tasks for service robotsArtificial Intelligence Review10.1007/s10462-022-10347-656:7(7137-7193)Online publication date: 6-Dec-2022
https://dl.acm.org/doi/10.1007/s10462-022-10347-6
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents