More Web Proxy on the site http://driver.im/

article

A probabilistic argumentation framework for reinforcement learning agents

Authors:

Régis Riveret,

Guido Governatori,

Antonino Rotolo,

Giovanni SartorAuthors Info & Claims

Autonomous Agents and Multi-Agent Systems, Volume 33, Issue 1-2

Pages 216 - 274

https://doi.org/10.1007/s10458-019-09404-2

Published: 01 March 2019 Publication History

Abstract

A bounded-reasoning agent may face two dimensions of uncertainty: firstly, the uncertainty arising from partial information and conflicting reasons, and secondly, the uncertainty arising from the stochastic nature of its actions and the environment. This paper attempts to address both dimensions within a single unified framework, by bringing together probabilistic argumentation and reinforcement learning. We show how a probabilistic rule-based argumentation framework can capture Markov decision processes and reinforcement learning agents; and how the framework allows us to characterise agents and their argument-based motivations from both a logic-based perspective and a probabilistic perspective. We advocate and illustrate the use of our approach to capture models of agency and norms, and argue that, in addition to providing a novel method for investigating agent types, the unified framework offers a sound basis for taking a mentalistic approach to agent profiles.

References

[1]

Alexy, R. (1989). A theory of legal argumentation: The theory of rational discourse as theory of legal justification. Oxford: Clarendon.

[2]

Amgoud, L. (2009). Argumentation for decision making. In Argumentation in artificial intelligence (pp. 301---320). Springer.

[3]

Artikis, A., Sergot, M., & Pitt, J. (2009). Specifying norm-governed computational societies. ACM Transactions on Computational Logic, 10(1), 1:1---1:42.

Digital Library

[4]

Artikis, A., Sergot, M., Pitt, J., Busquets, D., & Riveret, R. (2016). Specifying and executing open multi-agent systems. In Social coordination frameworks for social technical systems (pp. 197---212). Springer.

[5]

Atkinson, K., Baroni, P., Giacomin, M., Hunter, A., Prakken, H., Reed, C., et al. (2017). Towards artificial argumentation. AI Magazine, 38(3), 25---36.

Digital Library

[6]

Atkinson, K., & Bench-Capon, T. J. M. (2007). Practical reasoning as presumptive argumentation using action based alternating transition systems. Artificial Intellignence, 171(10---15), 855---874.

Digital Library

[7]

Baroni, P., Caminada, M., & Giacomin, M. (2011). An introduction to argumentation semantics. The Knowledge Engineering Review, 26(4), 365---410.

Digital Library

[8]

Baroni, P., Governatori, G., & Riveret, R. (2016). On labelling statements in multi-labelling argumentation. In Proceedings of the 22nd European conference on artificial intelligence (Vol. 285, pp. 489---497). IOS Press.

Digital Library

[9]

Bellman, R. (1956). Dynamic programming and Lagrange multipliers. Proceedings of the National Academy of Sciences of the United States of America, 42(10), 767.

[10]

Bench-Capon, T. J. M., & Atkinson, K. (2009). Abstract argumentation and values. In L. Rahwan & G. Simari (eds.) Argumentation in artificial intelligence. Springer.

[11]

Bertsekas, D. P. (1995). Dynamic programming and optimal control (Vol. 1). Belmont, MA: Athena Scientific.

Digital Library

[12]

Besnard, P., García, A. J., Hunter, A., Modgil, S., Prakken, H., Simari, G. R., et al. (2014). Introduction to structured argumentation. Argument & Computation, 5(1), 1---4.

[13]

Broersen, J., Dastani, M., Hulstijn, J., & van der Torre, L. (2002). Goal generation in the BOID architecture. Cognitive Science Quarterly, 2(3---4), 428---447.

[14]

Chen, S. H., & Huang, Y. C. (2005). Risk preference and survival dynamics. In: Agent-based simulation: From modeling methodologies to real-world applications, Agent-based social systems (Vol. 1, pp. 135---143). Tokyo: Springer.

[15]

Conte, R., & Castelfranchi, C. (1995). Cognitive and social action. London: University College of London Press.

[16]

Conte, R., & Castelfranchi, C. (2006). The mental path of norms. Ratio Juris, 19, 501---517.

[17]

Conte, R., Falcone, R., & Sartor, G. (1999). Introduction: Agents and norms: How to fill the gap? Artificial Intelligence and Law, 7(1), 1---15.

Digital Library

[18]

Cormen, T. H., Leiserson, C. E., Rivest, R. L., Stein, C., et al. (2001). Introduction to algorithms (Vol. 2). Cambridge: MIT press.

Digital Library

[19]

Dung, P. M. (1995). On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games. Artificial Intelligence, 77(2), 321---358.

Digital Library

[20]

Edmonds, B. (2004). How formal logic can fail to be useful for modelling or designing mas. In Regulated agent-based social systems, Lecture Notes in Computer Science (Vol. 2934, pp. 1---15). Springer.

[21]

Fasli, M. (2004). Formal systems and agent-based social simulation equals null? Journal of Artificial Societies and Social Simulation, 7(4), 1---7.

[22]

Fornara, N., & Colombetti, M. (2009). Specifying and enforcing norms in artificial institutions. In Declarative agent languages and technologies VI, Lecture Notes in Computer Science (Vol. 5397, pp. 1---17). Springer.

Digital Library

[23]

Fox, J., & Parsons, S. (1997). On using arguments for reasoning about actions and values. In Proceedings of the AAAI spring symposium on qualitative preferences in deliberation and practical reasoning.

[24]

Gao, Y., & Toni, F. (2014). Argumentation accelerated reinforcement learning for cooperativeulti-agent systems. In Proceedings of 21st European conference on artificial intelligence (pp. 333---338). IOS Press.

Digital Library

[25]

Gao, Y., Toni, F., & Craven, R. (2012). Argumentation-based reinforcement learning for robocup soccer keepaway. In Proceedings of 20th European conference on artificial intelligence (pp. 342---347). IOS Press.

Digital Library

[26]

Gaudou, B., Lorini, E., & Mayor, E. (2013). Moral guilt: An agent-based model analysis. In Advances in social simulation--Proceedings of the 9th conference of the european social simulation association (pp. 95---106).

[27]

Governatori, G., & Rotolo, A. (2008). BIO logical agents: Norms, beliefs, intentions in defeasible logic. Autonomous Agents and Multi-Agent Systems, 17(1), 36---69.

Digital Library

[28]

Hunter, A., & Thimm, M. (2017). Probabilistic reasoning with abstract argumentation frameworks. Journal of Artificial Intelligence Research, 59, 565---611.

Digital Library

[29]

Koller, D., & Friedman, N. (2009). Probabilistic graphical models: Principles and techniques--Adaptive computation and machine learning. Cambridge: The MIT Press.

Digital Library

[30]

Kostrikin, A. I., Manin, Y. I., & Alferieff, M. E. (1997). Linear algebra and geometry. Washington, DC: Gordon and Breach Science Publishers.

[31]

Modgil, S., & Caminada, M. (2009). Proof theories and algorithms for abstract argumentation frameworks. In Argumentation in artificial intelligence (pp. 105---129). Springer.

[32]

Muller, J., & Hunter, A. (2012). An argumentation-based approach for decision making. In 24th international conference on tools with artificial intelligence (Vol. 1, pp. 564---571). IEEE.

Digital Library

[33]

Ng, A., Harada, D., & Russell, S. (1999). Policy invariance under reward transformations: theory and application to reward shaping. In Proceedings of 16th international conference on machine learning (pp. 278---287).

Digital Library

[34]

Ng, A. Y., Coates, A., Diel, M., Ganapathi, V., Schulte, J., Tse, B., Berger, E., & Liang, E. (2006). Autonomous inverted helicopter flight via reinforcement learning. In Experimental robotics IX (pp. 363---372). Springer.

[35]

Oren, N. (2014). Argument schemes for normative practical reasoning (pp. 63---78). Berlin: Springer.

[36]

Parsons, S., & Fox, J. (1996). Argumentation and decision making: A position paper. In Practical reasoning (pp. 705---709). Springer.

Digital Library

[37]

Pattaro, E. (2005). The law and the right. In E. Pattaro (Ed.), Treatise of legal philosophy and general jurisprudence (Vol. 1). Berlin: Springer.

[38]

Pollock, J. L. (1995). Cognitive carpentry: A blueprint for how to build a person. Cambridge, MA: MIT Press.

Digital Library

[39]

Prakken, H. (2006). Combining sceptical epistemic reasoning with credulous practical reasoning. In Proceedings of the 1st conference on computational models of argument (pp. 311---322). IOS Press.

Digital Library

[40]

Prakken, H. (2011). An abstract framework for argumentation with structured arguments. Argument and Computation, 1(2), 93---124.

[41]

Prakken, H., & Sartor, G. (1997). Argument-based extended logic programming with defeasible priorities. Journal of Applied Non-Classical Logics, 7(1---2), 25---75.

[42]

Prakken, H., & Sartor, G. (2015). Law and logic: A review from an argumentation perspective. Artificial Intelligence, 227, 214---245.

Digital Library

[43]

Rahwan, I., & Simari, G. R. (Eds.). (2009). Argumentation in artificial Intelligence. Berlin: Springer.

Digital Library

[44]

Riveret, R., Baroni, P., Gao, Y., Governatori, G., Rotolo, A., & Sartor, G. (2018). A labelling framework for probabilistic argumentation. Annals of Mathamatics and Artificial Intelligence, 83(1), 21---71.

Digital Library

[45]

Riveret, R., Korkinof, D., Draief, M., & Pitt, J. V. (2015). Probabilistic abstract argumentation: An investigation with boltzmann machines. Argumentation & Computation, 6(2), 178---218.

[46]

Riveret, R., Pitt, J. V., Korkinof, D., & Draief, M. (2015). Neuro-symbolic agents: Boltzmann machines and probabilistic abstract argumentation with sub-arguments. In Proceedings of the 14th international conference on autonomous agents and multiagent systems (pp. 1481---1489). ACM.

Digital Library

[47]

Riveret, R., Rotolo, A., & Sartor, G. (2012). Probabilistic rule-based argumentation for norm-governed learning agents. Artificial Intelligence and Law, 20(4), 383---420.

Digital Library

[48]

Ross, A. (1958). On law and justice. London: Stevens.

[49]

Rummery, G. A., & Niranjan, M. (1994). On-line Q-learning using connectionist systems. Technical report. University of Cambridge.

[50]

Sartor, G. (2005). Legal reasoning: A cognitive approach to the law. Berlin: Springer.

[51]

Shams, Z., Vos, M. D., Oren, N., Padget, J., & Satoh, K. (2015). Argumentation-based normative practical reasoning. In Proceedings of the 3rd international workshop on theory and applications of formal argumentation, revised selected papers (pp. 226---242). Springer.

[52]

Simari, G. I., Shakarian, P., & Falappa, M. A. (2016). A quantitative approach to belief revision in structured probabilistic argumentation. Annals of Mathematics and Artificial Intelligence, 76(3), 375---408.

Digital Library

[53]

Stone, P., Sutton, R. S., & Kuhlmann, G. (2005). Reinforcement learning for robocup soccer keepaway. Adaptive Behavior, 13, 165---188.

[54]

Sutton, R. S., & Barto, A. (1998). Reinforcement learning: An introduction. Cambridge: MIT Press.

Digital Library

[55]

Tadepalli, P., Givan, R., & Driessens, K. (2004). Relational reinforcement learning: An overview. In Proceedings of the ICML04 workshop on relational reinforcement learning.

[56]

van der Hoek, W., Roberts, M., & Wooldridge, M. (2007). Social laws in alternating time: Effectiveness, feasibility, and synthesis. Synthese, 156(1), 1---19.

Cited By

Krejcar OKadir MSelamat A(2024)Norm Augmented Reinforcement Learning Agents With Synthesized Normative RulesJournal of Cases on Information Technology10.4018/JCIT.34565026:1(1-34)Online publication date: 30-Jul-2024
https://dl.acm.org/doi/10.4018/JCIT.345650
Aydoğan RKafali ÖArslan FJonker CSingh M(2021) Nova: Value-based Negotiation of NormsACM Transactions on Intelligent Systems and Technology10.1145/346505412:4(1-29)Online publication date: 1-Aug-2021
https://dl.acm.org/doi/10.1145/3465054
Neugebauer DRothe JSkiba K(2021)Complexity of Nonemptiness in Control Argumentation FrameworksSymbolic and Quantitative Approaches to Reasoning with Uncertainty10.1007/978-3-030-86772-0_9(117-129)Online publication date: 21-Sep-2021
https://dl.acm.org/doi/10.1007/978-3-030-86772-0_9

A probabilistic argumentation framework for reinforcement learning agents
1. Computing methodologies
  1. Artificial intelligence
    1. Distributed artificial intelligence
2. Theory of computation
  1. Models of computation

Recommendations

A labelling framework for probabilistic argumentation

The combination of argumentation and probability paves the way to new accounts of qualitative and quantitative uncertainty, thereby offering new theoretical and applicative opportunities. Due to a variety of interests, probabilistic argumentation is ...
Probabilistic rule-based argumentation for norm-governed learning agents

This paper proposes an approach to investigate norm-governed learning agents which combines a logic-based formalism with an equation-based counterpart. This dual formalism enables us to describe the reasoning of such agents and their interactions using ...
Norm Augmented Reinforcement Learning Agents With Synthesized Normative Rules: A Proposed Normative Agent Framework

The dynamic deontic (DD) is a norm synthesis framework that extracts normative rules from reinforcement learning (RL), however it was not designed to be applied in agent coordination. This study proposes a norm augmented reinforcement learning framework (...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Autonomous Agents and Multi-Agent Systems

Autonomous Agents and Multi-Agent Systems Volume 33, Issue 1-2

March 2019

274 pages

ISSN:1387-2532

Issue’s Table of Contents

Copyright © Copyright © 2019 The Author(s).

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 March 2019

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Krejcar OKadir MSelamat A(2024)Norm Augmented Reinforcement Learning Agents With Synthesized Normative RulesJournal of Cases on Information Technology10.4018/JCIT.34565026:1(1-34)Online publication date: 30-Jul-2024
https://dl.acm.org/doi/10.4018/JCIT.345650
Aydoğan RKafali ÖArslan FJonker CSingh M(2021) Nova: Value-based Negotiation of NormsACM Transactions on Intelligent Systems and Technology10.1145/346505412:4(1-29)Online publication date: 1-Aug-2021
https://dl.acm.org/doi/10.1145/3465054
Neugebauer DRothe JSkiba K(2021)Complexity of Nonemptiness in Control Argumentation FrameworksSymbolic and Quantitative Approaches to Reasoning with Uncertainty10.1007/978-3-030-86772-0_9(117-129)Online publication date: 21-Sep-2021
https://dl.acm.org/doi/10.1007/978-3-030-86772-0_9

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents