[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content

Advertisement

Springer Nature Link
Log in

Beyond Reinforcement Learning and Local View in Multiagent Systems

  • Technical Contribution
  • Published:
KI - Künstliche Intelligenz Aims and scope Submit manuscript

Abstract

Learning is an important component of an agent’s decision making process. Despite many messages in contrary, the fact is that, currently, in the multiagent community it is mostly likely that learning means reinforcement learning. Given this background, this paper has two aims: to revisit the “old days” motivations for multiagent learning, and to describe some of the work addressing the frontiers of multiagent systems and machine learning. The intention of the latter task is to try to motivate people to address the issues that are involved in the application of techniques from multiagent systems in machine learning and vice-versa.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. Henceforth, with some abuse, I use the term ML to designate supervised and unsupervised ML techniques. I do so because this is the view taken by most of the computer science community, not to mention other communities.

  2. It should be noted that back in 2000, game theoretic approaches were not necessarily combined with RL.

  3. In this paper, details about RL and MARL, as well as Markov decision processes, stochastic games, and Q-learning are omitted. The reader is referred to[10, 27, 40, 60].

References

  1. Agogino A, Tumer K (2006) Efficient agent-based cluster ensembles. In: Stone P, Weiss G (eds) AAMAS ’06: Proceedings of the 5th International Joint Conference on Autonomous agents and Multiagent Systems. ACM, New York, pp 1079–1086

    Chapter  Google Scholar 

  2. Alam S, Dobbie G, Riddle P (2009) Exploiting swarm behaviour of simple agents for clustering web users’ session data. In: Cao L (ed) Data mining and multi-agent integration, pp 61–75. Springer, US. doi:10.1007/978-1-4419-0522-2-4

  3. Babu T, Murty M, Subrahmanya S (2009) Multiagent systems for large data clustering. In: Cao L (ed) Data mining and multi-agent integration. pp 219–238. Springer, US. doi:10.1007/978-1-4419-0522-2-15

  4. Bazzan ALC (2009) Agents and data mining in bioinformatics: joining data gathering and automatic annotation with classification and distributed clustering. In: Cao L (ed) Proceedings of the Workshop on Agents and Data Mining Interaction, no. 5680 in Lecture Notes in Artificial Intelligence, pp 3–20. Springer, Berlin. URL http://www.inf.ufrgs.br/bazzan/downloads/5680_3.pdf

  5. Bazzan ALC (2013) Cooperative induction of decision trees. In: 2013 IEEE Symposium on Intelligent Agent (IA) pp 62–69. doi:10.1109/IA.2013.6595190

  6. Bekkerman R, Zilberstein S, Allan J (2007) Web page clustering using heuristic search in the web graph. In: Veloso MM (ed) IJCAI, pp 2280–2285

  7. Brahmi I, Yahia S, Aouadi H, Poncelet P (2012) Towards a multiagent-based distributed intrusion detection system using data mining approaches. In: Cao L, Bazzan AL, Symeonidis AL, Gorodetsky VI, Weiss G, Yu PS (eds) Agents and data mining interaction, Lecture Notes in Computer Science, vol 7103, pp 173–194. Springer, Berlin. doi:10.1007/978-3-642-27609-5-12

  8. Brazdil P, Gams M, Sian SS, Torgo L, de Velde WV (1991) Panel: learning in distributed systems and multi-agent environments. In: Kodratoff Y (ed) European Working Session on Learning (EWSL-91), Lecture Notes in Computer Science, vol. 482, pp 412–423. Springer. URL http://dblp.uni-trier.de/db/conf/ecml/ewsl91.html#BrazdilGSTV91

  9. Brazdil P, Muggleton S (2991) Learning to relate terms in a multiple agent environment. In: Kodratoff Y (ed) European Working Session on Learning (EWSL-91), Lecture Notes in Computer Science, vol 482, pp 424–439. Springer, Berlin. doi:10.1007/BFb0017035

  10. Buşoniu L, Babuska R, De Schutter B (2008) A comprehensive survey of multiagent reinforcement learning. IEEE Trans Syst Man Cybern Part C Appl Rev 38(2):156–172

    Article  Google Scholar 

  11. Cao L (2001) Data mining and multi-agent integration. Springer, Boston. doi:10.1007/978-1-4419-0522-2

  12. Cao L, Weiss G, Yu PS (2012) A brief introduction to agent mining. Auton Agents Multi Agent Syst 25(3):419–424

    Article  Google Scholar 

  13. Cao L, Luo C, Zhang C (2007) Agent-mining interaction: an emerging area. In: Gorodetsky V, Zhang C, Skormin VA, Cao L (eds) AIS-ADM, Lecture Notes in Computer Science, vol 4476, pp 60–73. Springer, Berlin. doi:10.1007/978-3-540-72839-9-5

  14. Caragea D, Silvescu A, Honavar V (2001) Analysis and synthesis of agents that learn from distributed dynamic data sources. In: Wermter S, Austin J, Willshaw D (eds) Emergent neural computational architectures based on neuroscience, Lecture Notes in Computer Science, vol 2036. Springer, Berlin, pp 547–559. doi:10.1007/3-540-44597-8-39

  15. Cetnarowicz K, Kisiel-Dorohinicki M, Nawarecki E (1996) The application of evolution process in multi-agent world to the prediction system. In: Tokoro M (ed) Proceedings of the International Conference on Multiagent Systems (ICMAS 96). AAAI Press, Menlo Park, pp 26–32

    Google Scholar 

  16. Chaimontree S, Atkinson K, Coenen F (2010) Clustering in a multi-agent data mining environment. In: Cao L, Bazzan AL, Gorodetsky V, Mitkas PA, Weiss G, Yu PS (eds) Agents and data mining interaction, Lecture Notes in Computer Science, vol 5980, pp 103–114. Springer, Berlin. doi:10.1007/978-3-642-15420-1-9

  17. Chaimontree S, Atkinson K, Coenen F (2012) A framework for multi-agent based clustering. Auton Agents Multi Agent Syst 25(3):425–446. doi:10.1007/s10458-011-9187-0

    Article  Google Scholar 

  18. Chernova S, Veloso MM (2007) Confidence-based policy learning from demonstration using gaussian mixture models. In: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems, pp 1310–1317

  19. da Silva JC, Giannella C, Bhargava R, Kargupta H, Klusch M (2005) Distributed data mining and agents. Eng Appl Artif Intell 18(7):791–807. doi:10.1016/j.engappai.2005.06.004

    Article  Google Scholar 

  20. Davies W, Edwards P (1995) Distributed learning: an agent-based approach to data-mining. In: Proceedings of Machine Learning Workshop on Agents that Learn from Other Agents

  21. dos Santos DS, de Oliveira D, Bazzan ALC (2009) A multiagent, multiobjective clustering algorithm. In:Cao L (ed) Data mining and multiagent integration. Springer, Berlin. URL http://link.springer.com/chapter/10.1007

  22. dos Santos DS, Bazzan ALC (2012) Distributed clustering for group formation and task allocation in multiagent systems: a swarm intelligence approach. Appl Soft Comput 12(8):2123–2131. doi:10.1016/j.asoc.2012.03.016

    Article  Google Scholar 

  23. Emele C, Norman T, Şensoy M, Parsons S (2012) Learning strategies for task delegation in norm-governed environments. Auton Agents Multi Agent Syst 25(3):499–525. doi:10.1007/s10458-012-9194-9

    Article  Google Scholar 

  24. Fiosins M, Fiosina J, Müller JP (2102) Change point analysis for intelligent agents in city traffic. In: Cao L, Bazzan AL, Symeonidis AL, Gorodetsky VI, Weiss G, Yu PS (eds) Agents and data mining interaction, Lecture Notes in Computer Science, vol 7103. Springer, Berlin, pp 195–210. doi:10.1007/978-3-642-27609-5-13

  25. Garruzzo S, Rosaci D (2008) Agent clustering based on semantic negotiation. ACM Trans Auton Adapt Syst 3(2):7:1–7:40. doi:10.1145/1352789.1352792

    Article  Google Scholar 

  26. Hindriks KV, Tykhonov D (2008) Opponent modelling in automated multi-issue negotiation using bayesian learning. In: Proceedings of the 7th Internatinal Joint Conference on Autonomous Agents and Multiagent Systems, pp 331–338

  27. Kaelbling LP, Littman M, Moore A (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285

    Google Scholar 

  28. Kao Y, Cheng K (2006) An ACO-based clustering algorithm. In: Proceedings of the Fifth International Workshop on Ant Colony Optimization and Swarm Intelligence—ANTS 2006, vol 4150, Lecture Notes in Computer Science. Springer, Brussels, pp 340–347

  29. Kargupta H, Hamzaoglu I, Stafford B, Stafford B (1997) Scalable, distributed data mining using an agent gased architecture. In: Proceedings of the Third International Conference on the Knowledge Discovery and Data Mining. AAAI Press, Menlo Park, California, USA, pp 211–214

  30. Kargupta H, Park B, Hershberger D, Johnson E (1999) Collective data mining: a new perspective toward distributed data mining. In: Advances in Distributed and Parallel Knowledge Discovery, vol 2. MIT Press, Cambridge, pp 131–174

  31. Kiekintveld C, Miller J, Jordan PR, Wellman MP (2001) Forecasting market prices in a supply chain game. In: 6th International Joint Conference on Autonomous Agents and Multiagent Systems, pp 1318–1325 (2007)

  32. Klusch M, Lodi S, Moro G (2003) Issues of agent-based distributed data mining. In: Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems. ACM, New York, NY, USA, pp 1034–1035. doi:10.1145/860575.860782

  33. Lemmens N, Tuyls K (2009) Stigmergic landmark foraging. In: Proceedings of the 8th International Joint Conference on Autonomous Agents and Multiagent Systems, pp 497–504

  34. Lumer ED, Faieta B (1994) Diversity and adaptation in populations of clustering ants. In: Proceedings of the third international conference on Simulation of adaptive behavior: from animals to animats. MIT Press, Cambridge, MA, USA, pp 501–508

  35. Mendoza MR, Bazzan ALC (submitted) Social choice in distributed classification tasks: dealing with vertically partitioned data. http://www.inf.ufrgs.br/maslab/pergamus/pubs/preprint_INS_bioinfovoting.pdf

  36. Modi PJ, Shen WM (2001) Collaborative multiagent learning for classification tasks. In: Proceedings of the 5th International Conference on Autonomous Agents, AGENTS ’01. ACM, New York, NY, USA, pp 37–38. doi:10.1145/375735.375854

  37. Nunes L, Oliveira EC (2004) Learning from multiple sources. In: Jennings N, Sierra C, Sonenberg L, Tambe M (eds) Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multi Agent Systems, AAMAS, vol 3. IEEE Computer Society, New York, pp 1106–1113

    Google Scholar 

  38. Ogston E, Overeinder B, van Steen M, Brazier F (2003) A method for decentralized clustering in large multi-agent systems. In: Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS ’03). ACM, New York pp 789–796. doi:10.1145/860575.860702

  39. Palou XR, Rovatsos M (2009) Collaborative agent-based learning with limited data exchange. In: Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems (AAMAS ’09). IFAAMAS, Richland, pp 1191–1192. URL http://dl.acm.org/citation.cfm?id=1558109.1558207

  40. Pan R, Dolog P, Xu G (2013) KNN-based clustering for improving social recommender systems. In: Cao L, Zeng Y, Symeonidis AL, Gorodetsky VI, Yu PS , Singh MP (eds) Agents and data mining interaction, Lecture Notes in Computer Science, vol 7607. Springer, Berlin, pp 115–125. doi:10.1007/978-3-642-36288-0-11

  41. Panait L, Luke S (2005) Cooperative multi-agent learning: the state of the art. Auton Agents Multi Agent Syst 11(3):387–434

    Article  Google Scholar 

  42. Peteiro-Barral D, Guijarro-Berdiñas B (2013) A survey of methods for distributed machine learning. Progress in AI 2(1):1–11

    Google Scholar 

  43. Piraveenan M, Prokopenko M, Wang P, Zeman A (2008) Decentralized multi-agent clustering in scale-free sensor networks. In: Fulcher J, Jain LC (eds) Computational intelligence: a compendium, studies in computational intelligence, vol 115. Springer, berlin, pp 485–515. doi:10.1007/978-3-540-78293-3-12

  44. Provost FJ, Hennessy DN (1996) Scaling up: distributed machine learning with cooperation. In: Proceedings of the Thirteenth National Conference on Artificial Intelligence. AAAI Press, Menlo park, pp 74–79

  45. Provost F, Kolluri V (1999) A survey of methods for scaling up inductive algorithms. Data Min Knowl Discov 3(2):131–169

    Article  Google Scholar 

  46. Riley P, Veloso M (2000) On behavior classification in adversarial environments. In: Parker LE, Bekey G, Barhen J (eds) Distributed Autonomous Robotic Systems 4. Springer, Berlin, pp 371–380

    Chapter  Google Scholar 

  47. Riley P, Veloso M, Kaminka G (2002) An empirical study of coaching. In: Asama H, Arai T, Fukuda T, Hasegawa T (eds) Distributed Autonomous Robotic Systems 5. Springer, Berlin, pp 215–224

    Chapter  Google Scholar 

  48. Santana LEA, Canuto AMP, Xavier Jr. JC, Campos AMC: A comparative analysis of data distribution methods in an agent-based neural system for classification tasks. In: HIS, p 9

  49. Santana LEO, Canuto AMP, Abreu MCC (2006) Analyzing the performance of an agent-based neural system for classification tasks using data distribution among the agents. In: IJCNN, pp 2951–2958

  50. Şensoy M, Yilmaz B, Norman T (2013) Discovering frequent patterns to bootstrap trust. In: Cao L, Zeng Y, Symeonidis A, Gorodetsky V, Yu P, Singh MP (eds) Agents and data mining interaction, Lecture Notes in Computer Science, vol 7607, Springer, Berlin, pp 93–104. doi:10.1007/978-3-642-36288-0-9

  51. Shelokar PS, Jayaraman VK, Kulkarni BD (2004) An ant colony approach for clustering. Anal Chim Acta 509(2):187–195

    Article  Google Scholar 

  52. Shoham Y, Powers R, Grenager T (2007) If multi-agent learning is the answer, what is the question? Artif Intell 171(7):365–377

    Article  MATH  MathSciNet  Google Scholar 

  53. Sian S (1991) Adaptation based on cooperative learning in multi-agent systems. In: Demazeau Y, Müller J (eds) Decentralized A.I., vol. 2. North-Holland, pp 257–272

  54. Stolfo S, Tselepis ALPS, Prodromidis AL, Tselepis S, Lee W, Fan DW, Chan PK (1997) JAM: Java agents for meta-learning over distributed databases. In: Proceedings of 3rd International Confence on Knowledge Discovery and Data Mining. AAAI Press, Malno park, pp 74–81

  55. Stone P, Veloso M (2000) Multiagent systems: a survey from a machine learning perspective. Auton Robots 8(3):345–383

    Article  Google Scholar 

  56. Stone P (2007a) Learning and multiagent reasoning for autonomous agents. In: The 20th International Joint Conference on Artificial Intelligence, pp 13–30

  57. Stone P (2007b) Multiagent learning is not the answer. It is the question. Artif Intell 171(7):402–405

    Article  MATH  Google Scholar 

  58. Sutton R, Barto A (1998) Reinforcement learning: an introduction. MIT Press, Cambridge

    Google Scholar 

  59. Tozicka J, Rovatsos M, Pechoucek M (2007) A framework for agent-based distributed machine learning and data mining. In: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS ’07), ACM, New York, NY, USA, pp 666–673. doi:10.1145/1329125.1329243

  60. Tuyls K, Weiss G (2012) Multiagent learning: basics, challenges, and prospects. AI Mag 33(3):41–52

    Google Scholar 

  61. Urban S, Jakob M, Pěchouček M (2010) Probabilistic modeling of mobile agents’ trajectories. In: Cao L, Bazzan AL, Gorodetsky V, Mitkas PA, Weiss G, Yu PS (eds) Agents and data mining interaction, Lecture Notes in Computer Science, vol. 5980, Springer, Berlin, pp 59–70. doi:10.1007/978-3-642-15420-1-6

  62. Wardeh M, Coenen F, Bench-Capon T (2012) Multi-agent based classification using argumentation from experience. Auton Agents Multi Agent Syst 25(3):447–474. doi:10.1007/s10458-012-9197-6

    Article  Google Scholar 

  63. Yang Y, Kamel M (2006) An aggregated clustering approach using multi-ant colonies algorithms. Pattern Recongn 39(7):1278–1289

    Article  MATH  Google Scholar 

  64. Zhang W, Wang Y (2012) Agent-based cluster analysis of tropical cyclone tracks in the western north pacific. In: Cao L, Bazzan A, Symeonidis A, Gorodetsky V, Weiss G, Yu P (eds) Agents and data mining interaction, Lecture Notes in Computer Science, vol. 7103, Springer, Berlin, pp 98–113. doi:10.1007/978-3-642-27609-5-8

Download references

Acknowledgments

Ana Bazzan is partially supported by CNPq.

Author information

Authors and Affiliations

  1. Instituto de Informática, UFRGS, Porto Alegre, Brazil

    Ana L. C. Bazzan

Authors
  1. Ana L. C. Bazzan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ana L. C. Bazzan.

Additional information

Ana Bazzan is partially supported by the Braz. Nac. Res. Council, CNPq.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bazzan, A.L.C. Beyond Reinforcement Learning and Local View in Multiagent Systems. Künstl Intell 28, 179–189 (2014). https://doi.org/10.1007/s13218-014-0312-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13218-014-0312-5

Keywords