Abstract
This paper presents strategies for speeding up the convergence of agents on swarm. Speeding up the learning of an agent is a complex task since the choice of inadequate updating techniques may cause delays in the learning process or even induce an unexpected acceleration that causes the agent to converge to a non-satisfactory policy. We have developed strategies for updating policies which combines local and global search using past policies. Experimental results in dynamic environments of different dimensions have shown that the proposed strategies are able to speed up the convergence of the agents while achieving optimal action policies, improving the coordination of agents in the swarm while deliberating.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wooldridge, M.J.: An Introduction to MultiAgent Systems. Wiley, Chichester (2002)
Ribeiro, R., Favarim F., Barbosa, M.A.C., Borges, A.P., Dordal, B.O., Koerich, A.L., Enembreck, F.: Unified algorithm to improve reinforcement learning in dynamic environments: an instance-based approach. In: 14th International Conference on Enterprise Information Systems (ICEIS’12), Wroclaw, Poland, pp. 229–238 (2012)
Mihaylov, M., Tuyls, K., Nowé, A.: Decentralized learning in wireless sensor networks. In: Taylor, M.E., Tuyls, K. (eds.) ALA 2009. LNCS, vol. 5924, pp. 60–73. Springer, Heidelberg (2010)
Chaharsooghi, S.K., Heydari, J., Zegordi, S.H.: A reinforcement learning model for supply chain ordering management: an application to the beer game. J. Decision Support Syst. 45(4), 949–959 (2008)
Dorigo, M.: Optimization, Learning and Natural Algorithms. Ph.D. thesis, Politecnico di Milano, Itália (1992)
Ribeiro, R., Enembreck, F.: A sociologically inspired heuristic for optimization algorithms: a case study on ant systems. expert systems with applications. Expert Syst. Appl. 40(5), 1814–1826 (2012)
Sudholt, D.: Theory of swarm intelligence. In: Proceedings of the 13th Annual Conference Companion on Genetic and Evolutionary Computation (GECCO ‘11), pp. 1381–1410. ACM, New York (2011)
Dorigo, M., Gambardella, L.M.: A study of some properties of Ant-Q. In: Proceedings of PPSN Fourth International Conference on Parallel Problem Solving From Nature, pp. 656–665 (1996)
Ribeiro, R., Borges, A.P., Enembreck, F.: Interaction models for multiagent reinforcement learning. In: International Conference on Computational Intelligence for Modelling Control and Automation - CIMCA08, Vienna, Austria, pp. 1–6 (2008)
Gambardella, L.M., Dorigo, M.: Ant-Q: a reinforcement learning approach to the TSP. In: Proceedings of ML-95, Twelfth International Conference on Machine Learning, pp. 252–260 (1995)
Reinelt, G.: TSPLIB - a traveling salesman problem library. ORSA J. Comput. 3, 376–384 (1991)
Dorigo, M., Maniezzo, V., Colorni, A.: Ant system: optimization by a colony of cooperting agents. IEEE Trans. Syst., Man, Cybern.-Part B 26(1), 29–41 (1996)
Watkins, C.J.C.H., Dayan, P.: Q-Learning. Mach. Learn. 8(3), 279–292 (1992)
Guntsch, M., Middendorf, M.: Applying population based ACO to dynamic optimization problems. In: Proceedings of Third International Workshop ANTS, pp. 111–122 (2003)
Sim, K.M., Sun, W.H.: Multiple ant-colony optimization for network routing. In: Proceedings of the First International Symposium on Cyber Worlds, pp. 277–281 (2002)
Li, Y., Gong, S.: Dynamic ant colony optimization for TSP. Int. J. Adv. Manuf. Technol. 22(7–8), 528–533 (2003)
Lee, S.G., Jung, T.U., Chung, T.C.: Improved ant agents system by the dynamic parameter decision. In Proceedings of the IEEE International Conference on Fuzzy Systems, pp. 666–669 (2001)
Gambardella, L.M., Taillard, E.D., Dorigo, M.: Ant colonies for the QAP. Technical report, IDSIA, Lugano, Switzerland (1997)
Stutzle, T., Hoos, H.: MAX-MIN Ant system and local search for the traveling salesman problem. In: Proceedings of the IEEE International Conference on Evolutionary Computation, pp. 309–314 (1997)
Guntsch, M., Middendorf, M.: Pheromone modification strategies for ant algorithms applied to dynamic TSP. In: Proceedings of the Workshop on Applications of Evolutionary Computing, pp. 213–222 (2001)
Christofides, N., Eilon, S.: Expected distances in distribution problems. Oper. Res. Q. 20, 437–443 (1969)
Tesauro, G.: Temporal difference learning and TD-Gammon. Commun. ACM 38(3), 58–68 (1995)
Enembreck, F., Ávila, B.C., Scalabrin, E.E., Barthes, J.P.: Distributed constraint optimization for scheduling in CSCWD. In: International Conference on Computer Supported Cooperative Work in Design, Santiago, vol. 1, pp. 252–257 (2009)
Hao, J., Leung, H.-F.: The dynamics of reinforcement social learning in cooperative multiagent systems. In: Proceedings of the 23rd. International Joint Conference on Artificial Intelligence (IJCAI’13), Beijing, China, pp. 184–190 (2013)
Kötzing, T., Frank, N., Röglin, H., Witt, C.: Theoretical analysis of two ACO approaches for the traveling salesman problem. Swarm Intell. 6(1), 1–21 (2012)
Brambilla, M., Ferrante, E., Birattari, M., Dorigo, M.: Swarm robotics: a review from the swarm engineering perspective. Swarm Intell. 7(1), 1–41 (2013)
Acknowledgements
This research is supported by the Program for Research Support of UTFPR - campus Pato Branco, DIRPPG (Directorate of Research and Post-Graduation) and Fundação Araucária (Araucaria Foundation of Parana State).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Ribeiro, R., Ronszcka, A.F., Barbosa, M.A.C., Enembreck, F. (2014). Coordinating Agents in Dynamic Environment. In: Hammoudi, S., Cordeiro, J., Maciaszek, L., Filipe, J. (eds) Enterprise Information Systems. ICEIS 2013. Lecture Notes in Business Information Processing, vol 190. Springer, Cham. https://doi.org/10.1007/978-3-319-09492-2_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-09492-2_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-09491-5
Online ISBN: 978-3-319-09492-2
eBook Packages: Computer ScienceComputer Science (R0)