Abstract
Reinforcement Learning is an optimal adaptive optimization method for stationary environments. For non-stationary environments where the transition function and reward structure change over time, the traditional algorithms seems to be ineffective in order to follow the environmental changes. In this paper we propose the Anomaly Detection Q-learning algorithm which increase learning abilities of standard Q-learning algorithm by applying Chauvenet’s criterion to detects anomalies.
This work is co-financed by European Union within European Social Fund.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Watkins, C.: Learning from Delayed Rewards. PhD thesis, University of Cambridge, England (1989)
Kaelbling, L.P., Littman, M.L., Moore, A.P.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)
Sutton, R., Barto, A., Williams, R.: Reinforcement learning is direct adaptive control. IEEE Control Systems Magazine, 19–22 (1992)
Doya, K.: Metalearning and neuromodulation. Neural Netw. 15(4), 495–506 (2002)
Murakoshi, K., Mizuno, J.: A parameter control method in reinforcement learning to rapidly follow unexpected environmental changes. Biosystems 77(1-3), 109–117 (2004)
Reinforcement Learning-based Control of Traffic Lights in Non-stationary Environments: A Case Study in a Microscopic Simulator. In: Dunin-Keplicz, B., Omicini, A., Padget, J.A. (eds.) Proceedings of the 4th European Workshop on Multi-Agent Systems EUMAS 2006, December 14-15. CEUR Workshop Proceedings, CEUR-WS.org, vol. 223 (2006)
Poprawski, R., Salejda, W.: Zasady opracowania wyników pomiarów. Oficyna Wydawnicza Politechniki Wrocławskiej (2009)
Littman, M.L., Dean, T.L., Kaelbling, L.P.: On the complexity of solving markov decision problems. In: Proc. of the Eleventh International Conference on Uncertainty in Artificial Intelligence, pp. 394–402 (1995)
Levy, H., Sidi, M.: Polling systems: Applications, modelling, and optimization. IEEE Trans. Commun. 38(10), 1750–1760 (1990)
Vishnevskii, V.M., Semenova, O.V.: Mathematical methods to study the polling systems. Automation and Remote Control 67(2), 173–220 (2006)
Documentation, C.I.: Quality of service solution guide, implementing diffserv for end-to-end quality of service, release 12.2, pp. 371–392 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Stanek, M., Kwasnicka, H. (2010). AdQL – Anomaly Detection Q-Learning in Control Multi-queue Systems with QoS Constraints. In: Jędrzejowicz, P., Nguyen, N.T., Howlet, R.J., Jain, L.C. (eds) Agent and Multi-Agent Systems: Technologies and Applications. KES-AMSTA 2010. Lecture Notes in Computer Science(), vol 6071. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13541-5_20
Download citation
DOI: https://doi.org/10.1007/978-3-642-13541-5_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13540-8
Online ISBN: 978-3-642-13541-5
eBook Packages: Computer ScienceComputer Science (R0)