Abstract
Recently Jaeger and others have put forth the paradigm of "reservoir computing" as a way of computing with highly recurrent neural networks. This reservoir is a collection of neurons randomly connected with each other of fixed weights. Amongst other things, it has been shown to be effective in temporal pattern recognition; and has been held as a model appropriate to explain how certain aspects of the brain work. (Particularly in its guise as “liquid state machine”, due to Maass et al.) In this work we show that although it is known that this model does have generalizability properties and thus is robust to errors in input, it is NOT resistant to errors in the model itself. Thus small malfunctions or distortions make previous training ineffective. Thus this model as currently presented cannot be thought of as appropriate as a biological model; and it also suggests limitations on the applicability in the pattern recognition sphere. However, we show that, with the enforcement of topological constraints on the reservoir, in particular that of small world topology, the model is indeed fault tolerant. Thus this implies that "natural" computational systems must have specific topologies and the uniform random connectivity is not appropriate.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Albert, R., Barabási, A.-L.: Topology of evolving networks: local events and universality, vol. 85, pp. 5234–5237 (2000)
Albert-László, B., Réka, A.: Emergence of Scaling in Random Networks. SCIENCE 286, 509–512 (1999)
Bianconi, G., Barabási, A.-L.: Competition and multiscaling in evolving networks. Europhys Lett. 54, 436 (2001)
Chklovskii, L.R.: Structural Properties of the Caenorhabditis elegans Neuronal Network (July 2009)
Danielle, B.S., Bullmore, E.: Small-world brain networks. Neuroscientist 12(6), 512–523 (2006)
Fernando, C., Sojakka, S.: Pattern Recognition in a Bucket. In: Banzhaf, W., Ziegler, J., Christaller, T., Dittrich, P., Kim, J.T. (eds.) ECAL 2003. LNCS (LNAI), vol. 2801, pp. 588–597. Springer, Heidelberg (2003)
Gütig, R., Sompolinsky, H.: The tempotron: a neuron that learns spike timing-based decision. Nature Neuroscience 9(3), 420–428 (2006)
Izhikevich, E.M.: Simple Model of Spiking Neurons. IEEE Transactions On Neural Networks 14(6), 1569–1572 (2003)
Jaeger, H.: "The ëcho state" approach to analysing and training recurrent neural networks. German National Research Institute for Computer Science (2001)
Jaeger, H.: The "echo state" approach to analysing and training recurrent neural networks. German National Research Institute for Computer Science (2001)
Lukosevicius, M., Jaeger, H.: Reservoir Computing Approaches to Recurrent Neural Network Training. Computer Scinece Review, 127–149 (2009)
Maass, W., Natschläger, T., Markram, H.: Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation 14(11), 2531–2560 (2002)
Maass, W.: Paradigms for computing with Spiking Neurons. Springer, Heidelberg (1999)
Maass, W., Natschlger, T., Markram, H.: Computational models for generic cortical microcircuits. In: Feng, J. (ed.) Computational Neuroscience: A Comprehensive Approach, pp. 575–605. Chapman & Hall/CRC, Boca Raton (2004)
Manevitz, L.M., Marom, S.: Modeling the process of rate selection in neuronal activity. Journal of Theoretical Biology 216(3), 337–343 (2002)
McCullough, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics 5, 115–127 (1943)
Widrow, B., Hoff, M.: Adaptive Switching Circuits. IRE WESCON Convention Record 4, 96–104 (1960)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Manevitz, L., Hazan, H. (2010). Stability and Topology in Reservoir Computing. In: Sidorov, G., Hernández Aguirre, A., Reyes García, C.A. (eds) Advances in Soft Computing. MICAI 2010. Lecture Notes in Computer Science(), vol 6438. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16773-7_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-16773-7_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-16772-0
Online ISBN: 978-3-642-16773-7
eBook Packages: Computer ScienceComputer Science (R0)