Global optimisation of neural network models via sequential sampling
Pages 410 - 416
Abstract
We propose a novel strategy for training neural networks using sequential sampling-importance resampling algorithms. This global optimisation strategy allows us to learn the probability distribution of the network weights in a sequential framework. It is well suited to applications involving on-line, nonlinear, non-Gaussian or non-stationary signal processing.
References
[1]
Carpenter, J., Clifford, P. and Fearnhead, P. (1997). An improved particle filter for non-linear problems, Technical report, Department of Statistics, Oxford University, England. Available at http://www.stats.ox.ac.uk/~clifford/index.htm.
[2]
de Freitas, J. F. G., Niranjan, M. and Gee, A. H. (1997). Hierarchichal Bayesian-Kalman models for regularisation and ARD in sequential learning, Technical Report CUED/F-INFENG/TR 307, Cambridge University, http://svrwww.eng.cam.ac.uk/~jfgf.
[3]
de Freitas, J. F. G., Niranjan, M. and Gee, A. H. (1998). Regularisation in sequential learning algorithms, in M. I. Jordan, M. J. Kearns and S. A. Solla (eds), Advances in Neural Information Processing Systems, Vol. 10, MIT Press.
[4]
de Freitas, J. F. G., Niranjan, M., Gee, A. H. and Doucet, A. (1998). Sequential Monte Carlo methods for optimisation of neural network models, Technical Report CUED/F-INFENG/TR 328, Cambridge University, http://svrwww.eng.cam.ac.uk/~jfgf.
[5]
Doucet, A. (1998). On sequential simulation-based methods for Bayesian filtering, Technical Report CUED/F-INFENG/TR 310, Cambridge University. Available at http://www.stats.bris.ac.uk:81/MCMC/pages/list.html.
[6]
Gordon, N. J., Salmond, D. J. and Smith, A. F. M. (1993). Novel approach to nonlinear/non-Gaussian Bayesian state estimation, IEE Proceedings-F 140(2): 107-113.
[7]
Pitt, M. K. and Shephard, N. (1997). Filtering via simulation: Auxiliary particle filters, Technical report, Department of Statistics, Imperial College of London, England. Available at http://www.nuff.ox.ac.uk/economics/papers.
[8]
Singhal, S. and Wu, L. (1988). Training multilayer perceptrons with the extended Kalman algorithm, in D. S. Touretzky (ed.), Advances in Neural Information Processing Systems, Vol. 1, San Mateo, CA, pp. 133-140.
Index Terms
- Global optimisation of neural network models via sequential sampling
Index terms have been assigned to the content through auto-classification.
Recommendations
Global optimisation of neural network models via sequential sampling
Proceedings of the 1998 conference on Advances in neural information processing systems IIGlobal sampling for sequential filtering over discrete state space
In many situations, there is a need to approximate a sequence of probability measures over a growing product of finite spaces. Whereas it is in general possible to determine analytic expressions for these probability measures, the number of computations ...
Comparison of Stochastic Global Optimization Methods to Estimate Neural Network Weights
Training a neural network is a difficult optimization problem because of numerous local minima. Many global search algorithms have been used to train neural networks. However, local search algorithms are more efficient with computational resources, and ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Publisher
MIT Press
Cambridge, MA, United States
Publication History
Published: 01 December 1998
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 26 Jan 2025