Abstract
We describe a directed acyclic graphical model that contains a hierarchy of linear units and a mechanism for dynamically selecting an appropriate subset of these units to model each observation. The non-linear selection mechanism is a hierarchy of binary units each of which gates the output of one of the linear units. There are no connections from linear units to binary units, so the generative model can be viewed as a logistic belief net (Neal 1992) which selects a skeleton linear model from among the available linear units. We show that Gibbs sampling can be used to learn the parameters of the linear and binary units even when the sampling is so brief that the Markov chain is far from equilibrium.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Everitt, B. S. (1984). An Introduction to Latent Variable Models. Chapman and Hall, London.
Ghahramani, Z. and Hinton, G. E. (1996). The EM algorithm for mixtures of factor analyzers. Technical Report CRG-TR-96–1 [ftp://ftp.cs.toronto.edu/pub/zoubin/tr-96-1.ps.gz], Department of Computer Science, University of Toronto.
Hinton, G. E., Dayan, P., Frey, B. J., and Neal, R. M. (1995). The wake-sleep algorithm for unsupervised neural networks. Science, 268: 1158–1161.
Hinton, G. E., Dayan, P., and Revow, M. (1997). Modeling the manifolds of Images of handwritten digits. IEEE Trans. Neural Networks, 8 (1): 65–74.
Hinton, G. E. and Ghahramani, Z. (1997). Generative models for discovering sparse distributed representations. Phil. Trans. Roy. Soc. London B, 352: 1177–1190.
Hull, J. J. (1994). A database for handwritten text recognition research. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16 (5): 550–554.
Jacobs, R. A., Jordan, M. I., Nowlan, S. J., and Hinton, G. E. (1991). Adaptive mixture of local experts. Neural Computation, 3: 79–87.
Jordan, M. I. and Jacobs, R. (1994). Hierarchical mixtures of experts and the EM algorithm. Neural Computation, 6: 181–214.
Neal, R. M. (1992). Connectionist learning of belief networks. Artificial Intelligence, 56: 71–113.
Neal, R. M. and Hinton, G. E. (1993). A new view of the EM algorithm that justifies incremental and other variants. Unpublished manuscript [ftp://ftp.cs.utoronto.ca/pub/radford/em.ps.z], Department of Computer Science, University of Toronto.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1998 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Hinton, G.E., Sallans, B., Ghahramani, Z. (1998). A Hierarchical Community of Experts. In: Jordan, M.I. (eds) Learning in Graphical Models. NATO ASI Series, vol 89. Springer, Dordrecht. https://doi.org/10.1007/978-94-011-5014-9_17
Download citation
DOI: https://doi.org/10.1007/978-94-011-5014-9_17
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-6104-9
Online ISBN: 978-94-011-5014-9
eBook Packages: Springer Book Archive