Abstract
The paper is focussing on some recent developments in nonparametric mixture distributions. It discusses nonparametric maximum likelihood estimation of the mixing distribution and will emphasize gradient type results, especially in terms of global results and global convergence of algorithms such as vertex direction or vertex exchange method. However, the NPMLE (or the algorithms constructing it) provides also an estimate of the number of components of the mixing distribution which might be not desirable for theoretical reasons or might be not allowed from the physical interpretation of the mixture model. When the number of components is fixed in advance, the before mentioned algorithms can not be used and globally convergent algorithms do not exist up to now. Instead, the EM algorithm is often used to find maximum likelihood estimates. However, in this case multiple maxima are often occuring. An example from a meta-analyis of vitamin A and childhood mortality is used to illustrate the considerable, inferential importance of identifying the correct global likelihood. To improve the behavior of the EM algorithm we suggest a combination of gradient function steps and EM steps to achieve global convergence leading to the EM algorithm with gradient function update (EMGFU). This algorithms retains the number of components to be exactly k and typically converges to the global maximum. The behavior of the algorithm is highlighted at hand of several examples.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Böhning D. 1982. Convergence of Simar's algorithm for finding the maximum likelihood estimate of a compound Poisson process. Annals of Statistics 10: 1006–1008.
Böhning D. 2000. Computer-Assisted Analysis of Mixtures and Applications. Meta-Analysis, Disease Mapping and Others. Chapman & Hall/CRC, Boca Raton.
Carlin B.P. and Louis T.A. 2000. Bayes and Empirical Bayes Methods for Data Analysis. Chapman & Hall, London.
Celeux G. 2001. Different points of view for choosing the number of components in a mixture model. In: Govaert G., Janssen J., and Limnios N. (Eds.), Proceedings of the 10th International Symposium on Applied Stochastic Models and Data Analysis, June 12-15. Compiegne, pp. 21–28.
Dempster A.P., Laird N.M., and Rubin D.B. 1977. Maximum likelihood estimation from incomplete data via the EM algorithm (with Discussion). Journal of the Royal Statistical Society B 39: 1–38.
Fawzi W.W., Chalmers T.C., Herrera M.G., and Mosteller F. 1993. Vitamin A supplementation and child mortality. A meta-analysis. Journal of the American Medical Association 269: 898–903.
Hasselblad V. 1969. Estimation of finite mixtures of distributions from the exponential family. Journal of the American Statistical Association 64: 1459–1471.
Laird N.M. 1978. Nonparametric maximum likelihood estimation of a mixing distribution. Journal of the American Statistical Association 73: 805–811.
Leroux B.G. 1992. Consistent estimation of a mixing distribution. Annals of Statistics 20: 1350–1360.
Lim, T.-O., Bakri R., Morad Z., and Hamid M.A. 2001. Bimodality in blood glucose distribution: Is it universal? Preprint of the Clinical Research Centre, c/o Department of Nephrology, Kualalumpur Hospital, Kualalumpur, Malaysia.
Lindsay B.G. 1983. The geometry of mixture likelihoods, Part I: A general theory. Annals of Statistics 11: 783–792.
Mclachlan G. and Peel D. 2000. Finite Mixture Models. Wiley, New York.
Seidel W., Mosler K., and Alker M. 2000. A cautionary note on likelihood ratio tests in mixture models. Annals of the Institute of Statistical Mathematics 52: 481–487.
Simar L. 1976. Maximum likelihood estimation of a compound Poisson process. Annals of Statistics 4: 1200–1209.
Thyrion P. 1960. Contribution à l'étude du bonus pour non sinsitre en assurance automobile. Astin Bulletin 1: 142–162.
Titterington D.M., Smith A.F.M., and Makov U.E. 1985. Statistical Analysis of Finite Mixture Distributions. Wiley, New York.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Böhning, D. The EM algorithm with gradient function update for discrete mixtures with known (fixed) number of components. Statistics and Computing 13, 257–265 (2003). https://doi.org/10.1023/A:1024222817645
Issue Date:
DOI: https://doi.org/10.1023/A:1024222817645