Abstract
The Semi-Naive Bayesian network (SNB) classifier, a probabilistic model with an assumption of conditional independence among the combined attributes, shows a good performance in classification tasks. However, the traditional SNBs can only combine two attributes into a combined attribute. This inflexibility together with its strong independency assumption may generate inaccurate distributions for some datasets and thus may greatly restrict the classification performance of SNBs. In this paper we develop a Bounded Semi-Naive Bayesian network (B-SNB) model based on direct combinatorial optimization. Our model can join any number of attributes within a given bound and maintains a polynomial time cost at the same time. This improvement expands the expressive ability of the SNB and thus provide potentials to increase accuracy in classification tasks. Further, aiming at relax the strong independency assumption of the SNB, we then propose an algorithm to extend the B-SNB into a finite mixture structure, named Mixture of Bounded Semi-Naive Bayesian network (MBSNB). We give theoretical derivations, outline of the algorithm, analysis of the algorithm and a set of experiments to demonstrate the usefulness of MBSNB in classification tasks. The novel finite MBSNB network shows a better classification performance in comparison with than other types of classifiers in this paper.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
D. M. Chickering. Learning bayesian networks is NP-complete. In D. Fisher and H.-J. Lenz, editors, Learning from Data. Springer-Verlag, 1995.
C. K. Chow and C. N. Liu. Approximating discrete probability distributions with dependence trees. IEEE Trans. on Information Theory, 14:462–467, 1968.
Pedro Domingos and Pazzani Michael. On the optimality of the simple baysian classifier under zero-one loss. Machine Learning, 29:103–130, 1997.
N. Friedman, D. Geiger, and M. Goldszmidt. Bayesian network classifiers. Machine Learning, 29:131–161, 1997.
R. Kohavi. A study of cross validation and bootstrap for accuracy estimation and model selection. In Proceedings of the 14th IJCAI, pages 338–345. San Francisco, CA: Morgan Kaufmann, 1995.
I. Kononenko. Semi-naive bayesian classifier. In Proceedings of sixth European Working Session on Learning, pages 206–219. Springer-Verlag, 1991.
N. M. Laird, A. P. Dempster, and D.B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. J. Royal Statist. Society, B39:1–38, 1977.
P. Langley, W. Iba, and K. Thompson. An analysis of bayesian classifiers. In Proceedings of AAAI-92, pages 223–228, 1992.
M. Meila and M. Jordan. Learning with mixtures of trees. Journal of Machine Learning Research, 1:1–48, 2000.
Patrick M. Murphy. UCI repository of machine learning databases. In School of Information and Computer Science, University of California, Irvine, 2003.
M. J. Pazzani. Searching dependency in bayesian classifiers. In D. Fisher and H.-J. Lenz, editors, Learning from data: Artificial intelligence and statistics V, pages 239–248. New York, NY: Springer-Verlag, 1996.
J. Pearl. Probabilistic Reasoning in Intelligent Systems: networks of plausible inference. Morgan Kaufmann, CA, 1988.
J. R. Quinlan. C4.5: programs for machine learning. San Mateo, California: Morgan Kaufmann Publishers, 1993.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Huang, K., King, I., Lyu, M.R. (2003). Finite Mixture Model of Bounded Semi-naive Bayesian Networks Classifier. In: Kaynak, O., Alpaydin, E., Oja, E., Xu, L. (eds) Artificial Neural Networks and Neural Information Processing — ICANN/ICONIP 2003. ICANN ICONIP 2003 2003. Lecture Notes in Computer Science, vol 2714. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44989-2_15
Download citation
DOI: https://doi.org/10.1007/3-540-44989-2_15
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40408-8
Online ISBN: 978-3-540-44989-8
eBook Packages: Springer Book Archive