Abstract
Lack of relevant data is a major challenge for learning Bayesi-an networks (BNs) in real-world applications. Knowledge engineering techniques attempt to address this by incorporating domain knowledge from experts. The paper focuses on learning node probability tables using both expert judgment and limited data. To reduce the massive burden of eliciting individual probability table entries (parameters) it is often easier to elicit constraints on the parameters from experts. Constraints can be interior (between entries of the same probability table column) or exterior (between entries of different columns). In this paper we introduce the first auxiliary BN method (called MPL-EC) to tackle parameter learning with exterior constraints. The MPL-EC itself is a BN, whose nodes encode the data observations, exterior constraints and parameters in the original BN. Also, MPL-EC addresses (i) how to estimate target parameters with both data and constraints, and (ii) how to fuse the weights from different causal relationships in a robust way. Experimental results demonstrate the superiority of MPL-EC at various sparsity levels compared to conventional parameter learning algorithms and other state-of-the-art parameter learning algorithms with constraints. Moreover, we demonstrate the successful application to learn a real-world software defects BN with sparse data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Fenton, N., Neil, M.: Risk Assessment and Decision Analysis with Bayesian Networks. CRC Press, New York (2012)
Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: The combination of knowledge and statistical data. Mach. Learn. 20(3), 197–243 (1995)
Hutchinson, R.A., Niculescu, R.S., Keller, T.A., Rustandi, I., Mitchell, T.M.: Modeling fMRI data generated by overlapping cognitive processes with unknown onsets using hidden process models. NeuroImage 46(1), 87–104 (2009)
Yet, B., Perkins, Z., Fenton, N., Tai, N., Marsh, W.: Not just data: A method for improving prediction with knowledge. J. Biomed. Inform. 48, 28–37 (2014)
Šingliar, T., Hauskrecht, M.: Learning to detect incidents from noisily labeled data. Mach. Learn. 79(3), 335–354 (2010)
Liao, W., Ji, Q.: Learning Bayesian network parameters under incomplete data with domain knowledge. Pattern Recogn 42(11), 3046–3056 (2009)
Wellman, M.P.: Fundamental concepts of qualitative probabilistic networks. Artif. Intell. 44(3), 257–303 (1990)
Druzdzel, M.J., Van Der Gaag, L.C.: Elicitation of probabilities for belief networks: Combining qualitative and quantitative information. In: Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann Publishers Inc, pp. 141–148. Morgan Kaufmann, San Francisco (1995)
Cano, A., Masegosa, A.R., Moral, S.: A method for integrating expert knowledge when learning Bayesian networks from data. IEEE Trans. on Sys. Man Cyber. Part B 41(5), 1382–1394 (2011)
Niculescu, R.S., Mitchell, T., Rao, B.: Bayesian network learning with parameter constraints. J. Mach. Learn. Res. 7, 1357–1383 (2006)
Yang, S., Natarajan, S.: Knowledge intensive learning: Combining qualitative constraints with causal independence for parameter learning in probabilistic models. In: Blockeel, H., Kersting, K., Nijssen, S., Železný, F. (eds.) ECML PKDD 2013, Part II. LNCS, vol. 8189, pp. 580–595. Springer, Heidelberg (2013)
Zhou, Y., Fenton, N., Neil, M.: Bayesian network approach to multinomial parameter learning using data and expert judgments. Int. J. Approx. Reasoning 55(5), 1252–1268 (2014)
Altendorf, E.E.: Learning from sparse data by exploiting monotonicity constraints. In: Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence, pp. 18–26. Morgan Kaufmann Publishers Inc., San Francisco (2005)
van der Gaag, L.C., Renooij, S., Geenen, P.L.: Lattices for studying monotonicity of Bayesian networks. In: Proceedings of the 3rd European Workshop on Probabilistic Graphical Models, Prague, Czech Republic, pp. 99–106 (2006)
van der Gaag, L.C., Tabachneck-Schijf, H.J.M., Geenen, P.L.: Verifying monotonicity of bayesian networks with domain experts. Int. J. Approx. Reasoning 50(3), 429–436 (2009)
Neil, M., Tailor, M., Marquez, D.: Inference in hybrid Bayesian networks using dynamic discretization. Stat. and Comput. 17(3), 219–233 (2007)
Lunn, D.J., Thomas, A., Best, N., Spiegelhalter, D.: WinBUGS-a Bayesian modelling framework: concepts, structure, and extensibility. Stat. and Comput. 10(4), 325–337 (2000)
Shenoy, P.P., West, J.C.: Inference in hybrid Bayesian networks using mixtures of polynomials. Int. J. Approx. Reasoning 52(5), 641–657 (2011)
Shenoy, P.P.: Two issues in using mixtures of polynomials for inference in hybrid Bayesian networks. Int. J. Approx. Reasoning 53(5), 847–866 (2012)
Feelders, A., van der Gaag, L.: Learning Bayesian network parameters under order constraints. Int. J. Approx. Reasoning 42(1), 37–53 (2006)
Xiang, Y., Jia, N.: Modeling causal reinforcement and undermining for efficient CPT elicitation. IEEE Trans. on Knowl. and Data Eng. 19(12), 1708–1718 (2007)
Xiang, Y., Truong, M.: Acquisition of causal models for local distributions in Bayesian networks. IEEE Trans. on Cyber (2013), doi:10.1109/TCYB.2013.2290775
Neil, M., Chen, X., Fenton, N.: Optimizing the calculation of conditional probability tables in hybrid Bayesian networks using binary factorization. IEEE Trans. on Knowl. and Data Eng. 24(7), 1306–1312 (2012)
AgenaRisk (2014), http://www.agenarisk.com/
Cover, T.M., Thomas, J.: Entropy, relative entropy and mutual information. Elements of Information Theory, 12–49 (1991)
Lauritzen, S., Spiegelhalter, D.: Local computations with probabilities on graphical structures and their application to expert systems. J. Roy. Statist. Soc. Ser. B, 157–224 (1988)
Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Mach. Learn. 29(2-3), 131–163 (1997)
Korb, K.B., Nicholson, A.E.: Bayesian Artificial Intelligence. CRC Press, New York (2003)
Fenton, N., Neil, M., Marsh, W., Hearty, P., Radliński, Ł., Krause, P.: On the effectiveness of early life cycle defect prediction with Bayesian nets. Empirical Softw. Engg. 13(5), 499–537 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Zhou, Y., Fenton, N., Neil, M. (2014). An Extended MPL-C Model for Bayesian Network Parameter Learning with Exterior Constraints. In: van der Gaag, L.C., Feelders, A.J. (eds) Probabilistic Graphical Models. PGM 2014. Lecture Notes in Computer Science(), vol 8754. Springer, Cham. https://doi.org/10.1007/978-3-319-11433-0_38
Download citation
DOI: https://doi.org/10.1007/978-3-319-11433-0_38
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11432-3
Online ISBN: 978-3-319-11433-0
eBook Packages: Computer ScienceComputer Science (R0)