More Web Proxy on the site http://driver.im/

research-article

Being Bayesian about learning Bayesian networks from ordinal data

Author:

Marco GrzegorczykAuthors Info & Claims

Volume 170, Issue C

https://doi.org/10.1016/j.ijar.2024.109205

Published: 24 July 2024 Publication History

Abstract

In this paper we propose a Bayesian approach for inferring Bayesian network (BN) structures from ordinal data. Our approach can be seen as the Bayesian counterpart of a recently proposed frequentist approach, referred to as the ‘ordinal structure expectation maximization’ (OSEM) method. Like for the OSEM method, the key idea is to assume that each ordinal variable originates from a Gaussian variable that can only be observed in discretized form, and that the dependencies in the latent Gaussian space can be modeled by BNs; i.e. by directed acyclic graphs (DAGs). Our Bayesian method combines the ‘structure MCMC sampler’ for DAG posterior sampling, a slightly modified version of the ‘Bayesian metric for Gaussian networks having score equivalence’ (BGe score), the concept of the ‘extended rank likelihood’, and a recently proposed algorithm for posterior sampling the parameters of Gaussian BNs. In simulation studies we compare the new Bayesian approach and the OSEM method in terms of the network reconstruction accuracy. The empirical results show that the new Bayesian approach leads to significantly improved network reconstruction accuracies.

References

[1]

J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann, San Francisco, CA, USA, 1988.

Digital Library

[2]

R. Neapolitan, Probabilistic Reasoning in Expert Systems: Theory and Algorithms, CreateSpace, Scotts, Valley CA, USA, 1989.

[3]

D. Koller, N. Friedman, Probabilistic Graphical Models: Principles and Techniques, Adaptive Computation and Machine Learning series, MIT Press, Cambridge, MA, USA, ISBN 9780262013192, 2009.

[4]

D.M. Chickering, Learning Bayesian networks is NP-complete, in: D. Fisher, H.J. Lenz (Eds.), Learning from Data: Artificial Intelligence and Statistics, vol. 5, Springer, New York, 1996, pp. 121–130.

[5]

P. Spirtes, C. Glymour, R. Scheines, Causation, Prediction, and Search, 2nd ed., MIT Press, 2001.

[6]

M. Kalisch, P. Bühlmann, Estimating high-dimensional directed acyclic graphs with the PC-algorithm, J. Mach. Learn. Res. 8 (2007) 613–636.

Digital Library

[7]

D. Marella, P. Vicard, Bayesian network structural learning from complex survey data: a resampling based approach, Stat. Methods Appl. 31 (2022) 981–1013.

[8]

R.R. Bouckaert, Properties of Bayesian belief network learning algorithms, in: R.L. de Mántaras, D. Poole (Eds.), UAI'94: Proceedings of the Tenth Annual Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann, 1994, pp. 102–109.

[9]

D. Chickering, D. Geiger, D. Heckerman, Learning Bayesian networks: Search methods and experimental results, in: Proceedings of Fifth Conference on Artificial Intelligence and Statistics, Society for Artificial Intelligence in Statistics, Ft. Lauderdale, FL, 1995, pp. 112–128.

[10]

J. Cussens, Bayesian network learning with cutting planes, in: F. Cozman, A. Pfeffer (Eds.), UAI 2011: Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence, AUAI Press, 2011, pp. 153–160.

[11]

A.C. Constantinou, Y. Liu, N.K. Kitson, K. Chobtham, Z. Guo, Effective and efficient structure learning with pruning and model averaging strategies, Int. J. Approx. Reason. 151 (2022) 292–321.

[12]

D. Madigan, J. York, Bayesian graphical models for discrete data, Int. Stat. Rev. 63 (1995) 215–232.

[13]

P. Giudici, R. Castelo, Improving Markov chain Monte Carlo model search for data mining, Mach. Learn. 50 (2003) 127–158.

[14]

N. Friedman, D. Koller, Being Bayesian about network structure: A Bayesian approach to structure discovery in Bayesian networks, Mach. Learn. 50 (2003) 95–126.

[15]

M. Grzegorczyk, D. Husmeier, Improving the structure MCMC sampler for Bayesian networks by introducing a new edge reversal move, Mach. Learn. 71 (2008) 265–305.

[16]

J. Kuipers, G. Moffa, Partition MCMC for inference on acyclic digraphs, J. Am. Stat. Assoc. 112 (2017) 282–299.

[17]

P. Nandy, A. Hauser, M.H. Maathuis, High-dimensional consistency in score-based and hybrid structure learning, Ann. Stat. 46 (2018) 3151–3183.

[18]

M. Scutari, C.E. Graafland, J.M. Gutiérrez, Who learns better Bayesian network structures: Constraint-based, score-based or hybrid algorithms?, in: International Conference on Probabilistic Graphical Models, PMLR, 2018, pp. 416–427.

[19]

J. Kuipers, P. Suter, G. Moffa, Efficient sampling and structure learning of Bayesian networks, J. Comput. Graph. Stat. 31 (2022) 639–650.

[20]

N.K. Kitson, A.C. Constantinou, Z. Guo, Y. Liu, K. Chobtham, A survey of Bayesian network structure learning, Artif. Intell. Rev. 56 (2023) 8721–8814.

[21]

R. Cui, P. Groot, T. Heskes, Copula PC algorithm for causal discovery from mixed data, in: P. Frasconi, N. Landwehr, G. Manco, J. Vreeken (Eds.), Machine Learning and Knowledge Discovery in Databases, Springer International Publishing, Cham, 2016, pp. 377–392.

[22]

M. Tsagris, G. Borboudakis, V. Lagani, I. Tsamardinos, Constraint-based causal discovery with mixed data, Int. J. Data Sci. Anal. 6 (2018) 19–30.

[23]

T. Talvitie, R. Eggeling, M. Koivisto, Learning Bayesian networks with local structure, mixed variables, and exact algorithms, Int. J. Approx. Reason. 115 (2019) 69–95.

[24]

X. Luo, G. Moffa, J. Kuipers, Learning Bayesian networks from ordinal data, J. Mach. Learn. Res. 22 (2021) 1–44.

[25]

D. Heckerman, D. Geiger, Learning Bayesian networks: A unification for discrete and Gaussian domains, in: Proceedings of the 11th Annual Conference on Uncertainty in Artificial Intelligence (UAI-95), Morgan Kaufmann, San Francisco, CA, 1995, pp. 274–282.

[26]

D. Geiger, D. Heckerman, Parameter priors for directed acyclic graphical models and the characterization of several probability distributions, Ann. Stat. 30 (2002) 1412–1440.

[27]

J. Kuipers, G. Moffa, D. Heckerman, Addendum on the scoring of Gaussian directed acyclic graphical models, Ann. Stat. 42 (2014) 1689–1691.

[28]

M. Scutari, Learning Bayesian networks with the bnlearn R package, J. Stat. Softw. 35 (2010) 1–22.

[29]

M. Scutari, Bayesian network constraint-based structure learning algorithms: Parallel and optimized implementations in the bnlearn R package, J. Stat. Softw. 77 (2017) 1–20.

[30]

M. Scutari, J.-B. Denis, Bayesian Networks: With Examples in R, Chapman & Hall, Boca Raton, FL, 2014.

[31]

M. Kalisch, M. Mächler, D. Colombo, M.H. Maathuis, P. Bühlmann, Causal inference using graphical models with the R package pcalg, J. Stat. Softw. 47 (2012) 1–26.

[32]

P. Suter, J. Kuipers, G. Moffa, N. Beerenwinkel, Bayesian structure learning and sampling of Bayesian networks with the R package BiDAG, J. Stat. Softw. 105 (2023) 1–31.

[33]

P. Hoff, Extending the rank likelihood for semiparametric estimation, Ann. Appl. Stat. 1 (2007) 265–283.

[34]

M. Grzegorczyk, Being Bayesian about learning Gaussian Bayesian networks from incomplete data, Int. J. Approx. Reason. 160 (2023),.

Digital Library

[35]

A. Dobra, A. Lenkoski, A. Rodriguez, Bayesian inference for general Gaussian graphical models with application to multivariate lattice data, J. Am. Stat. Assoc. 106 (2011) 1418–1433.

[36]

A. Mohammadi, F. Abegaz, E. Heuvel, E.C. Wit, Bayesian Modelling of Dupuytren Disease by Using Gaussian Copula Graphical Models, J. R. Stat. Soc., Ser. C, Appl. Stat. 66 (2016) 629–645.

[37]

S. Lauritzen, Graphical Models, Clarendon Press, Oxford, 1996.

[38]

D. Geiger, D. Heckerman, Learning Gaussian networks, in: Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann, San Francisco, CA, 1994, pp. 235–243.

[39]

D.M. Chickering, Learning Equivalence Classes of Bayesian-Network Structures, J. Mach. Learn. Res. 2 (2002) 445–498.

[40]

R.D. Shachter, R. Kenley, Gaussian influence diagrams, Manag. Sci. 35 (1989) 527–550.

[41]

N. Friedman, Learning belief networks in the presence of missing values and hidden variables, in: D.H. Fisher (Ed.), Proceedings of the Fourteenth International Conference on Machine Learning (ICML), Morgan Kaufmann, Nashville, Tennessee, USA, 1997, pp. 125–133.

[42]

G. Schwarz, Estimating the dimension of a model, Ann. Stat. 6 (1978) 461–464.

[43]

G. Wei, M. Tanner, A Monte Carlo implementation of the EM algorithm and the poor man's data augmentation algorithms, J. Am. Stat. Assoc. 85 (1990) 699–704.

[44]

M. Scutari, P. Howell, D.J. Balding, I. Mackay, Multiple Quantitative Trait Analysis Using Bayesian Networks, Genetics 198 (2014) 129–137.

[45]

K. Sachs, O. Perez, D. Pe'er, D. Lauffenburger, G. Nolan, Protein-signaling networks derived from multiparameter single-cell data, Science 308 (2005) 523–529.

[46]

A.J. Hartemink, Principled Computational Methods for the Validation and Discovery of Genetic Regulatory Networks, Ph.D. thesis School of Electrical Engineering and Computer Science, Massachusetts Institute of Technology (MIT), MA, US, 2001.

[47]

K. Mardia, J. Kent, J. Bibby, Multivariate Analysis, Academic Press, 1979.

Cited By

Gong CZhang CYao DBi JLi WXu Y(2024)Causal Discovery from Temporal Data: An Overview and New PerspectivesACM Computing Surveys10.1145/370529757:4(1-38)Online publication date: 23-Nov-2024
https://dl.acm.org/doi/10.1145/3705297

Recommendations

Bayesian learning of Bayesian networks with informative priors

This paper presents and evaluates an approach to Bayesian model averaging where the models are Bayesian nets (BNs). A comprehensive study of the literature on structural priors for BNs is conducted. A number of prior distributions are defined using ...
Being Bayesian about learning Gaussian Bayesian networks from incomplete data
Abstract
We propose a Bayesian model averaging (BMA) approach for inferring the structure of Gaussian Bayesian networks (BNs) from incomplete data, i.e. from data with missing values. Our method builds on the ‘Bayesian metric for Gaussian ...
New skeleton-based approaches for Bayesian structure learning of Bayesian networks

Automatically learning the graph structure of a single Bayesian network (BN) which accurately represents the underlying multivariate probability distribution of a collection of random variables is a challenging task. But obtaining a Bayesian solution to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image International Journal of Approximate Reasoning

International Journal of Approximate Reasoning Volume 170, Issue C

Jul 2024

273 pages

Issue’s Table of Contents

The Author(s).

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 24 July 2024

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gong CZhang CYao DBi JLi WXu Y(2024)Causal Discovery from Temporal Data: An Overview and New PerspectivesACM Computing Surveys10.1145/370529757:4(1-38)Online publication date: 23-Nov-2024
https://dl.acm.org/doi/10.1145/3705297

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents