More Web Proxy on the site http://driver.im/

article

Free access

Learning Non-Stationary Dynamic Bayesian Networks

Authors:

Joshua W. Robinson,

Alexander J. HarteminkAuthors Info & Claims

The Journal of Machine Learning Research, Volume 11

Pages 3647 - 3680

Published: 01 December 2010 Publication History

Abstract

Learning dynamic Bayesian network structures provides a principled mechanism for identifying conditional dependencies in time-series data. An important assumption of traditional DBN structure learning is that the data are generated by a stationary process, an assumption that is not true in many important settings. In this paper, we introduce a new class of graphical model called a non-stationary dynamic Bayesian network, in which the conditional dependence structure of the underlying data-generation process is permitted to change over time. Non-stationary dynamic Bayesian networks represent a new framework for studying problems in which the structure of a network is evolving over time. Some examples of evolving networks are transcriptional regulatory networks during an organism's development, neural pathways during learning, and traffic patterns during the day. We define the non-stationary DBN model, present an MCMC sampling algorithm for learning the structure of the model from time-series data under different assumptions, and demonstrate the effectiveness of the algorithm on both simulated and biological data.

References

[1]

Amr Ahmed and Eric P. Xing. TESLA: Recovering time-varying networks of dependencies in social and biological studies. Proceedings of the National Academy of Sciences, 106(29):11878-11883, 2009.

[2]

Michelle N. Arbeitman, Eileen E.M. Furlong, Farhad Imam, Eric Johnson, Brian H. Null, Bruce S. Baker, Mark A. Krasnow, Matthew P. Scott, Ronald W. Davis, and Kevin P. White. Gene expression during the life cycle of Drosophila melanogaster. Science, 5590(297):2270-2275, Sep 2002.

[3]

Matthew J. Beal and Zoubin Ghahramani. Variational Bayesian learning of directed graphical models with hidden variables. Bayesian Analysis, 1(4):793-832, 2006.

[4]

Allister Bernard and Alexander J. Hartemink. Informative structure priors: Joint learning of dynamic regulatory networks from multiple types of data. In Pacific Symposium on Biocomputing, volume 10, pages 459-470. World Scientific, Jan 2005.

[5]

Wray Buntine. A guide to the literature on learning probabilistic networks from data. IEEE Transactions on Knowledge and Data Engineering, 8(2):195-210, 1996.

Digital Library

[6]

Carlos M. Carvalho and Mike West. Dynamic matrix-variate graphical models. Bayesian Analysis, 2(1):69-98, 2007.

[7]

David Maxwell Chickering, Dan Geiger, and David Heckerman. Learning Bayesian networks is NP-Hard. Microsoft Research Technical Report MSR-TR-94-17, Microsoft, Nov 1994.

[8]

David Maxwell Chickering, Dan Geiger, and David Heckerman. Learning Bayesian networks: Search methods and experimental results. In Proceedings of the 5th International Workshop on Artificial Intelligence and Statistics, pages 112-128. Society for Artificial Intelligence in Statistics, Jan 1995.

[9]

Lonnie Chrisman. A roadmap to research on Bayesian networks and other decomposable probabilistic models. CMU technical report, School of Computer Science, CMU, May 1998.

[10]

Luís Miguel de Campos, Juan M. Fernandez-Luna, José Antonio Gámez, and José Miguel Puerta. Ant colony optimization for learning Bayesian networks. International Journal of Approximate Reasoning, 31(3):291-311, Nov 2002.

[11]

Stuart J. Elgar, Jun Han, and Michael V. Taylor. mef2 activity levels differentially affect gene expression during Drosophila muscle development. Proceedings of the National Academy of Sciences, 105(3):918-923, Jan 2008.

[12]

Nir Friedman. Learning belief networks in the presence of missing values and hidden variables. In Proceedings of the 14th International Conference on Machine Learning, pages 125-133. Morgan Kaufmann Publishers, 1997.

Digital Library

[13]

Nir Friedman and Zohar Yakhini. On the sample complexity of learning Bayesian networks. In Proceedings of the 12th Conference on Uncertainty in Artificial Intelligence, pages 274-282. Morgan Kaufmann Publishers Inc., Oct 1996.

Digital Library

[14]

Nir Friedman, Kevin Murphy, and Stuart Russell. Learning the structure of dynamic probabilistic networks. In Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence (UAI98), pages 139-147. Morgan Kaufmann Publishers Inc., 1998.

Digital Library

[15]

Nir Friedman, Michal Linial, Iftach Nachman, and Dana Pe'er. Using Bayesian networks to analyze expression data. In Research in Computational Molecular Biology (RECOMB00), volume 4, pages 127-135. ACM Press, Apr 2000.

Digital Library

[16]

Zoubin Ghahramani and Geoffrey E. Hinton. Variational learning for switching state-space models. Neural Computation, 12(4):963-996, Apr 2000.

Digital Library

[17]

Paolo Giudici and Robert Castelo. Improving Markov chain Monte Carlo model search for data mining. Machine Learning, 50(1-2), Jan 2003.

[18]

Paolo Giudici, Peter Green, and Claudia Tarantola. Efficient model determination for discrete graphical models. Technical report, Athens University of Economics and Business, 1999.

[19]

Marco Grzegorczyk, Dirk Husmeier, Kieron D. Edwards, Peter Ghazal, and Andrew J. Millar. Modelling non-stationary gene regulatory processes with a non-homogeneous Bayesian network and the allocation sampler. Bioinformatics, 24(18):2071-2078, Jul 2008.

Digital Library

[20]

Fan Guo, Wenjie Fu, Yanxin Shi, and Eric P. Xing. Reverse engineering temporally rewiring gene networks. In NIPS workshop on New Problems and Methods in Computational Biology, Dec 2006.

[21]

Fan Guo, Steve Hanneke, Wenjie Fu, and Eric P. Xing. Recovering temporally rewiring networks: A model-based approach. In Proceedings of the 24th International Conference on Machine Learning (ICML07), Jun 2007.

Digital Library

[22]

Steve Hanneke and Eric P. Xing. Discrete temporal models of social networks. In Workshop on Statistical Network Analysis at the 23rd International Conference on Machine Learning, Jun 2006.

Digital Library

[23]

Alexander J. Hartemink, David K. Gifford, Tommi S. Jaakkola, and Richard A. Young. Using graphical models and genomic expression data to statistically validate models of genetic regulatory networks. In Pacific Symposium on Biocomputing, volume 6, pages 422-433. World Scientific, Jan 2001.

[24]

David Heckerman, Dan Geiger, and David Maxwell Chickering. Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning, 20(3):197-243, Sep 1995.

Digital Library

[25]

Reimar Hofmann and Volker Tresp. Discovering structure in continuous variables using Bayesian networks. In Advances in Neural Information Processing Systems 8 (NIPS95), pages 500-506. MIT Press, Dec 1995.

[26]

Rhea R. Kimpo, Frederic E. Theunissen, and Allison J. Doupe. Propagation of correlated activity through multiple stages of a neural circuit. Journal of Neuroscience, 23(13):5750-5761, 2003.

[27]

Mladen Kolar, Le Song, Amr Ahmed, and Eric P. Xing. Estimating time-varying networks. The Annals of Applied Statistics, 4(1):94-123, Mar 2010.

[28]

Paul K. Krause. Learning probabilistic networks. The Knowledge Engineering Review, 13(4):321- 351, 1998.

Digital Library

[29]

Wai Lam and Fahiem Bacchus. Learning Bayesian belief networks: An approach based on the MDL principle. Computational Intelligence, 10(4):269-293, Jul 1994.

[30]

Pedro Larrañaga, Mikel Poza, Yosu Yurramendi, Roberto H. Murga, and Cindy M.H. Kuijpers. Structure learning of Bayesian networks by genetic algorithms: A performance analysis of control parameters. IEEE Journal on Pattern Analysis and Machine Intelligence, 18(9):912-926, Sep 1996.

Digital Library

[31]

Nicholas M. Luscombe, M. Madan Babu, Haiyuan Yu, Michael Snyder, Sarah A. Teichmann, and Mark Gerstein. Genomic analysis of regulatory network dynamics reveals large topological changes. Nature, 431:308-312, Sep 2004.

[32]

David Madigan, Jeremy York, and Denis Allard. Bayesian graphical models for discrete data. International Statistical Review, 63(2):215-232, Aug 1995.

[33]

Dimitris Margaritis. Distribution-free learning of Bayesian network structure in continuous domains. In Proceedings of the 20th National Conference on Artificial Intelligence (AAAI05), pages 825-830. AAAI Press / The MIT Press, Jul 2005.

Digital Library

[34]

Kevin Murphy. Learning Bayesian network structure from sparse data sets. UC Berkeley technical report 990, Computer Science Department, University of California at Berkeley, May 2001.

[35]

Thomas Sandmann, Lars J. Jensen, Janus S. Jakobsen, MichalM. Karzynski, Michael P. Eichenlaub, Peer Bork, and Eileen E.M. Furlong. A temporal map of transcription factor activity: mef2 directly regulates target genes at all stages of muscle development. Developmental Cell, 10(6): 797-807, Jun 2006.

[36]

V. Anne Smith, Erich D. Jarvis, and Alexander J. Hartemink. Influence of network topology and data collection on network inference. In Pacific Symposium on Biocomputing, volume 8, pages 164-175. World Scientific, Jan 2003.

[37]

V. Anne Smith, Jing Yu, Tom V. Smulders, Alexander J. Hartemink, and Erich D. Jarvis. Computational inference of neural information flow networks. PLoS Computational Biology, 2(11): 1436-1449, Nov 2006.

[38]

Joe Suzuki. Learning Bayesian belief networks based on the minimum description length principle: An efficient algorithm using the branch and bound technique. In Proceedings of the 13th International Conference on Machine Learning (ICML96), pages 462-470. Morgan Kaufmann Publishers Inc., Jul 1996.

[39]

Makram Talih and Nicolas Hengartner. Structural learning with time-varying components: Tracking the cross-section of financial time series. Journal of the Royal Statistical Society B, 67(3):321- 341, Jun 2005.

[40]

Claudia Tarantola. MCMC model determination for discrete graphical models. Statistical Modelling, 4(1):39-61, Apr 2004.

[41]

Stanley Wasserman and Philippa E. Pattison. Logit models and logistic regressions for social networks: I. An introduction to Markov graphs and p*. Psychometrika, 61(3):401-425, Sep 1996.

[42]

Xiang Xuan and Kevin Murphy. Modeling changing dependency structure in multivariate time series. In Proceedings of the 24th International Conference on Machine Learning (ICML07), Jun 2007.

Digital Library

[43]

Wentao Zhao, Erchin Serpedin, and Edward R. Dougherty. Inferring gene regulatory networks from time series data using the minimum description length principle. Bioinformatics, 22(17): 2129-2135, Sep 2006.

Digital Library

Cited By

Gong CZhang CYao DBi JLi WXu Y(2024)Causal Discovery from Temporal Data: An Overview and New PerspectivesACM Computing Surveys10.1145/370529757:4(1-38)Online publication date: 23-Nov-2024
https://dl.acm.org/doi/10.1145/3705297
Deeb ASeto MPan Y(2022)Piecewise-deterministic Quasi-static Pose Graph SLAM in Unstructured Dynamic EnvironmentsJournal of Intelligent and Robotic Systems10.1007/s10846-022-01739-5106:2Online publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1007/s10846-022-01739-5
Du YWang JFeng WPan SQin TXu RWang CDemartini GZuccon GCulpepper JHuang ZTong H(2021)AdaRNNProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482315(402-411)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482315
Show More Cited By

Index Terms

Learning Non-Stationary Dynamic Bayesian Networks
1. Computing methodologies
  1. Machine learning
2. Mathematics of computing
  1. Probability and statistics
    1. Statistical paradigms
      1. Time series analysis

Recommendations

Non-stationary dynamic Bayesian networks
NIPS'08: Proceedings of the 21st International Conference on Neural Information Processing Systems

A principled mechanism for identifying conditional dependencies in time-series data is provided through structure learning of dynamic Bayesian networks (DBNs). An important assumption of DBN structure learning is that the data are generated by a ...
Bayesian learning of Bayesian networks with informative priors

This paper presents and evaluates an approach to Bayesian model averaging where the models are Bayesian nets (BNs). A comprehensive study of the literature on structural priors for BNs is conducted. A number of prior distributions are defined using ...
Learning Dynamic Bayesian Networks Structure Based on Bayesian Optimization Algorithm
ISNN '07: Proceedings of the 4th international symposium on Neural Networks: Part II--Advances in Neural Networks

An optimization algorithm for dynamic Bayesian networks (DBN) based on Bayesian optimization algorithm (BOA) is developed for learning and constructing the DBN structure. In this paper, we first introduce some basic theories and concepts of probability ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image The Journal of Machine Learning Research

The Journal of Machine Learning Research Volume 11, Issue

3/1/2010

3637 pages

ISSN:1532-4435

EISSN:1533-7928

Issue’s Table of Contents

Publisher

JMLR.org

Publication History

Published: 01 December 2010

Published in JMLR Volume 11

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
558
Total Downloads

Downloads (Last 12 months)44
Downloads (Last 6 weeks)12

Reflects downloads up to 22 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gong CZhang CYao DBi JLi WXu Y(2024)Causal Discovery from Temporal Data: An Overview and New PerspectivesACM Computing Surveys10.1145/370529757:4(1-38)Online publication date: 23-Nov-2024
https://dl.acm.org/doi/10.1145/3705297
Deeb ASeto MPan Y(2022)Piecewise-deterministic Quasi-static Pose Graph SLAM in Unstructured Dynamic EnvironmentsJournal of Intelligent and Robotic Systems10.1007/s10846-022-01739-5106:2Online publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1007/s10846-022-01739-5
Du YWang JFeng WPan SQin TXu RWang CDemartini GZuccon GCulpepper JHuang ZTong H(2021)AdaRNNProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482315(402-411)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482315
Villa SStella F(2018)Learning continuous time Bayesian networks in non-stationary domains (extended abstract)Proceedings of the 27th International Joint Conference on Artificial Intelligence10.5555/3304652.3304819(5656-5660)Online publication date: 13-Jul-2018
https://dl.acm.org/doi/10.5555/3304652.3304819
Wang YGao HChen G(2018)Predictive complex event processing based on evolving Bayesian networksPattern Recognition Letters10.1016/j.patrec.2017.05.008105:C(207-216)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.1016/j.patrec.2017.05.008
Motzek AMöller R(2017)Indirect causes in dynamic Bayesian networks revisitedJournal of Artificial Intelligence Research10.5555/3176788.317678959:1(1-58)Online publication date: 1-May-2017
https://dl.acm.org/doi/10.5555/3176788.3176789
Lu JKolar MLiu H(2017)Post-regularization inference for time-varying nonparanormal graphical modelsThe Journal of Machine Learning Research10.5555/3122009.324206018:1(7401-7478)Online publication date: 1-Jan-2017
https://dl.acm.org/doi/10.5555/3122009.3242060
Schwaller LRobin S(2017)Exact Bayesian inference for off-line change-point detection in tree-structured graphical modelsStatistics and Computing10.1007/s11222-016-9689-327:5(1331-1345)Online publication date: 1-Sep-2017
https://dl.acm.org/doi/10.1007/s11222-016-9689-3
Chu VWong RChi CZhou WHo I(2017)The design of a cloud-based tracker platform based on system-of-systems service architectureInformation Systems Frontiers10.1007/s10796-017-9768-919:6(1283-1299)Online publication date: 1-Dec-2017
https://dl.acm.org/doi/10.1007/s10796-017-9768-9
Bendtsen M(2017)Regimes in baseball players' career dataData Mining and Knowledge Discovery10.1007/s10618-017-0510-531:6(1580-1621)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.1007/s10618-017-0510-5
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents