Shrinkage Estimator for Bayesian Network Parameters

Maximum likelihood estimates (MLEs) are commonly used to parameterize Bayesian networks. Unfortunately, these estimates frequently have unacceptably high variance and often overfit the training data. Laplacian correction can be used to smooth the MLEs towards a uniform distribution. However, the uniform distribution may represent an unrealistic relationships in the domain being modeled and can add an unreasonable bias. We present a shrinkage estimator for domains with hierarchically related random variables that smoothes MLEs towards other distributions found in the training data. Our methods are quick enough to be performed during Bayesian network structure searches. On both a simulated and a real-world neuroimaging domain, we empirically demonstrate that our estimator yields superior parameters in the presence of noise and greater likelihoods on left-out data.