Bayesian networks (BN) are used in a big range of applications but they have one issue concerning parameter learning. In real application, training data are always incomplete or some nodes are hidden. To deal with this problem many learning parameter algorithms are suggested foreground EM, Gibbs sampling and RBE algori…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Bayesian EM”
As an automatic method of determining model complexity using the training data alone, Bayesian linear regression provides us a principled way to select hyperparameters. But one often needs approximation inference if distribution assumption is beyond Gaussian distribution. In this paper, we propose a Bayesian linear reg…
In recent years there has been a flurry of works on learning Bayesian networks from data. One of the hard problems in this area is how to effectively learn the structure of a belief network from incomplete data- that is, in the presence of missing values or hidden variables. In a recent paper, I introduced an algorithm…
Gradient-EM Bayesian meta-learning accelerates adaptation with reduced computation and improved robustness.
problem Efficient and robust adaptation to new tasks with uncertainty assessment.
method Extends Bayesian meta-learning with gradient-EM algorithm, decoupling inner-update from meta-update.
result Improves accuracy with less computation cost and enhanced robustness to uncertainty.
This paper uses dynamical systems to analyze and ensure convergence of the Bayesian EM algorithm.
problem Ensuring convergence of the Bayesian EM algorithm in incomplete-data scenarios.
method Applying Lyapunov stability theory to discrete-time dynamical systems.
result Conditions for convergence and potential for fast convergence of MAP-EM are established.
This paper re-examines the problem of parameter estimation in Bayesian networks with missing values and hidden variables from the perspective of recent work in on-line learning [Kivinen & Warmuth, 1994]. We provide a unified framework for parameter estimation that encompasses both on-line learning, where the model is c…
Multivariate Bayesian Last Layer for Regression with Uncertainty Quantification and Decompositionstat.ML
Bayesian neural network models improve uncertainty quantification in multivariate regression.
problem Uncertainty quantification in multivariate regression models with heteroscedastic noise.
method Proposes Bayesian Last Layer neural network models and EM algorithms for parameter learning.
result Capable of disentangling aleatoric and epistemic uncertainty.
Paper compares hard and soft EM for BN learning from incomplete data.
problem Learning BNs from incomplete data using EM algorithms.
method Investigates the impact of imputation vs. belief propagation in hard and soft EM.
result A decision tree can guide practitioners in choosing the best EM algorithm.
EDML is a recently proposed algorithm for learning MAP parameters in Bayesian networks. In this paper, we present a number of new advances and insights on the EDML algorithm. First, we provide the multivalued extension of EDML, originally proposed for Bayesian networks over binary variables. Next, we identify a simplif…
Estimates parameters in max-linear Bayesian networks with noise.
problem Causal inference in extreme-value settings with noise parameters.
method Max-plus algebra and logarithm transformation, normal distribution estimation, EM algorithm and quadratic optimization.
result An estimator of a parameter for each edge in a DAG is normally distributed.
Clarifies EM algorithm and variational Bayesian inference concepts.
problem Gaps in AI literature understanding of EM and variational concepts.
method Tutorial presentation of EM algorithm, variational Bayesian inference, and autoencoded variational Bayes.
result Establishes clear links between EM and variational methods.
Paper proves EM algorithm convergence for mixtures of discrete and continuous parameters.
problem Nontrivial convergence analysis for EM algorithms with mixed-integer parameters.
method Introduces conditions for EM convergence in mixed-integer optimization.
result Proves convergence of EM-based sparse Bayesian learning algorithm.
Paper improves BN structure learning from incomplete data.
problem Learning BN structure from incomplete data.
method Node-Average Likelihood (NAL) approach.
result NAL proves consistent and identifiable for conditional Gaussian BNs.
The NLMS algorithm with time-variant optimum stepsize derived from a Bayesian network perspectivestat.ML
In this article, we derive a new stepsize adaptation for the normalized least mean square algorithm (NLMS) by describing the task of linear acoustic echo cancellation from a Bayesian network perspective. Similar to the well-known Kalman filter equations, we model the acoustic wave propagation from the loudspeaker to th…
The speed of convergence of the Expectation Maximization (EM) algorithm for Gaussian mixture model fitting is known to be dependent on the amount of overlap among the mixture components. In this paper, we study the impact of mixing coefficients on the convergence of EM. We show that when the mixture components exhibit …
In classical Hawkes process, the baseline intensity and triggering kernel are assumed to be a constant and parametric function respectively, which limits the model flexibility. To generalize it, we present a fully Bayesian nonparametric model, namely Gaussian process modulated Hawkes process and propose an EM-variation…
The Gaussian mixture model is a classic technique for clustering and data modeling that is used in numerous applications. With the rise of big data, there is a need for parameter estimation techniques that can handle streaming data and distribute the computation over several processors. While online variants of the Exp…
This paper addresses the estimation of parameters of a Bayesian network from incomplete data. The task is usually tackled by running the Expectation-Maximization (EM) algorithm several times in order to obtain a high log-likelihood estimate. We argue that choosing the maximum log-likelihood estimate (as well as the max…
Identification of Gaussian Process State-Space Models with Particle Stochastic Approximation EMstat.ML
Gaussian process state-space models (GP-SSMs) are a very flexible family of models of nonlinear dynamical systems. They comprise a Bayesian nonparametric representation of the dynamics of the system and additional (hyper-)parameters governing the properties of this nonparametric representation. The Bayesian formalism e…
In this paper we develop an Expectation Maximization(EM) algorithm to estimate the parameter of a Yule-Simon distribution. The Yule-Simon distribution exhibits the "rich get richer" effect whereby an 80-20 type of rule tends to dominate. These distributions are ubiquitous in industrial settings. The EM algorithm presen…
Bayesian model fuses diverse microbiome data types.
problem Challenges in fusing different types of microbiome data.
method Flexible multinomial-Gaussian generative model with variational EM algorithm.
result Inferred latent variables provide common dimensionality reduction and predictive posterior distribution.
Proposes an EM algorithm for high-dimensional Markov-switching VAR models.
problem Estimating regime shifts in high-dimensional time series data.
method Approximate EM algorithm for Markov-switching VAR models.
result Established consistency of the proposed EM algorithm in high dimensions.
This paper presents a Bayesian approach to symbol and phase inference in a phase-unsynchronized digital receiver. It primarily extends [Quinn 2011] to the multi-symbol case, using the variational Bayes (VB) approximation to deal with the combinatorial complexity of the phase inference in this case. The work provides a …
The paper improves error bounds for Bayesian quadrature in noisy settings.
problem Improving error bounds for Bayesian quadrature in noisy settings.
method Develops a two-step meta-algorithm to relate average-case quadrature error to -function approximation error.
result Provides new average-case results for various kernels and noise settings.
Deep model learns complex latent codes without assuming factor structure.
problem Learning latent codes with complex, non-factorial distributions.
method Deep generative factor analysis with beta process prior and stochastic EM algorithm.
result Preliminary results show model can approximate complex distributions.
Bayesian fairnesscs.LG
We consider the problem of how decision making can be fair when the underlying probabilistic model of the world is not known with certainty. We argue that recent notions of fairness in machine learning need to explicitly incorporate parameter uncertainty, hence we introduce the notion of {\em Bayesian fairness} as a su…
A new method clusters data from multiple sources using a mixture of multilayer SBMs.
problem Aggregating multiple clustering results from different data sources.
method Uses a mixture of multilayer Stochastic Block Models (SBM) to group co-membership matrices.
result Identifies and clusters observations based on their specificities within components.
We consider the problem of discriminative factor analysis for data that are in general non-Gaussian. A Bayesian model based on the ranks of the data is proposed. We first introduce a new {\em max-margin} version of the rank-likelihood. A discriminative factor model is then developed, integrating the max-margin rank-lik…
New Bayesian method for sparse multidimensional item response theory.
problem Sparse interpretable explanations for questionnaire data.
method Bayesian EM algorithm for sparse factor loadings.
result Reliable recovery of factor dimensionality and latent structure.
Bayesian SPCA method tackles orthogonality constraint with spike and slab prior.
problem Bayesian SPCA method for high-dimensional data with orthogonality constraint.
method Parameter-expanded coordinate ascent variational inference (PX-CAVI) with spike and slab prior.
result PX-CAVI algorithm outperforms existing SPCA approaches in performance.
Bayes beats Cross Validation: Efficient and Accurate Ridge Regression via Expectation Maximizationstat.ML
Bayesian EM method improves ridge regression tuning without LOOCV's limitations.
problem Tuning ridge regression hyper-parameter efficiently and accurately.
method Bayesian formulation of ridge regression with expectation maximization (EM) procedure.
result Guaranteed unique optimal solution for large without candidate specification.
In this paper, Bayesian parameter estimation through the consideration of the Maximum A Posteriori (MAP) criterion is revisited under the prism of the Expectation-Maximization (EM) algorithm. By incorporating a sparsity-promoting penalty term in the cost function of the estimation problem through the use of an appropri…
In this paper we address the problem of learning the structure of a Bayesian network in domains with continuous variables. This task requires a procedure for comparing different candidate structures. In the Bayesian framework, this is done by evaluating the {em marginal likelihood/} of the data given a candidate struct…
Hidden Markov Models (HMM) have been used for several years in many time series analysis or pattern recognitions tasks. HMM are often trained by means of the Baum-Welch algorithm which can be seen as a special variant of an expectation maximization (EM) algorithm. Second-order training techniques such as Variational Ba…
Paper derives constraints for Bayesian Knowledge Tracing parameters.
problem Issues with EM algorithm in BKT parameter estimation.
method From first principles, derives constraints on BKT parameter space.
result Novel algorithm respects derived constraints for parameter estimation.
New approach improves AI's handling of incomplete data.
problem Improving AI's ability to work with incomplete data.
method Proposes a new likelihood-free EM algorithm for faster, more efficient inference.
result More statistically efficient than masking approach and faster than conventional EM.
Proposes EM for sparse horseshoe estimation.
problem Sparse estimation of sparse parameter vectors using the horseshoe prior.
method Expectation-Maximisation (EM) procedure for MAP estimates.
result Our approach performs comparable or superior to state-of-the-art methods.
New mixture models for clustering and density estimation of unknown distributions.
problem Clustering and density estimation of data with unknown distributions.
method Two fitting methods: EM algorithm and Bayesian non-parametric method using Gibbs sampler.
result Effective clustering and density estimation of data with unknown distributions.
Study uses DNNs for real-time EM inversion, highlighting model errors and proposing solutions.
problem Model errors in DNNs affect real-time geosteering decisions in EM measurements.
method Bayesian ensemble smoothing with DNNs for thousands of model evaluations, identifying multimodality.
result Model errors can lead to biased estimates, necessitating error reduction techniques.
Bayesian graphical models are a useful tool for understanding dependence relationships among many variables, particularly in situations with external prior information. In high-dimensional settings, the space of possible graphs becomes enormous, rendering even state-of-the-art Bayesian stochastic search computationally…
A Max-Product EM Algorithm for Reconstructing Markov-tree Sparse Signals from Compressive Samplesstat.ML
We propose a Bayesian expectation-maximization (EM) algorithm for reconstructing Markov-tree sparse signals via belief propagation. The measurements follow an underdetermined linear model where the regression-coefficient vector is the sum of an unknown approximately sparse signal and a zero-mean white Gaussian noise wi…
In this paper, we develop a Bayesian evidence maximization framework to solve the sparse non-negative least squares (S-NNLS) problem. We introduce a family of probability densities referred to as the Rectified Gaussian Scale Mixture (R- GSM) to model the sparsity enforcing prior distribution for the solution. The R-GSM…
In this article, we propose a new class of priors for Bayesian inference with multiple Gaussian graphical models. We introduce fully Bayesian treatments of two popular procedures, the group graphical lasso and the fused graphical lasso, and extend them to a continuous spike-and-slab framework to allow self-adaptive shr…
Scalable Bayesian Learning for State Space Models using Variational Inference with SMC Samplersstat.ML
We present a scalable approach to performing approximate fully Bayesian inference in generic state space models. The proposed method is an alternative to particle MCMC that provides fully Bayesian inference of both the dynamic latent states and the static parameters of the model. We build up on recent advances in compu…
In this work, dynamic Bayesian multinets are introduced where a Markov chain state at time t determines conditional independence patterns between random variables lying within a local time window surrounding t. It is shown how information-theoretic criterion functions can be used to induce sparse, discriminative, and c…
In this paper, the use of the Generalized Beta Mixture (GBM) and Horseshoe distributions as priors in the Bayesian Compressive Sensing framework is proposed. The distributions are considered in a two-layer hierarchical model, making the corresponding inference problem amenable to Expectation Maximization (EM). We prese…
Finite mixture model is an important branch of clustering methods and can be applied on data sets with mixed types of variables. However, challenges exist in its applications. First, it typically relies on the EM algorithm which could be sensitive to the choice of initial values. Second, biomarkers subject to limits of…
Anomaly Detection has several important applications. In this paper, our focus is on detecting anomalies in seller-reviewer data using tensor decomposition. While tensor-decomposition is mostly unsupervised, we formulate Bayesian semi-supervised tensor decomposition to take advantage of sparse labeled data. In addition…