The paper uses Gaussian mixture models for Bayesian networks and proposes an optimization algorithm.
problem Modeling nodes in Bayesian networks with complex distributions.
method Gaussian mixture models combined with double iteration algorithm.
result The double iteration algorithm optimizes Gaussian mixture models effectively.
Novel framework for Bayesian reinforcement learning infers value function distributions.
problem Bayesian reinforcement learning's challenges in inferring value function distributions.
method Inferential Induction framework for Bayesian reinforcement learning, developing Bayesian Backwards Induction algorithm.
result Proposed algorithm is competitive with state-of-the-art methods.
Two novel distributed VB algorithms improve Bayesian inference in sensor networks.
problem Efficient inference in Bayesian frameworks for sensor networks.
method Two novel distributed VB algorithms for general Bayesian inference, using stochastic natural gradient and ADMM.
result Distributed algorithms perform nearly as well as centralized ones, demonstrating excellent performance.
BaMANI uses ensemble learning to improve Bayesian network inference.
problem Bayesian network inference's reliance on specific algorithms can obscure causal relationships.
method Developed an ensemble learning approach to marginalize algorithm impact.
result Improved accuracy and reliability of causal network predictions.
Bayesian learning rule unifies and generalizes various machine learning algorithms.
problem Machine learning algorithms are diverse and not always understood.
method Bayesian principles and natural gradients are used to derive algorithms.
result Derives a wide range of algorithms including classical and modern ones.
Bayesian model compares ML algorithms on various datasets.
problem Comparing multiple machine learning algorithms across multiple datasets.
method Bayesian Bradley-Terry model, defining regions of practical equivalence (ROPE).
result Allows nuanced statements and ROPE definitions for algorithm comparison.
We review three algorithms for Latent Dirichlet Allocation (LDA). Two of them are variational inference algorithms: Variational Bayesian inference and Online Variational Bayesian inference and one is Markov Chain Monte Carlo (MCMC) algorithm -- Collapsed Gibbs sampling. We compare their time complexity and performance.…
We present new algorithms for learning Bayesian networks from data with missing values using a data augmentation approach. An exact Bayesian network learning algorithm is obtained by recasting the problem into a standard Bayesian network learning problem without missing data. To the best of our knowledge, this is the f…
Responds to comments on Bayesian Logic Regression algorithm, provides extensions and tutorial.
problem Improving Bayesian Logic Regression algorithm and its applications.
method Summarizes and responds to discussants' comments, provides extensions and tutorial.
result Extensions and tutorial for the Bayesian Logic Regression algorithm.
New algorithms minimize PAC-Bayesian C-Bound for majority voting, leading to scalable and accurate predictors.
problem Improving majority vote classifiers using PAC-Bayesian bounds.
method Directly optimizing PAC-Bayesian guarantees on the C-Bound with gradient descent.
result Self-bounding majority vote learning algorithms with scalable and accurate predictors.
New algorithm tackles big data Bayesian problems with latent variables.
problem Bayesian computing for large-scale problems with missing data and dimension jumping.
method Extended stochastic gradient MCMC with latent variables.
result Highly scalable and more efficient than traditional MCMC algorithms.
Paper proposes an efficient algorithm for learning sparse Bayesian networks from discrete high-dimensional data.
problem Learning sparse structure Bayesian networks from high-dimensional discrete data.
method Score function for sparse DAG, block-wise stochastic coordinate descent with variance reduction.
result The proposed algorithm outperforms existing methods in synthetic data benchmarks.
Machine learning provides algorithms that can learn from data and make inferences or predictions on data. Bayesian networks are a class of graphical models that allow to represent a collection of random variables and their condititional dependencies by directed acyclic graphs. In this paper, an inference algorithm for …
Vanilla Bayesian optimization performs well in high dimensions.
problem Bayesian optimization's poor performance in high-dimensional problems.
method Identified and addressed degeneracies, proposed scaling of Gaussian process lengthscale prior.
result Vanilla Bayesian optimization outperforms existing algorithms in high-dimensional tasks.
A new method for high-dimensional Bayesian optimization.
problem Challenges in extending BO to high dimensions.
method Expected Coordinate Improvement (ECI) criterion for high-dimensional Bayesian optimization.
result Significantly better results than standard BO and competitive results with state-of-the-art methods.
Paper proposes fully Bayesian approach for RVM classification, improving accuracy especially in imbalanced data.
problem Difficulty in conducting RVM classification due to lack of closed-form solution for weight parameter posterior.
method Proposes Generic Bayesian and Fully Bayesian approaches with hierarchical hyperprior structure.
result Improves classification performance, especially in imbalanced data.
New algorithm learns Bayesian networks robustly in nearly-linear time.
problem Learning Bayesian networks with adversarially corrupted samples.
method Developed a nearly-linear time algorithm connecting robust learning to robust mean estimation.
result First nearly-linear time algorithm with comparable error guarantees.
EDAs with matrix transpose improve Bayesian structure learning performance.
problem Improving Bayesian structure learning performance.
method Introducing a matrix transpose mutation operator for EDAs in Bayesian structure learning.
result EDAs with transpose mutation give markedly better performance than conventional EDAs.
Explosive growth in data and availability of cheap computing resources have sparked increasing interest in Big learning, an emerging subfield that studies scalable machine learning algorithms, systems, and applications with Big Data. Bayesian methods represent one important class of statistic methods for machine learni…
Bayesian coresets improve scalable Bayesian inference.
problem Efficiently approximating posterior inference with a subset of data.
method Sparsity constrained optimization and accelerated optimization methods.
result Explicit convergence rate guarantees and superior performance compared to state-of-the-art.
In recent years there has been a flurry of works on learning Bayesian networks from data. One of the hard problems in this area is how to effectively learn the structure of a belief network from incomplete data- that is, in the presence of missing values or hidden variables. In a recent paper, I introduced an algorithm…
Bayesian methods for machine learning have been widely investigated, yielding principled methods for incorporating prior information into inference algorithms. In this survey, we provide an in-depth review of the role of Bayesian methods for the reinforcement learning (RL) paradigm. The major incentives for incorporati…
Finding an optimal parameter of a black-box function is important for searching stable material structures and finding optimal neural network structures, and Bayesian optimization algorithms are widely used for the purpose. However, most of existing Bayesian optimization algorithms can only handle vector data and canno…
Survey of Bayesian learning for neural networks.
problem Limitations of Bayesian learning in practical applications.
method Introduction to Bayesian Neural Networks and algorithms for inference.
result Discussion of standard and recent approaches for Bayesian inference in neural networks.
Deep learning and genetic algorithms speed up cosmological Bayesian inference.
problem Substantial computational demands in Bayesian inference for cosmological parameter estimation.
method Deep learning using feedforward neural networks to approximate likelihood functions dynamically, optimized with genetic algorithms.
result Significant speed-up in Bayesian inference process for cosmological models and datasets.
Bayesian optimisation algorithm for unknown search spaces with sub-linear regret.
problem Efficient optimisation of expensive black-box functions in unknown search spaces.
method Expands search space over iterations based on a hyperharmonic series, scales to high dimensions.
result Sub-linear regret growth for both algorithms.
Library learns Bayesian networks from mixed data without discretization.
problem Learning Bayesian networks from mixed data (discrete and continuous variables).
method Proposes an algorithm for structural and parameter learning of Bayesian networks from mixed data using a mixed MI score function and Gaussian approximation. Offers two graph structure enumeration algorithms.
result Advantages in solving approximation and gap recovery problems on synthetic and real datasets.
Bayesian algorithm improves best-arm identification within fixed budget.
problem Maximizing probability of identifying optimal arm within fixed budget.
method Proposes Bayesian elimination algorithm and derives upper bound on misidentification probability.
result Upper bound on misidentification probability reflects prior quality and matches lower bound.
Bayesian Reinforcement Learning (RL) is capable of not only incorporating domain knowledge, but also solving the exploration-exploitation dilemma in a natural way. As Bayesian RL is intractable except for special cases, previous work has proposed several approximation methods. However, these methods are usually too sen…
Bayesian bandit algorithms with approximate inference improve regret bounds in stochastic linear bandits.
problem Theoretical justification for Bayesian bandit algorithms with approximate inference in stochastic linear bandits.
method Proposed a theoretical framework to analyze approximate inference impact and conducted frequentist regret analysis on LinTS and LinBUCB.
result LinTS and LinBUCB preserve their original regret upper bounds with larger constant terms in approximate inference settings.
Bayesian HMM for protein alignment state estimation.
problem Estimating the MAP state sequence for a Bayesian HMM.
method Training data with Dirichlet priors, iterative algorithms for MAP path estimation.
result Bayesian setup outperforms frequentist estimation in protein alignment.
New algorithms accelerate SVGD convergence using deep unfolding.
problem Improving the speed of SVGD convergence.
method Integrating deep unfolding into SVGD for parameter learning.
result Proposed algorithms achieve faster convergence in various tasks.
Bayesian algorithm improves word representations using semantic taxonomy.
problem Improving word representations in semantic taxonomy.
method Bayesian Hierarchical Words Representation (BHWR) learning algorithm combining Variational Bayes and semantic taxonomy modeling.
result BHWR produces better representations for rare words.
Bayesian Algorithm Execution uses mutual information to infer properties of black-box functions efficiently.
problem Estimating computable properties of expensive black-box functions with limited evaluations.
method Sequentially choosing queries that maximize mutual information with respect to the algorithm's output.
result InfoBAX reduces query counts by up to 500 times compared to the original algorithm.
Semiparametric Bayesian networks combine parametric and nonparametric models for flexible data analysis.
problem Combining the advantages of parametric and nonparametric models for flexible data analysis.
method Semiparametric Bayesian networks combining parametric and nonparametric conditional probability distributions. Modifications of two algorithms for structure learning from data.
result Accurately learns the combination of parametric and nonparametric components, comparable to state-of-the-art methods.
New algorithms for Bayesian inference in decentralized learning.
problem Bayesian inference in decentralized learning settings.
method Decentralized SGLD and Decentralized SGHMC.
result Convergence of iterates to target distribution in 2-Wasserstein distance.
Paper improves Bayesian inference in federated learning with new algorithm VR-FALD*.
problem Bayesian inference in federated learning with communication bottlenecks and statistical heterogeneity.
method Federated Averaging Langevin Dynamics (FALD) and VR-FALD*.
result VR-FALD* corrects client drift due to statistical heterogeneity, improving convergence.
Bayesian neural network models improve uncertainty quantification in multivariate regression.
problem Uncertainty quantification in multivariate regression models with heteroscedastic noise.
method Proposes Bayesian Last Layer neural network models and EM algorithms for parameter learning.
result Capable of disentangling aleatoric and epistemic uncertainty.
Bayesian optimization has shown to be a fundamental global optimization algorithm in many applications: ranging from automatic machine learning, robotics, reinforcement learning, experimental design, simulations, etc. The most popular and effective Bayesian optimization relies on a surrogate model in the form of a Gaus…
This paper introduces a set of algorithms for Monte-Carlo Bayesian reinforcement learning. Firstly, Monte-Carlo estimation of upper bounds on the Bayes-optimal value function is employed to construct an optimistic policy. Secondly, gradient-based algorithms for approximate upper and lower bounds are introduced. Finally…
A new two-step MH method for Bayesian EL computation.
problem Complex likelihood support in Bayesian EL.
method Hierarchical Metropolis Hastings with reversible jump MCMC.
result Improved sampling from BayesEL posteriors.
Coherent uncertainty quantification is a key strength of Bayesian methods. But modern algorithms for approximate Bayesian posterior inference often sacrifice accurate posterior uncertainty estimation in the pursuit of scalability. This work shows that previous Bayesian coreset construction algorithms---which build a sm…
In literature there are several studies on the performance of Bayesian network structure learning algorithms. The focus of these studies is almost always the heuristics the learning algorithms are based on, i.e. the maximisation algorithms (in score-based algorithms) or the techniques for learning the dependencies of e…
This paper extends the work in [Suzuki, 1996] and presents an efficient depth-first branch-and-bound algorithm for learning Bayesian network structures, based on the minimum description length (MDL) principle, for a given (consistent) variable ordering. The algorithm exhaustively searches through all network structures…
We introduce a means of automating machine learning (ML) for big data tasks, by performing scalable stochastic Bayesian optimisation of ML algorithm parameters and hyper-parameters. More often than not, the critical tuning of ML algorithm parameters has relied on domain expertise from experts, along with laborious hand…
FEDHC learns Bayesian networks efficiently for continuous data.
problem Efficiently learning Bayesian networks for continuous data.
method Forward Early Dropping Hill Climbing (FEDHC) algorithm.
result FEDHC is computationally efficient and produces accurate Bayesian networks.
Bayesian models offer great flexibility for clustering applications---Bayesian nonparametrics can be used for modeling infinite mixtures, and hierarchical Bayesian models can be utilized for sharing clusters across multiple data sets. For the most part, such flexibility is lacking in classical clustering methods such a…
Proposes a new algorithm for learning continuous-time Bayesian network structures.
problem Lack of constraint-based algorithms for continuous-time Bayesian networks.
method Develops a constraint-based algorithm using statistical tests for conditional independence.
result The proposed algorithm is more accurate with variables having more than two values.