Bayesian optimization (BO) is a model-based approach for gradient-free black-box function optimization. Typically, BO is powered by a Gaussian process (GP), whose algorithmic complexity is cubic in the number of evaluations. Hence, GP-based BO cannot leverage large amounts of past or related function evaluations, for e…
Paper proposes a new method for Bayesian linear regression using spike-and-slab priors.
problem Identifying predictors with similar relationships in linear regression models.
method Hierarchical Bayesian models with spike-and-slab priors and a Gibbs sampler.
result The proposed method outperforms previous methods in simulations and real data analysis.
New methods combine model predictions to avoid linear mixtures' limitations.
problem Combining predictions from different models to avoid linear mixtures' limitations.
method Log-linear pooling (locking) and quantum superposition (quacking) to optimise model weights.
result Demonstrated locking method with illustrative example and practical application.
Bayesian MAML outperforms MAML in meta learning tasks with theoretical guarantees.
problem Theoretical understanding of Bayesian MAML's superiority over MAML.
method Comparison of meta test risks between Bayesian MAML and MAML in meta linear regression.
result Bayesian MAML has provably lower meta test risks than MAML in both distribution agnostic and linear centroid cases.
Paper develops robust Bayesian models for linear regression under adversarial perturbations.
problem Ensuring reliable machine learning models under data perturbations.
method Formulates adversarial Bregman divergence loss, computes adversarial perturbation, introduces adversarially robust posteriors, derives generalization certificates.
result Derives first rigorous generalization certificates for adversarially robust Bayesian linear regression.
This paper improves Bayesian neural nets by using local linearization.
problem Underfitting in Bayesian neural networks.
method Local linearization of Bayesian neural networks to create a generalized linear model (GLM) for predictions.
result The GLM predictive resolves common underfitting problems of the Laplace approximation.
Work proposes a new framework to improve uncertainty estimation in deep Bayesian models.
problem Traditional training procedures underestimate uncertainty in NLMs, leading to unreliable predictions.
method Introduces a novel training framework that captures useful predictive uncertainties for out-of-distribution inputs.
result Demonstrates that traditional methods for NLMs significantly underestimate uncertainty and propose a new framework to address this issue.
A novel Bayesian framework for private linear regression with MCMC.
problem Private linear regression in a distributed setting.
method Generative statistical model, MCMC algorithms, fast Bayesian estimation.
result The proposed methods provide well-rounded estimation and prediction.
New method improves uncertainty estimation in Bayesian deep learning models.
problem Underestimation of predictive uncertainty in Neural Linear Models (NLMs).
method Proposes a novel training method to capture useful predictive uncertainties and incorporate domain knowledge.
result Traditional training procedures for NLMs can drastically underestimate uncertainty in data-scarce regions.
Simple linear models outperform complex BO methods in high dimensions.
problem Overcoming the curse of dimensionality in Bayesian optimization.
method Bayesian linear regression with linear kernels, applied to high-dimensional search spaces.
result Simple linear models match or outperform state-of-the-art BO methods in high-dimensional tasks.
BayesBoost combines boosting and Bayesian methods for linear mixed models, improving uncertainty estimation and variable selection.
problem Lack of straightforward uncertainty estimation for parameters in high-dimensional linear mixed models.
method BayesBoost: Combines boosting and Bayesian inference for linear mixed models.
result Improves uncertainty estimation and variable selection in linear mixed models.
Solla discusses neural processing using statistical physics and Bayesian methods.
problem Understanding neural information processing through statistical physics.
method Bayesian inference, Gibbs description, Generalized Linear Models, dimensionality reduction.
result Connection between neural processing and statistical physics.
A scalable method for Bayesian inference in large linear models.
problem High computational cost in Bayesian linear models for large networks.
method Sample-based inference and g-prior for hyperparameter selection.
result Linearised neural network inference on large datasets (ResNet-18, ResNet-50, U-Net).
As an automatic method of determining model complexity using the training data alone, Bayesian linear regression provides us a principled way to select hyperparameters. But one often needs approximation inference if distribution assumption is beyond Gaussian distribution. In this paper, we propose a Bayesian linear reg…
New method uses diffusion models for Bayesian inverse problems.
problem Solving Bayesian inverse problems with linear-Gaussian models.
method Decoupled Diffusion Sequential Monte Carlo (DDSMC) method.
result Asymptotically exact solution demonstrated on various data types.
BAVART model combines VAR and BART for non-linear forecasting.
problem Overly restrictive linearity assumption in VAR models.
method Combining VAR with Bayesian additive regression trees (BART).
result BAVART model yields highly competitive forecasts.
Matrix completion aims to predict missing elements in a partially observed data matrix which in typical applications, such as collaborative filtering, is large and extremely sparsely observed. A standard solution is matrix factorization, which predicts unobserved entries as linear combinations of latent variables. We g…
Bayesian optimization improved for biased data.
problem Adversarial bias in observations, especially hidden confounders.
method Reduction to dueling bandits, information-directed sampling (IDS).
result First efficient kernelized algorithm with regret guarantees.
The article describe the model, derivation, and implementation of variational Bayesian inference for linear and logistic regression, both with and without automatic relevance determination. It has the dual function of acting as a tutorial for the derivation of variational Bayesian inference for simple models, as well a…
The neural linear model is a simple adaptive Bayesian linear regression method that has recently been used in a number of problems ranging from Bayesian optimization to reinforcement learning. Despite its apparent successes in these settings, to the best of our knowledge there has been no systematic exploration of its …
Bayesian linear networks reveal optimal depth and width trade-offs.
problem Understanding how depth, width, and dataset size affect model quality in linear networks.
method Zero noise Bayesian inference with Gaussian weight priors and mean squared error.
result Optimal predictions at infinite depth and maximized Bayesian model evidence at infinite depth.
This dissertation uses ILP to learn Bayesian network structures efficiently.
problem Learning the structure of Bayesian networks from data.
method Integer Linear Programming formulation with cluster constraints and cutting planes.
result The approach finds feasible solutions for Bayesian network structures efficiently.
Revises Bayesian model averaging for foundation models.
problem Ensemble pre-trained and lightly-finetuned foundation models for improved classification performance.
method Introduces trainable linear classifiers and computationally cheaper model averaging scheme (OMA).
result Ensembled models can better predict on various datasets.
Study forecasts stock returns on JSE using SGDLMs capturing cross-series dependencies.
problem Accurate forecasting of multivariate time series data.
method Simultaneous Graphical Dynamic Linear Models (SGDLMs) with customised DLMs and importance sampling/mean-field variational Bayes.
result SGDLMs accurately forecast stock data on JSE and respond to market changes.
In this work, we study the use of logistic regression in manufacturing failures detection. As a data set for the analysis, we used the data from Kaggle competition Bosch Production Line Performance. We considered the use of machine learning, linear and Bayesian models. For machine learning approach, we analyzed XGBoost…
Robust Bayesian models are appealing alternatives to standard models, providing protection from data that contains outliers or other departures from the model assumptions. Historically, robust models were mostly developed on a case-by-case basis; examples include robust linear regression, robust mixture models, and bur…
New method corrects Laplace/BIC errors in singular models, revealing effective dimension.
problem Laplace/BIC errors in singular models due to incorrect effective dimension assumption.
method RLCT (real log canonical threshold) to correct effective dimension in linear models.
result Correct evidence slope and effective dimension estimation in linear settings.
IDS improves sparse linear bandits by balancing information and regret.
problem Sparse linear bandits in high-dimensional decision-making.
method Information-directed sampling (IDS) with Bayesian regret bounds and empirical Bayesian sparse posterior sampling.
result IDS nearly matches existing lower bounds and significantly reduces regret.
FBMS R package simplifies Bayesian model selection and averaging.
problem Complex regression settings with multi-modal posterior landscapes.
method Efficient MJMCMC and GMJMCMC algorithms for Bayesian model exploration.
result FBMS effectively handles Bayesian generalized linear and nonlinear models.
RNN-HAR model improves VaR forecasting with long-memory and non-linear dynamics.
problem Efficiently forecasting Value at Risk (VaR) with long-memory and non-linear realized volatility.
method Loss-based generalized Bayesian inference with Sequential Monte Carlo for model estimation and prediction.
result RNN-HAR model consistently outperforms other VaR forecasting models.
A scalable Bayesian linear regression framework for spatial data.
problem Scalable methodologies for analyzing large spatial datasets.
method Conjugate Bayesian linear regression framework.
result Exact sampling from joint posterior distribution without iterative algorithms.
Paper introduces VBG for Bayesian causal structure and mechanism learning.
problem Bayesian causal structure learning with uncertainty over models.
method Variational Bayes-DAG-GFlowNet (VBG) method.
result VBG outperforms existing methods in modeling posterior over DAGs and mechanisms.
Bayesian method tests Granger causality in functional time series.
problem Testing Granger causality between functional time series.
method Bayesian dynamic linear models (DLM) and Bayes Factor.
result Captures Granger causality between yield curves and weather conditions.
Bayesian method selects subsets for LMMs with structured dependence.
problem Subset selection challenge in LMMs with structured dependence.
method Bayesian decision analysis with Mahalanobis loss function.
result Optimal linear coefficients for subsets and cardinality constraints.
Thompson sampling used for linear bandits with normal-gamma priors.
problem Optimizing decisions in uncertain environments with linear dependencies and unknown parameters.
method Bayesian Thompson sampling with multivariate normal-gamma priors.
result Derivation of a Bayesian regret bound for the approach.
Bayesian bandit algorithms with approximate inference improve regret bounds in stochastic linear bandits.
problem Theoretical justification for Bayesian bandit algorithms with approximate inference in stochastic linear bandits.
method Proposed a theoretical framework to analyze approximate inference impact and conducted frequentist regret analysis on LinTS and LinBUCB.
result LinTS and LinBUCB preserve their original regret upper bounds with larger constant terms in approximate inference settings.
Bayesian nonparametric machine learning improves instrumental variable inference.
problem Estimating causal effects with nonlinear relationships.
method Bayesian Additive Regression Trees (BART) for estimating functions and Dirichlet Process mixtures for error terms.
result Dramatic improvements in inference with nonlinear data, no manual tuning required.
Bayesian approach learns linear networks from high-dimensional data.
problem Learning high-dimensional linear Bayesian networks.
method Iterative estimation of topological ordering and parents using inverse partial covariance matrix with Bayesian regularization.
result The method successfully recovers network structure under certain conditions.
MCGDiff uses SGM to guide SMC for solving ill-posed linear inverse problems.
problem Solving ill-posed linear inverse problems in Bayesian settings.
method Exploiting SGM structure, defining a sequence of intermediate problems, and using SMC methods.
result MCGDiff outperforms competing methods in Bayesian ill-posed inverse problems.
A fast MCMC sampler for sparse Bayesian inference.
problem Sparse Bayesian inference problems with high computational cost.
method Asynchronous Gibbs sampler extended with data sub-sampling.
result The Markov chain admits an invariant distribution that recovers the main signal with high probability.
Study clarifies Bayesian generalization error in CBM for 3-layered linear neural networks.
problem Understanding the generalization error in concept bottleneck models.
method Mathematical analysis of Bayesian generalization error and free energy in CBM for 3-layered linear neural networks.
result CBM significantly alters the parameter region and Bayesian generalization error compared to standard models.
Develops CLDS models to model neural activity with nonlinear dynamics.
problem Complex, nonlinear dynamics in neural population activity.
method Conditionally Linear Dynamical System (CLDS) models using Gaussian Process (GP) priors.
result CLDS models can perform well even in data-limited conditions.
Bayesian method clusters time series with varying dynamics.
problem Modeling and clustering time series with unknown number of clusters and dynamics.
method Hierarchical Dirichlet process and Gaussian process for modeling time series patterns and variations.
result Efficiently clusters time series with varying dynamics without unnecessary proliferation of clusters.
QLA improves Bayesian uncertainty estimation for DNNs without increasing computational cost.
problem Overconfident out-of-distribution predictions from DNNs.
method Proposes Quadratic Laplace Approximation (QLA) to improve Bayesian uncertainty quantification.
result QLA yields modest yet consistent uncertainty estimation improvements over Linearized Laplace Approximation (LLA) on five regression datasets.
BART and MOTR-BART improve tree-based predictions with local linear models.
problem Non-linearity and high-order interactions in data.
method Bayesian Additive Regression Trees (BART) and Model Trees BART (MOTR-BART) using piecewise linear functions.
result MOTR-BART achieves equal or better performance with fewer trees than BART.
Bayesian inverse problems solved with Gaussian models for PDEs.
problem Solving inverse problems with limited data for PDEs.
method Constructing PDE-informed Gaussian priors for Bayesian inversion.
result PDE-informed Gaussian priors outperform traditional priors.
Bayesian optimization sped up to linear time.
problem Expensive function evaluations and cubic computational complexity.
method Flexible binary partitioning of the search space.
result Linear computational complexity and superior optimization performance.
Automates model selection for GLMs using optimization.
problem Automating model selection for generalized linear models.
method Mixed-integer conic optimization for feature subset selection and model constraints.
result Optimization of AIC and BIC criteria with multicollinearity constraints.