The paper develops scalable variational inference for Bayesian neural networks under model and parameter uncertainty.
problem Combining structural and parameter uncertainties in scalable Bayesian neural networks.
method Adapted variational inference with reparametrization for model space constraints.
result Comparable accuracy with sparse inference compared to ordinary BNNs.
Paper introduces combining model and parameter uncertainty in BNNs.
problem Combining model and parameter uncertainty in scalable BNNs.
method Adapted variational inference with reparametrization for model space constraints.
result Sparsification of BNNs structure via Bayesian model averaging and selection.
We present a scalable approach to performing approximate fully Bayesian inference in generic state space models. The proposed method is an alternative to particle MCMC that provides fully Bayesian inference of both the dynamic latent states and the static parameters of the model. We build up on recent advances in compu…
New coin sampling method for Bayesian inference without learning rates.
problem Scalable Bayesian inference with learning rate tuning issues.
method Coin sampling for gradient-based Bayesian inference.
result Comparable performance to other ParVI algorithms without learning rate tuning.
We develop a scalable method for Bayesian neural networks with stochastic differential equations.
problem Uncertainty quantification in deep neural networks.
method Gradient-based stochastic variational inference in continuous-depth Bayesian neural networks.
result Gradient estimator with zero variance as the approximation improves.
Bayesian coresets improve scalable Bayesian inference.
problem Efficiently approximating posterior inference with a subset of data.
method Sparsity constrained optimization and accelerated optimization methods.
result Explicit convergence rate guarantees and superior performance compared to state-of-the-art.
Stochastic Bayesian Neural Network improves scalability and performance.
problem Challenges in calculating posterior distribution in Bayesian Neural Networks.
method Maximizes Evidence Lower Bound using Stochastic Evidence Lower Bound objective function.
result Demonstrates improved performance and scalability over previous algorithms.
URSABench benchmarks Bayesian methods for deep learning models.
problem Scalability issues in Bayesian inference for deep learning.
method Open-source benchmark suite for assessing approximate Bayesian inference methods.
result Initial results show promise for addressing uncertainty and robustness in deep learning.
Explosive growth in data and availability of cheap computing resources have sparked increasing interest in Big learning, an emerging subfield that studies scalable machine learning algorithms, systems, and applications with Big Data. Bayesian methods represent one important class of statistic methods for machine learni…
Bayesian inference engines improve density estimation accuracy and scalability.
problem Constructing accurate and scalable probability density functions.
method Bayesian inference engines (no-U-turn sampling and expectation propagation) with binning strategy.
result Density estimates have excellent comparative performance and scale well to large sample sizes.
Bayesian Hierarchical Invariant Prediction refines ICP for better scalability and prior integration.
problem Improving computational scalability and invariance testing for causal inference.
method Bayesian Hierarchical structure to test invariance under heterogeneous data.
result Demonstrated improved scalability and potential as an alternative to ICP.
Scalable model checking for stochastic systems using Gaussian Processes and Bayesian Neural Networks.
problem Efficiently verifying properties of stochastic systems with high-dimensional parameter spaces.
method Stochastic Variational Smoothed Model Checking (SV-smMC) using Gaussian Processes and Bayesian Neural Networks.
result SV-smMC scales to larger datasets and enables application to high-dimensional parameter spaces.
Automated Bayesian coreset construction for scalable inference.
problem Designing scalable and theoretically sound automated inference algorithms.
method Sparsity-constrained variational inference within an exponential family.
result Automated Bayesian coreset construction with improved KL divergence to the exact posterior.
Method scales Bayesian inference to large datasets and robustifies against outliers.
problem Scalability and robustness to outliers in Bayesian inference.
method Variational inference with β-divergence and Riemannian coresets. result Efficiently constructs cleansed data summaries robust to outliers.
Develops a flexible deep autoencoding topic model with scalable hybrid Bayesian inference.
problem Flexible and interpretable document analysis models.
method DATM with hybrid Bayesian inference, including topic-layer-adaptive stochastic gradient Riemannian MCMC and Weibull variational encoder.
result Demonstrates scalability and efficacy on big corpora in unsupervised and supervised learning tasks.
New method calibrates LLMs for safety-critical tasks with scalable Bayesian inference.
problem Overconfidence in LLMs after fine-tuning for specific tasks.
method Orthogonalized Low-Rank Adapters (PoLAR) with variational Bayesian inference.
result Scalable and well-calibrated uncertainty estimation for LLMs.
A new method improves Bayesian deep learning by balancing scalability and accuracy.
problem Scalability issues in Bayesian neural networks.
method Collapsed inference scheme that performs Bayesian model averaging using collapsed samples.
result Significant improvements over existing methods in predictive performance and uncertainty estimation.
Scalable GAMs using sparse variational Gaussian processes.
problem Flexible modeling of data beyond linear models.
method Bayesian treatment of GAMs using Gaussian processes (GPs) with sparse representation and additive structure.
result Efficient and well-calibrated Bayesian treatment of GAMs.
Bayesian approach improves performance in Gaussian process models.
problem Scalable posterior estimation in Gaussian process models.
method Revisiting variational inference techniques with Bayesian treatment of inducing variables and hyper-parameters.
result State-of-the-art performance demonstrated across various regression and classification problems.
Bayesian method predicts individual and crowd preferences from small data.
problem Difficult to predict preferences from limited personal data and noisy labels.
method Combines matrix factorization with Gaussian processes for scalable inference.
result Method predicts preferences for new users and items not in training set.
This work connects BNNs to GPs, providing scalable inference and identifying key properties.
problem Scaling and inference challenges in Bayesian neural networks.
method General convergence from BNNs to GPs, new covariance function, and scalable Nyström approximation.
result Established a scalable maximum a posterior (MAP) training and prediction procedure.
Deep model learns complex latent codes without assuming factor structure.
problem Learning latent codes with complex, non-factorial distributions.
method Deep generative factor analysis with beta process prior and stochastic EM algorithm.
result Preliminary results show model can approximate complex distributions.
Celeste is a procedure for inferring astronomical catalogs that attains state-of-the-art scientific results. To date, Celeste has been scaled to at most hundreds of megabytes of astronomical images: Bayesian posterior inference is notoriously demanding computationally. In this paper, we report on a scalable, parallel v…
We present a novel, scalable and Bayesian approach to modelling the occurrence of pairs of symbols (i,j) drawn from a large vocabulary. Observed pairs are assumed to be generated by a simple popularity based selection process followed by censoring using a preference function. By basing inference on the well-founded pri…
Paper tackles scalable VFL with data augmentation and amortized inference.
problem Collaborative model estimation across multiple clients with distinct covariates.
method Data augmentation, amortized variational approximation, factorized likelihoods.
result Scalable Bayesian VFL framework for various models.
This work simplifies Bayesian inference for neural networks by identifying influential parameter directions.
problem High computational complexity in Bayesian inference for neural networks due to high-dimensional parameter space.
method Constructing an active subspace of influential parameter directions to reduce dimensionality.
result Effective and scalable Bayesian inference achieved via reduced active subspace.
VMoER improves uncertainty quantification in MoE layers for scalable foundation models.
problem Uncertainty quantification in large-scale models like MoE layers.
method Structured Bayesian approach with amortized variational inference over routing logits and temperature parameter inference.
result Improves routing stability, reduces calibration error, and increases AUROC by 12%.
Proposes MOPED method for choosing priors in Bayesian DNNs.
problem Challenges in specifying meaningful priors for deep neural networks.
method Two-stage hierarchical modeling with empirical Bayes.
result MOPED enables scalable variational inference and reliable uncertainty quantification.
Datasets are growing not just in size but in complexity, creating a demand for rich models and quantification of uncertainty. Bayesian methods are an excellent fit for this demand, but scaling Bayesian inference is a challenge. In response to this challenge, there has been considerable recent work based on varying assu…
ABI adapts to graph data for fast, scalable inference.
problem Challenges in inference on graph-structured data.
method Amortized Bayesian Inference (ABI) framework for graph data.
result ABI successfully addresses challenges in graph data inference.
A scalable Bayesian linear regression framework for spatial data.
problem Scalable methodologies for analyzing large spatial datasets.
method Conjugate Bayesian linear regression framework.
result Exact sampling from joint posterior distribution without iterative algorithms.
Develops scalable Bayesian inference methods for neural networks.
problem Lack of model uncertainty in deep learning leading to overconfident predictions.
method Linearised Laplace approximation, conjugate Gaussian-linear models, stochastic gradient descent, sample-based EM algorithm.
result Equips neural networks with model uncertainty using scalable methods.
PHP connects to ReLU neural networks for scalable Bayesian inference.
problem Scalability and Bayesian inference in two-layer ReLU neural networks.
method PHP with Gaussian prior, decomposition propositions, annealed sequential Monte Carlo.
result PHP provides an alternative scalable representation for two-layer ReLU neural networks.
We present a hybrid algorithm for Bayesian topic models that combines the efficiency of sparse Gibbs sampling with the scalability of online stochastic inference. We used our algorithm to analyze a corpus of 1.2 million books (33 billion words) with thousands of topics. Our approach reduces the bias of variational infe…
Bayesian methods improve inference for cumulative probit models on large datasets.
problem Challenges in Bayesian inference for large cumulative probit models.
method Proposed scalable algorithms using Variational Bayes and Expectation Propagation.
result Superior computational performance and accuracy compared to MCMC.
A scalable Bayesian inference method for mixed-effects models in systems biology.
problem Scalable Bayesian inference for complex hierarchical mixed-effects models in systems biology.
method Constructing amortized approximations of likelihood and posterior distributions, refined for each individual dataset.
result Our method is both fast and competitive in statistical accuracy compared to exact pseudomarginal Bayesian inference.
NPE improves scalability and efficiency for ERGMs.
problem Scalability and efficiency issues in Bayesian ERGM estimation.
method Neural posterior estimation (NPE) for ERGMs using neural network density estimation.
result NPE provides more efficient and scalable inference for ERGMs.
BLISS detects and separates astronomical sources quickly and accurately.
problem Detecting and separating overlapping astronomical sources in large images.
method Bayesian Light Source Separator (BLISS) using deep generative models and variational inference.
result BLISS can process megapixel images in seconds and produce highly accurate catalogs.
Bayesian neural networks improved with scalable approximate inference.
problem Performing approximate Bayesian inference in complex models like neural networks.
method Two models: primary for prediction, secondary for posterior approximation; optimised via gradient descent on posterior predictive distribution.
result Approach scales better than MCMC and more expressive than VIs, without adversarial training.
Bayesian hypergraph inference models disease pathways from EHR data.
problem Modeling rare diseases influenced by shared risk factors.
method Bayesian hypergraph inference framework reframing multi-disease modeling.
result Interpretable disease pathways and well-calibrated uncertainty quantification.
The paper proposes a scalable framework for uncertainty quantification and propagation in surrogate-based Bayesian inference.
problem Uncertainty in surrogate models and its impact on inference and decision-making.
method Bayesian inference methods for surrogate models with measurement data.
result Scalable framework for uncertainty quantification and propagation in surrogate models.
Paper introduces deep structured mixtures of Gaussian processes for scalable GP approximations.
problem Scalability issues with Gaussian Processes (GPs).
method Deep structured mixtures of GP experts for scalable approximate inference.
result Deep structured mixtures provide better predictive uncertainties and competitive performance.
SwISS improves scalability of Bayesian inference for large datasets.
problem Scalability issues in Bayesian inference for large datasets.
method Divide-and-conquer approach with SwISS for recombining sub-posterior samples.
result SwISS accurately approximates the original posterior distribution.
A scalable method for Bayesian inference in large linear models.
problem High computational cost in Bayesian linear models for large networks.
method Sample-based inference and g-prior for hyperparameter selection.
result Linearised neural network inference on large datasets (ResNet-18, ResNet-50, U-Net).
Develops scalable model for learning velocity fields in complex traffic scenarios.
problem Learning heterogeneous and dynamic velocity fields in complex traffic scenarios.
method Nonparametric Bayesian modeling with hierarchical Dirichlet process and infinite hidden Markov model, Gaussian process prior, and scalable approximate inference.
result Demonstrates effective scalability and applicability to real-world traffic data.
Bayesian neural networks simplified with input augmentation.
problem Uncertainty in deep learning models.
method Layer-wise input augmentation to induce uncertainty distributions.
result State-of-the-art performance in uncertainty representation.
VPR improves posterior uncertainty quantification by combining VI and predictive resampling.
problem Inaccurate posterior sampling with MCMC due to computational constraints.
method Variational predictive resampling (VPR) that uses VI's predictive strength and imputes future observations.
result VPR converges to the exact Bayesian posterior in a Gaussian location model and improves uncertainty quantification.
Scalable Bayesian LASSO using variational inference for large p and n.
problem Large-scale regression with sparsity constraints.
method Variational Bayesian inference with scale mixtures of Normal distributions.
result The method achieves comparable performance to Bayesian LASSO but at a lower computational cost.