The paper connects a proximal method to stochastic filters and Bayes updates.
problem Large-scale optimization and probabilistic methods for regression.
method Explicit form of Bayes updates for linear regression and general sequential setting.
result The incremental proximal method can be realized by the Kalman filter for linear-quadratic cost functions.
New PAC-Bayes method updates priors without losing confidence information.
problem Lack of sequential prior updates in PAC-Bayes without losing confidence information.
method Recursive PAC-Bayes decomposition of expected loss.
result Sequential prior updates with no information loss.
New method handles unknown task boundaries in continual learning.
problem Catastrophic forgetting in neural networks.
method Fixed-point equations for online variational Bayes optimization.
result Approximates online Bayes update for non-stationary data.
New decision-theoretic characterization separates belief and decision posteriors.
problem Understanding the conditions under which loss-based updating coincides with Bayesian updating.
method Decision-theoretic approach to distinguish belief and decision posteriors.
result Generalized Bayes coincides with ordinary Bayesian updating only if the loss is proportional to negative log-likelihood.
Study variational Bayes for high-dimensional linear regression with sparse priors.
problem Sparse high-dimensional linear regression model selection.
method Mean-field spike and slab variational Bayes approximation, oracle inequalities, coordinate-ascent variational inference (CAVI), prioritized updating scheme.
result Mean-field VB approximation converges to the sparse truth at optimal rate and gives optimal prediction.
Test-time training adapts a pretrained model to each prompt via parameter updates, improving accuracy under pretraining-to-test distribution shifts.
problem Improving accuracy of pretrained models under distribution shifts.
method Explaining TTT behavior through a decision-theoretic lens.
result TTT reduces prediction error when updates are spectrally matched to the prompt's signal-to-noise ratio and aligned with query-relevant eigen-directions.
A new optimization algorithm for Gaussian Variational Inference on precision matrices.
problem Complex models with positive definite constraints on covariance matrices.
method Manifold Gaussian Variational Bayes (MGVBP) with natural gradient updates.
result Empirically validated as a feasible and efficient solution for VI in complex models.
Simplified Variational Bayes for easier inference.
problem Complex derivation of Variational Bayes.
method 3-step recipe to identify posterior form and directly write updates.
result Easier, faster, shorter derivation of Variational Bayes.
Bayesian framework reduces online optimization regret.
problem Sequential optimization in dynamic environments with bounded losses.
method PAC-Bayes theory and Bayesian updating principles.
result Achieves O ( T ) \mathcal{O}(\sqrt{T}) O ( T ) regret for bounded losses. New priors can update posteriors without re-estimating likelihoods.
problem Degradation of classification approaches when class priors change.
method Recompute posteriors using recovered likelihoods from original posteriors and new priors.
result Dynamic update of original posteriors is possible without re-estimating likelihoods.
New approach to meta-learning with variational Bayes for unlabeled data.
problem Improving machine learning systems' adaptability to small data.
method Generative meta-learning using AEVB and mean-field variational Bayes.
result Relevant VB updates do not depend on generative neural networks for certain models.
A method for selecting pseudo-labeled data in semi-supervised learning using generalized Bayes and soft revision.
problem Selecting pseudo-labeled data for semi-supervised learning with robustness to uncertainty.
method Using credal sets and the Gamma-Maximin method with soft revision to update priors and select pseudo-labeled data.
result The Gamma-Maximin method with soft revision can achieve promising results, especially in scenarios with low labeled data proportions.
We present SDA-Bayes, a framework for (S)treaming, (D)istributed, (A)synchronous computation of a Bayesian posterior. The framework makes streaming updates to the estimated posterior according to a user-specified approximation batch primitive. We demonstrate the usefulness of our framework, with variational Bayes (VB) …
A novel Bayesian computation method using importance weighting improves numerical stability and performance.
problem Bayesian computation stability and performance issues.
method Nonparametric approach via feature means, importance weighting, and kernel Bayes' rule.
result Importance weighted kernel Bayes' rule yields superior numerical stability and performance.
New PAC-Bayes training method improves model generalization for unbounded loss.
problem Improving generalization of complex models under unbounded loss.
method Established new PAC-Bayes bound for unbounded loss, jointly training prior and posterior.
result Outperforms existing PAC-Bayes training algorithms and matches ERM accuracy.
A framework uses variational Bayes for solving inverse problems efficiently.
problem Solving inverse problems in various dimensions with flexibility and accuracy.
method Variational Bayes approximations with message passing and factor graph approach.
result Efficient algorithm updates for higher dimensions and computational advantage over MCMC.
A new framework for clustering with uncertainty quantification.
problem Lack of uncertainty quantification in clustering methods.
method Generalized Bayes framework using Gibbs posteriors and loss functions.
result Efficient algorithms for clustering and uncertainty quantification.
Bayes-CPACE optimally explores continuous BAMDPs.
problem Model uncertainty in continuous state and action spaces.
method Covering state-belief-action space with samples, exploiting Lipschitz continuity.
result Near-optimal value function computed efficiently.
Efficiently identifies important variables in binary outcomes using variational Bayes.
problem Bayesian variable selection for binary outcomes with computational challenges.
method Mean-field variational Bayes approximation with closed-form updates and efficient inference algorithm.
result Successfully identifies important variables and is orders of magnitude faster than MCMC.
Bayes-assisted confidence sequences improve efficiency for bounded means.
problem Efficient uncertainty quantification for bounded IID means without parametric assumptions.
method Bayesian working predictive model selects adaptive martingale updates maximizing predictive log-growth.
result Asymptotically log-optimal performance with informative priors reducing width and sampling effort.
Bayesian unlearning uses Bayes' rule to remove data from a model, but faces challenges in obtaining the exact posterior.
problem Removing data from a trained model while maintaining model accuracy.
method Uses Laplace approximation and Variational Inference to approximate the updated posterior.
result Insights on the applicability of Bayesian unlearning in practical scenarios for neural networks.
Bayesian inference improved with classifier-based misspecification detection and tempering.
problem Model misspecification in Bayesian inference leads to overly concentrated posteriors.
method Probabilistic classifiers trained on simulated vs. observed data to estimate model misspecification and tempering level.
result Estimation of negative KL divergence provides useful diagnostic and update method.
Bayesian framework for policy learning in decision problems.
problem Maximizing expected welfare in decision-making problems.
method Loss-based Bayesian updating and squared-loss surrogate for welfare maximization.
result General Bayes posterior over decision rules with Gaussian pseudo-likelihood interpretation.
Bayes Factor Surprise enables rapid adaptation to changing environments.
problem Learning in volatile, non-stationary stochastic environments.
method Bayesian inference in a hierarchical model with a Bayes Factor Surprise probability ratio.
result Novel surprise-based algorithms improve parameter estimation and performance.
BAM integrates new data while selectively remembering past observations.
problem Slow adaptation and convergence to incorrect parameter values in non-stationary environments.
method Bayes' theorem with adaptive memory selection.
result BAM generalizes and demonstrates continuous adaptation in changing environments.
Quantum annealing improves VB inference, avoiding local minima.
problem Variational Bayes inference stuck in local minima.
method Quantum annealing approach to VB inference.
result Quantum annealing variational Bayes (QAVB) outperforms classical VB.
Fast variational Bayes methods improve geospatial data analysis speed and accuracy.
problem Inaccurate and slow variational Bayes methods for large geospatial data.
method Combination of calculus of variations, closed-form gradient updates, and linear response corrections.
result Comparable accuracy to spNNGP with reduced computational costs and faster speed.
Proposes an efficient algorithm for mHealth that makes real-time physical activity suggestions.
problem Training complex models efficiently in real-time mHealth applications.
method Streamlined empirical Bayes procedure for fitting linear mixed effects models.
result Improves accuracy and speed over state-of-the-art approaches in mHealth applications.
Optimal adaptive experiment for choosing best treatment with binary outcomes.
problem Choosing the best treatment from binary options in an adaptive experiment.
method Adaptive experiment with two phases: treatment allocation and choice. Neyman allocation method used.
result Neyman allocation is minimax and Bayes optimal, matching lower bounds for regret.
Gradient-EM Bayesian meta-learning accelerates adaptation with reduced computation and improved robustness.
problem Efficient and robust adaptation to new tasks with uncertainty assessment.
method Extends Bayesian meta-learning with gradient-EM algorithm, decoupling inner-update from meta-update.
result Improves accuracy with less computation cost and enhanced robustness to uncertainty.
A novel kernel approach for model selection in simulator-based models.
problem Model selection for simulator-based statistical models with limited prior knowledge.
method Iteratively updates model weights and parameters using Bayes' rule and kernel recursive ABC algorithm.
result Demonstrates effectiveness on dynamical systems in ecology and epidemiology.
Robust Kalman filtering method for outlier detection.
problem Outliers and misspecified measurement models in state-space models.
method Combines generalised Bayesian inference with Kalman filters for robustness and efficiency.
result Matches or outperforms other robust filtering methods at lower computational cost.
PROBE algorithm efficiently solves sparse high-dimensional linear regression.
problem Sparse high-dimensional linear regression models with complex parameter spaces.
method Partitioned empirical Bayes ECM algorithm for computationally efficient MAP estimation.
result PROBE algorithm provides robust and efficient coordinate-wise optimization.
Derives VMP for LDA, simplifying inference for topic modeling.
problem Manual derivation of VMP equations for LDA is challenging and time-consuming.
method Detailed derivation of VMP update equations for LDA.
result Enables easier implementation of VMP for LDA models.
Proposes GRU-ODE-Bayes for modeling irregularly observed time series.
problem Challenges of modeling sporadically observed multidimensional time series.
method Continuous-time GRU, Bayesian update network, Fokker-Planck dynamics.
result Demonstrates superior performance on synthetic and real-world data.
Maximizes robustness in Bayesian experimental design under model uncertainty.
problem Brittleness of Bayesian experimental design under model misspecification.
method Formulates as a max--min game, uses Sibson's α-MI, and adopts PAC-Bayes framework.
result Establishes robust belief update and conditional information gain measure.
VAR-GPs solve continual learning by updating posteriors sequentially.
problem Catastrophic forgetting in sequential learning tasks.
method Sparse inducing point approximations and auto-regressive variational distribution.
result VAR-GPs prevent catastrophic forgetting and outperform baselines.
PF-RNNs use particle filtering to model uncertainty in RNNs for better sequential data prediction.
problem Highly variable and noisy sequential data.
method PF-RNNs maintain a latent state distribution as a set of particles, updating with Bayes rule.
result PF-RNNs outperform standard RNNs on various sequence prediction tasks.
We study how to communicate findings of Bayesian inference to third parties, while preserving the strong guarantee of differential privacy. Our main contributions are four different algorithms for private Bayesian inference on proba-bilistic graphical models. These include two mechanisms for adding noise to the Bayesia…
ALBU improves LDA performance on small datasets.
problem Limited data impairs variational Bayes in LDA.
method Approximate Loopy Belief message passing algorithm.
result ALBU learns latent distributions more accurately than VB, especially for smaller data sets.
Bayesian method optimizes interventions for causal discovery.
problem Active interventions are needed for causal discovery when observational data is insufficient.
method Bayesian optimization-based approach using observational data and pre-experimental evaluation of interventions.
result Demonstrated effectiveness through various experiments.
Streaming variational Bayes (SVB) is successful in learning LDA models in an online manner. However previous attempts toward developing online Monte-Carlo methods for LDA have little success, often by having much worse perplexity than their batch counterparts. We present a streaming Gibbs sampling (SGS) method, an onli…
A new concept of confidence in learning is defined and analyzed.
problem Understanding and quantifying trust in learning processes.
method Formal axioms, continuum measures, vector fields, loss functions.
result Confidence can be represented and optimized in learning.
We analyze a model of learning and belief formation in networks in which agents follow Bayes rule yet they do not recall their history of past observations and cannot reason about how other agents' beliefs are formed. They do so by making rational inferences about their observations which include a sequence of independ…
A new method improves Bayesian inference for multimodal posteriors.
problem Insensitivity to well-separated modes in multimodal posteriors.
method Weighted Kernel Stein Discrepancy method.
result Significantly improved mode sensitivity compared to standard KSD-Bayes.
Bayesian neural networks update beliefs with soft evidence, improving accuracy and calibration.
problem Updating neural network weights with uncertain or soft evidence.
method Developed two algorithms to approximate Jeffrey's rule for updating neural network weights.
result Jeffrey-based methods outperform traditional approaches in accuracy and calibration, especially in noisy data.
This work introduces a fixed-point optimization for variational inference.
problem Improving quantified uncertainty in predictions by optimizing a simplified distribution over parameters.
method Projective integral updates for high-dimensional variational inference.
result Efficient quasirandom quadrature sequence for mean-field distributions, leading to quasi-Newton variational Bayes (QNVB).
ITF improves DSR but inflates curvature, while marginal likelihood reduces it, affecting QoIs.
problem Curvature mismatch between teacher forcing and marginal likelihood in chaotic dynamical systems.
method Comparing objective-induced curvatures of ITF and marginal likelihood in a probabilistic switching augmentation of AL-RNNs.
result Curvature inflation by ITF and reduction by marginal likelihood affect dynamical quantities of interest.