TFB simplifies Bayesian LLM uncertainty estimation without extra training.
problem Estimating uncertainty in LLM responses remains challenging.
method Training-Free Bayesianization (TFB) that transforms low-rank adapters into Bayesian ones without additional training.
result TFB achieves superior uncertainty estimation and generalization compared to existing methods.
Bayesian tensor train method recovers streaming data with high accuracy.
problem Recovering high-order, incomplete, and noisy streaming data.
method Bayesian tensor train decomposition using streaming variational Bayes method.
result The proposed SPTT algorithm excels in recovering streaming data compared to state-of-the-art methods.
Unified Bayesian Optimization framework for model selection balancing effectiveness and training efficiency.
problem Balancing model effectiveness and training efficiency in machine learning model selection.
method Proposes a unified Bayesian Optimization framework to jointly optimize model effectiveness and training efficiency.
result Models selected using the proposed framework significantly improve training efficiency while maintaining strong effectiveness.
Wide stochastic networks show Gaussian behavior and improve training with PAC-Bayesian methods.
problem Analyzing and training over-parameterised neural networks with large width.
method Establishing Gaussian behavior for a stochastic architecture, applying PAC-Bayesian training.
result PAC-Bayesian training on large but finite-width networks outperforms standard methods.
Paper tackles model vulnerabilities by reconstructing training data.
problem Reconstructing training data from model parameters poses a security risk.
method Developed a mathematical framework and score matching method for both Bayesian and non-Bayesian models.
result First score matching framework for reconstructing data in Bayesian models.
OOD-trained Bayesian neural networks perform similarly to frequentist methods in uncertainty quantification.
problem Bayesian neural networks struggle in out-of-distribution (OOD) detection tasks.
method Incorporated out-of-distribution data into Bayesian inference through four different methods.
result OOD-trained Bayesian neural networks are competitive with frequentist baselines.
Comment on recent paper suggesting adversarially trained BNNs are more robust, finds no strong evidence.
problem Evaluating the robustness of adversarially trained Bayesian Neural Networks (BNNs) against adversarial attacks.
method Developed a new type of adversarial attack to incorporate the stochastic nature of BNNs, used to evaluate robustness.
result No strong evidence of higher robustness of adversarially trained BNNs.
BayesFlow trains neural networks for fast Bayesian inference.
problem Fast Bayesian inference for complex models.
method Amortized neural networks for intractable posterior distributions.
result Fast inference through pre-trained neural networks.
Trained neural networks perform Bayesian reasoning for tasks beyond their initial scope.
problem Performing Bayesian reasoning for tasks outside the trained neural networks' initial scope.
method Used deep generative models as priors and classification/regression networks as constraints. Approximated Bayesian inference through variational or sampling techniques.
result The approach built on top of already trained networks, expanding the addressable questions.
Bayesian Neural Networks improve OOD detection with limited data.
problem Limited training data hinders reliable OOD detection.
method Bayesian Neural Networks with expected logit vectors.
result Bayesian methods outperform deterministic methods in small data settings.
Bayesian framework improves deep classifier reliability.
problem Overconfident models under dataset shift.
method Bayesian inference with out-of-distribution data augmentation.
result Reliable uncertainty estimates for deep classifiers.
BLoB fine-tunes LLMs with Bayesian methods to improve uncertainty estimation.
problem Overconfidence in LLMs during inference for domain-specific tasks.
method Continuous Bayesian low-rank adaptation during fine-tuning.
result BLoB improves generalization and uncertainty estimation in LLMs.
This paper connects masked pre-training to Bayesian model selection.
problem Understanding the success of masked pre-training and its generalization.
method The paper shows masked pre-training corresponds to maximizing the marginal likelihood.
result Masked pre-training with a suitable scoring function maximizes the marginal likelihood.
Bayesian method improves SOM training for dynamic data.
problem Training Self Organizing Maps (SOM) for non-stationary data.
method Variational Bayesian approach with adaptive neighborhood radius.
result Method outperforms other adaptive methods in high dimensions.
New method trains partial Bayesian neural networks efficiently.
problem Challenges in approximating multi-modal latent variable distributions in pBNNs.
method Formulates pBNN training as a Feynman--Kac model and uses sequential Monte Carlo samplers.
result Proposed training scheme outperforms state of the art in predictive performance.
Bayesian neural networks learn weights with closed-form updates.
problem Efficiently learning Bayesian neural networks with closed-form updates.
method Closed-form Bayesian inference for online learning of Gaussian-weighted BNNs.
result Closed-form expressions for sequential/online training of BNNs.
Bayesian HMM for protein alignment state estimation.
problem Estimating the MAP state sequence for a Bayesian HMM.
method Training data with Dirichlet priors, iterative algorithms for MAP path estimation.
result Bayesian setup outperforms frequentist estimation in protein alignment.
UA-SABI uses surrogates to speed up Bayesian inference for expensive models.
problem Inference for computationally expensive models is slow and uncertain.
method Combines surrogate modeling with Amortized Bayesian Inference (ABI) to propagate uncertainties.
result Reliable, fast, and repeated Bayesian inference for expensive models is achieved.
Bayesian Neural Networks help quantify uncertainty in deep learning predictions.
problem Uncertainty quantification in deep learning predictions.
method Bayesian statistics applied to neural networks.
result Design, implementation, training, and evaluation of Bayesian Neural Networks.
Bayesian optimization improved with variational last layer training.
problem Bayesian optimization performance on complex input correlations.
method Connecting variational Bayesian last layer training to exact conditioning in Gaussian processes, developing an efficient online training algorithm.
result VBLL networks significantly outperform GPs and match well-tuned GPs on benchmark tasks.
We correct for sampling bias in training models to improve real-world performance.
problem Sampling bias causes discrepancies between lab and real-world model performance.
method Bayesian risk minimization and derived bias-corrected loss functions.
result Our approach integrates seamlessly into current learning paradigms and improves model performance.
SSVI efficiently trains sparse Bayesian neural networks with minimal compression and performance loss.
problem Efficiently training Bayesian neural networks with uncertainty quantification.
method SSVI optimizes a sparse subspace basis selection and its parameters alternately, guided by weight distribution statistics.
result SSVI achieves significant compression (10-20x model size reduction) with minimal performance drop (under 3%) and FLOPs reduction (up to 20x) compared to dense Variational Inference.
Improved uncertainty estimation in neural networks with VBLL.
problem Improving uncertainty estimation in neural networks.
method Deterministic variational formulation for training Bayesian last layer neural networks.
result Improves predictive accuracy, calibration, and out-of-distribution detection.
Bayesian tensor train kernel machine uses Laplace approximation for scalable GP regression.
problem Scalability limitations of Gaussian process regression.
method Bayesian tensor train kernel machine with Laplace approximation and variational inference.
result VI replaces cross-validation and offers up to 65x faster training.
This paper develops efficient federated learning and unlearning methods in Bayesian models.
problem Managing epistemic uncertainty and legal right to be forgotten in decentralized networks.
method Develops federated variational inference solutions based on decentralized local free energy minimization.
result Demonstrates efficient unlearning mechanisms in federated learning and unlearning.
BayesAdapter turns pre-trained NNs into reliable BNNs with minimal overhead.
problem Scalability, accessibility, and reliability of Bayesian neural networks.
method Bayesian fine-tuning of pre-trained deterministic NNs to variational BNNs.
result BayesAdapter produces more reliable posteriors with less training overhead.
New method improves uncertainty estimation in Bayesian deep learning models.
problem Underestimation of predictive uncertainty in Neural Linear Models (NLMs).
method Proposes a novel training method to capture useful predictive uncertainties and incorporate domain knowledge.
result Traditional training procedures for NLMs can drastically underestimate uncertainty in data-scarce regions.
Decentralized learning over social networks using Bayesian methods.
problem Training models on distributed data in a decentralized manner.
method Decentralized Bayesian learning algorithm over a graph, using peer-to-peer model aggregation.
result Systematic treatment of model aggregation over arbitrary connected graphs with strong analytic guarantees.
We present a new method to approximate posterior probabilities of Bayesian Network using Deep Neural Network. Experiment results on several public Bayesian Network datasets shows that Deep Neural Network is capable of learning joint probability distri- bution of Bayesian Network by learning from a few observation and p…
Adaptive Gaussian process models for efficient Bayesian inference.
problem Expensive forward models in Bayesian inference.
method Fully Bayesian approach with adaptive training designs maximizing expected improvement.
result Adaptive designs lead to more accurate posterior estimation at lower cost.
Bayesian Attention Networks compress data by focusing on key training samples.
problem Lossless data compression for efficiency.
method Bayesian Attention Networks with attention factors and latent space.
result Efficient prediction using a few correlated training samples.
We consider the problem of learning Bayesian network classifiers that maximize the marginover a set of classification variables. We find that this problem is harder for Bayesian networks than for undirected graphical models like maximum margin Markov networks. The main difficulty is that the parameters in a Bayesian ne…
Proposes a method for training Bayesian neural networks using synthetic data from Raman and CARS spectra.
problem Limited real observations in Raman and CARS spectroscopy.
method Log-Gaussian Gamma Processes and Bayesian Neural Networks.
result Trained Bayesian neural networks provide accurate estimates of Raman and CARS spectra with uncertainty quantification.
Paper uses RL and diffusion models to solve Bayesian inverse problems.
problem Bayesian inverse problems with latent biases.
method Relative Trajectory Balance (RTB) for RL, conditional diffusion models, off-policy backtracking exploration.
result RTB improves diffusion model posteriors for inverse problems.
Bayesian approach to selecting data for machine learning.
problem Iterative data selection in machine learning algorithms.
method Embedding data selection into decision theory and deriving Bayes-optimal criteria.
result Mitigates confirmation bias in data selection.
Bayesian deep learning reduces overfitting and uncertainty measurement.
problem Overfitting and lack of model uncertainty in deep neural networks.
method Variational inference to approximate posterior distribution with normal prior.
result Test error reduced by 15% compared to classical deep learning.
Bayesian geoacoustic inversion improved using MDN.
problem Efficiently solving Bayesian geoacoustic inversion problems.
method Deriving geoacoustic statistics from multidimensional posterior density using MDN, training the network on the whole parameter space.
result The network provides reliable predictions and good generalization performance, solving problems in seconds.
Bayesian theory explains abrupt emergence of copy subcircuit in attention.
problem Understanding the abrupt emergence of the copy subcircuit in attention during training.
method Deriving a closed-form posterior over the attention matrix and reducing it to a low-dimensional order parameter space.
result Derive a phase transition in the amount of training data.
Automatic Chemical Design is a framework for generating novel molecules with optimized properties. The original scheme, featuring Bayesian optimization over the latent space of a variational autoencoder, suffers from the pathology that it tends to produce invalid molecular structures. First, we demonstrate empirically …
Bayesian approach improves sparse PCE for high-dimensional problems.
problem Sparse PCE struggles with high-dimensional uncertainty and underdetermined situations.
method Joint shrinkage priors and MCMC for sparse PCE with uncertainty estimation.
result Bayesian PCE achieves sparse representations with higher polynomial degrees.
Bayesian method improves forecasting of nonseparable Hamiltonian systems with noise.
problem Forecasting nonseparable Hamiltonian systems with multiplicative noise.
method Bayesian approach using deep learning and reduced-order modeling.
result Bayesian method yields up to 724 times improvement in forecasting accuracy.
Bayesian nonparametric models improve OOD detection, especially with complex covariance structures.
problem Improving out-of-distribution detection methods, especially in complex scenarios.
method Proposes Bayesian nonparametric mixture models with hierarchical priors that generalize the Mahalanobis distance score.
result Bayesian nonparametric methods outperform existing OOD methods, especially in complex scenarios.
Bayesian optimization improves hyperparameter tuning for deep learning.
problem Efficient tuning of hyperparameters in deep learning is expensive and time-consuming.
method Bayesian optimization exploiting iterative learning progress.
result Our algorithm identifies optimal hyperparameters faster than existing methods.
Bayesian REX learns Atari games from demonstrations efficiently.
problem Bayesian reward learning for complex control problems is computationally intractable.
method Bayesian Reward Extrapolation (Bayesian REX) pre-trains a low-dimensional feature encoding and uses preferences to perform fast Bayesian inference.
result Bayesian REX learns Atari games from demonstrations in 5 minutes, competitive with state-of-the-art methods.
A new metric estimates classifier accuracy using only training data.
problem Assessing classifier accuracy without cross-validation.
method Bayesian Area Under the ROC Curve (CBAUC) metric for linear classifiers.
result The CBAUC is faster and more accurate than conventional AUC estimators.
MPHD transfers knowledge across different domains for Bayesian optimization.
problem Optimizing functions with unknown or diverse domains.
method MPHD uses neural nets to map domain-specific contexts to GP specifications, enabling transfer learning across heterogeneous search spaces.
result MPHD improves black-box function optimization performance on diverse domains.
Proposes efficient multi-fidelity Bayesian optimization for deep neural network hyperparameter tuning.
problem Time-consuming validation error evaluation for hyperparameter tuning in deep neural networks.
method Introduces trace-aware knowledge-gradient acquisition function and a provably convergent optimization method.
result Outperforms state-of-the-art alternatives for hyperparameter tuning of deep neural networks.
This paper proposes a method to automatically compress neural networks using Bayesian tensor decomposition.
problem Challenges in directly applying tensor compression in neural network training.
method Bayesian tensorized neural network with automatic rank selection.
result Produces significantly more compact neural networks (7.4x to 137x) directly from training.