This paper improves deep learning by integrating Bayesian inference into network structure learning.
problem Bayesian inference in high-dimensional, over-parameterized neural networks.
method Developed an efficient stochastic variational inference approach to learn both network structure and weights.
result Empirically, the method exhibits competitive predictive performance and preserves Bayesian benefits.
Bayesian Neural Networks help quantify uncertainty in deep learning predictions.
problem Uncertainty quantification in deep learning predictions.
method Bayesian statistics applied to neural networks.
result Design, implementation, training, and evaluation of Bayesian Neural Networks.
Bayesian methods enhance deep learning models by improving reliability and uncertainty.
problem Improving reliability and uncertainty awareness in deep learning models.
method Approximate Bayesian inference techniques, including SG-MCMC and VI, applied to deep learning models.
result Enhanced posterior inference for deep learning models, particularly in neural networks and generative models.
Few Bayesian layers near output capture model uncertainty in deep learning.
problem Capturing model uncertainty in deep learning models.
method Varying the number and position of Bayesian layers in a network, comparing performance on active learning with MNIST dataset.
result Few Bayesian layers near the output can capture model uncertainty efficiently.
In this paper we introduce ZhuSuan, a python probabilistic programming library for Bayesian deep learning, which conjoins the complimentary advantages of Bayesian methods and deep learning. ZhuSuan is built upon Tensorflow. Unlike existing deep learning libraries, which are mainly designed for deterministic neural netw…
Bayesian Neural Networks improve uncertainty estimation in deep learning.
problem Lack of robustness and sensitivity to out-of-distribution samples in DNNs.
method Empirical evaluation of Bayesian Neural Networks against point estimate DNNs.
result Bayesian Neural Networks provide better uncertainty quantification and performance.
Study binary activated deep neural networks using PAC-Bayesian theory.
problem Generalization bounds for binary activated deep neural networks.
method Developed an end-to-end framework and provided PAC-Bayesian generalization bounds.
result Nonvacuous PAC-Bayesian generalization bounds for binary activated deep neural networks.
Bayesian state estimation improves accuracy for unobservable power distribution systems.
problem State estimation for unobservable distribution systems.
method Deep learning approach with distribution learning, Monte Carlo training, and Bayesian bad-data detection.
result Deep learning outperforms existing benchmarks in state estimation accuracy.
Method selects the best deep learner for time-series prediction using Bayesian networks.
problem Selecting the most effective deep learning model for time-series prediction.
method Bayesian network selects deep learners based on input variables and cluster training data.
result Threshold value determines which deep learners predict time-series data robustly.
Review of priors in Bayesian deep learning models.
problem The importance of prior choices in Bayesian deep learning models.
method Overview of different priors and methods of learning priors from data.
result Motivate practitioners to think carefully about prior specification.
Bayesian regularizations are explicitly implemented in CNNs, improving deep learning generalization.
problem Improving generalization in deep learning models.
method Introduced a novel probabilistic representation for CNN hidden layers and demonstrated their Bayesian nature.
result CNNs have explicitly Bayesian regularizations based on Bayesian regularization theory.
Bayesian deep learning improves accuracy and calibration without sacrificing scalability.
problem Bayesian inference's potential for deep neural networks.
method Marginalization over optimization, using neural networks' inherent structure and inductive biases.
result Improvements in accuracy and calibration compared to standard training methods.
Bayesian deep learning reduces overfitting and uncertainty measurement.
problem Overfitting and lack of model uncertainty in deep neural networks.
method Variational inference to approximate posterior distribution with normal prior.
result Test error reduced by 15% compared to classical deep learning.
BayesDLL offers a PyTorch library for Bayesian deep learning with large models.
problem Bayesian inference for large-scale deep networks.
method Variational inference, MC-dropout, stochastic-gradient MCMC, Laplace approximation.
result BayesDLL can handle Vision Transformers and pre-trained model weights as priors.
We present a new method to approximate posterior probabilities of Bayesian Network using Deep Neural Network. Experiment results on several public Bayesian Network datasets shows that Deep Neural Network is capable of learning joint probability distri- bution of Bayesian Network by learning from a few observation and p…
Study how depth affects inference in deep Bayesian neural networks.
problem Understanding how depth impacts inference in overparameterized linear Bayesian neural networks.
method Interpreting finite deep linear Bayesian neural networks as scale mixtures of Gaussian process predictors.
result Advances analytical understanding of how depth affects inference in a simple class of Bayesian neural networks.
Bayesian neural networks simplified with input augmentation.
problem Uncertainty in deep learning models.
method Layer-wise input augmentation to induce uncertainty distributions.
result State-of-the-art performance in uncertainty representation.
Bayesian network learns data invariances without augmentation.
problem Learning invariances in neural networks without manual design.
method Bayesian approach infers weight-sharing schemes from data.
result Model outperforms non-invariant networks on specific tasks.
Introduces new metrics to measure performance of deep Bayesian neural networks.
problem Lack of specific criteria to measure performance of deep Bayesian neural networks.
method Proposes several metrics including model calibration, data rejection ability, and uncertainty divergence.
result Introduces more specific criteria for measuring deep Bayesian neural network performance.
Bayesian free energy remains bounded for deep ReLU networks in overparametrized cases.
problem Understanding the generalization performance of deep ReLU neural networks.
method Analyzes Bayesian free energy in overparametrized deep ReLU neural networks.
result Bayesian free energy is bounded even in overparametrized deep ReLU networks.
Bayes posterior yields worse predictions than simpler methods in deep neural networks.
problem Understanding and improving predictive performance in Bayesian deep learning.
method Careful MCMC sampling and evaluation of cold posteriors.
result Cold posteriors yield significantly better predictive performance than the true Bayes posterior.
Bayesian sparsification reduces deep neural network complexity.
problem Complexity of deep neural networks limits their performance.
method Combines Bayesian shrinkage priors with stochastic variational inference.
result Bayesian model reduction (BMR) is a more efficient alternative for pruning model weights.
Bayesian deep learning improves neural network accuracy and generalization.
problem Improving accuracy and calibration of deep neural networks.
method Bayesian marginalization and deep ensembles to approximate marginalization, and tempering for calibrating predictive distributions.
result Bayesian approaches improve deep neural networks' accuracy and generalization.
Bayesian Deep Learning tackles inverse problems with neural networks and approximate computations.
problem Solving inverse problems with indirect measurements and uncertainties.
method Bayesian Deep Learning, using neural networks and approximate computations.
result Effective solutions for inverse problems using Bayesian Deep Learning.
Proposes MOPED method for choosing priors in Bayesian DNNs.
problem Challenges in specifying meaningful priors for deep neural networks.
method Two-stage hierarchical modeling with empirical Bayes.
result MOPED enables scalable variational inference and reliable uncertainty quantification.
This paper connects RND, deep ensembles, and Bayesian inference, providing a unified theoretical perspective.
problem Uncertainty quantification in deep learning models.
method Analysis of Random Network Distillation (RND) within the neural tangent kernel framework.
result The uncertainty signal from RND is equivalent to the predictive variance of a deep ensemble and can be made to mirror the centered posterior predictive distribution of Bayesian inference.
This work simplifies Bayesian inference for neural networks by identifying influential parameter directions.
problem High computational complexity in Bayesian inference for neural networks due to high-dimensional parameter space.
method Constructing an active subspace of influential parameter directions to reduce dimensionality.
result Effective and scalable Bayesian inference achieved via reduced active subspace.
DEEP-BO optimizes hyperparameters of deep networks, outperforming existing methods.
problem Hyperparameter optimization of deep networks is challenging due to the complexity and sensitivity of DNN performance.
method Enhanced Bayesian Optimization (DEEP-BO) specifically designed for deep networks, incorporating diversification, early termination, and parallelism.
result DEEP-BO outperforms or matches other state-of-the-art methods on six DNN benchmarks.
Bayesian Neural Network improves calibration of deep probabilistic models.
problem Uncalibrated probabilities from deep neural networks limit their use in critical scenarios.
method Decoupled Bayesian Neural Network to map uncalibrated probabilities to calibrated ones.
result Our approach consistently improves calibration and provides more reliable probabilistic models.
DropConnect improves uncertainty estimation in Bayesian deep networks.
problem Modeling uncertainty in Bayesian deep networks for safety-critical applications.
method Developed a theoretical framework to approximate Bayesian inference for DNNs using MC-DropConnect.
result Significant improvement in both prediction accuracy and uncertainty estimation quality.
Enhances construction input modeling with Bayesian deep neural networks.
problem Deriving reliable simulation input models from construction data.
method Bayesian deep neural networks integrated with multi-source construction data.
result Derives detailed input models for construction operations.
This paper distills Bayesian posterior expectations for deep neural networks.
problem Improving deep neural network performance and uncertainty quantification.
method Develops a framework for distilling expectations from Bayesian posterior distributions using Monte Carlo samples.
result The framework successfully distills posterior predictive distribution and expected entropy.
Bayesian neural networks tutorial via MCMC in Python.
problem Bayesian inference for parameter estimation and uncertainty quantification in deep learning models.
method MCMC sampling methods to implement Bayesian inference, including advanced proposal distributions.
result Challenges in sampling multi-modal posterior distributions for Bayesian neural networks.
Bayesian deep learning tackles uncertainty in high-dimensional systems.
problem Uncertainty quantification in high-dimensional stochastic partial differential equations.
method Bayesian neural network (BNN) and Hamiltonian Monte Carlo (HMC) for efficient sampling of posterior distributions.
result The method efficiently handles high-dimensional problems with almost independent computational cost.
Bayesian deep learning method improves model uncertainty and robustness.
problem Overfitting and model uncertainty in deep neural networks.
method Variational inference with multivariate normal distributions and correlated parameters.
result The method successfully reduces overfitting and improves model uncertainty on MNIST and CIFAR-10 datasets.
New method uses graph neural networks for neural architecture search.
problem Finding optimal neural architectures efficiently.
method Bayesian graph neural network for feature extraction and graph Bayesian optimization.
result Significantly outperforms existing methods in benchmark tasks.
Bayesian approach adapts deep network structure for continual learning.
problem Training neural networks with sequential or streaming tasks.
method Bayesian approach to learn deep network structure for each task.
result Model performs comparably or better than recent advances in continual learning.
Bayesian approach helps detect adversarial examples in neural networks.
problem Detecting adversarial examples in deep neural networks.
method Adversarial spheres and approximate Bayesian inference for a linear model.
result Bayesian methods provide better predictions of adversarial points.
Improves deep networks' performance by applying Bayesian learning to batch normalization.
problem Improving generalization performance of deep networks.
method Bayesian learning applied to deterministic normalization techniques.
result Significant improvement in generalization performance compared to traditional methods.
Study on Bayesian deep linear networks with multiple outputs and convolutional layers.
problem Characterize feature learning in finite-width Bayesian deep linear networks.
method Exact and analytical formulas for joint and posterior distributions, using large deviation theory.
result Quantitative description of feature learning in infinite-width regime.
Global inducing points improve Bayesian neural network performance.
problem Improving Bayesian neural network performance.
method Adapting correlated approximate posterior to all layers in a Bayesian neural network and deep Gaussian processes using learned global inducing points.
result State-of-the-art performance on CIFAR-10 (86.7%) without data augmentation or tempering.
Bayesian sparsification improves complex-valued neural networks by 50-100x with minimal performance loss.
problem Efficiently compressing complex-valued neural networks for embedded systems.
method Extending Sparse Variational Dropout to complex-valued networks and conducting a numerical study.
result Achieved state-of-the-art performance on MusicNet with 50-100x compression.
Bayesian inference with deep, weakly nonlinear networks is solved rigorously.
problem Bayesian inference with neural networks of specific structure.
method Perturbative analysis of fully connected neural networks with a shaped nonlinearity.
result Neural network Bayesian inference can be equivalent to kernel methods under certain conditions.
Bayesian neural networks predict AD severity from EEG data.
problem Developing low-cost, non-invasive biomarkers for AD diagnosis and progression.
method Bayesian deep neural networks using QEEG markers.
result Bayesian approach provides uncertainty bounds for AD severity prediction.
Bayesian deep learning predicts price movements from LOBs, improving trading profits.
problem Predicting price movements from limit order books for better trading decisions.
method Applies dropout variational inference to deep neural networks, using uncertainty information for position sizing.
result Bayesian techniques improve predictive performance and deliver useful uncertainty information for trading.
PAC-Bayesian bounds show fully connected DNNs with Gaussian priors match minimax rates.
problem Theoretical limits of fully connected deep neural networks with Gaussian priors.
method PAC-Bayesian bounds for fully connected Bayesian DNNs with Gaussian priors.
result PAC-Bayesian bounds match minimax-optimal rates in Besov space for nonparametric regression and binary classification.
Bayesian inference for wide neural networks using Edgeworth expansion.
problem Analyzing the non-Gaussian behavior of wide neural networks in Bayesian inference.
method Proposed a non-Gaussian distribution using multivariate Edgeworth expansion for finite-width neural networks.
result Derived non-Gaussian posterior distribution in Bayesian regression tasks.
Stabilizes deep Bayesian neural networks with self-stabilizing priors.
problem Brittleness and difficulty in training deep Bayesian neural networks.
method Signal propagation theory, reformulated ELBO, self-stabilizing priors.
result Improved convergence and robustness in training deeper networks and noisier settings.