Bayesian neural networks (BNN) can estimate the uncertainty in predictions, as opposed to non-Bayesian neural networks (NNs). However, BNNs have been far less widely used than non-Bayesian NNs in practice since they need iterative NN executions to predict a result for one data, and it gives rise to prohibitive computat…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “BNN inference”
New BNN model proves optimal posterior concentration and enables practical inference.
problem Improving generalization and uncertainty quantification in deep neural networks.
method Proposes a new node-sparse BNN model with theoretical guarantees and a novel MCMC algorithm for inference.
result Proves near minimax optimal posterior concentration rate and adaptiveness to true model smoothness.
A new framework for lightweight BNNs learns heteroscedastic uncertainties efficiently.
problem Learning heteroscedastic uncertainties from BNNs for lightweight networks.
method Embedding heteroscedastic variances into BNN parameters and using moment propagation for inference.
result Improves predictive performance for lightweight BNNs without increasing parameter count.
New BNN method reduces training time and model size.
problem Overconfident predictions in deep learning models.
method Designing STF-BNN for efficient scaling of BNNs.
result Significantly reduces training time and model size compared to vanilla BNNs.
Bayesian Neural Networks (BNNs) place priors over the parameters in a neural network. Inference in BNNs, however, is difficult; all inference methods for BNNs are approximate. In this work, we empirically compare the quality of predictive uncertainty estimates for 10 common inference methods on both regression and clas…
Bayesian neural networks are vulnerable to adversarial attacks.
problem Adversarial robustness of Bayesian neural networks.
method Examination of adversarial robustness through three tasks: label prediction, adversarial example detection, and semantic shift detection.
result Bayesian neural networks are highly susceptible to adversarial attacks.
Novel framework for Bayesian neural networks incorporating task-specific constraints.
problem Task-specific constraints in supervised model deployment.
method Introduces Output-Constrained BNN (OC-BNN) framework.
result OC-BNNs effectively incorporate prior expert knowledge and desiderata like safety and fairness.
New algorithms improve binary neural network configurations.
problem Training binary neural networks efficiently.
method Stochastic message passing algorithms (BP and SP) for discrete inference.
result Stochastic BP and SP find better BNN configurations.
Paper proposes efficient BNN inference flow to reduce computation and memory costs.
problem High computation complexity in Bayesian Neural Networks (BNNs) limits deployment in power-constrained systems.
method Feature decomposition and memorization strategy to reduce computations and a memory-friendly computing framework to reduce memory overhead.
result Reduces computation by about half and energy consumption by 73% with 14% area overhead.
SSVI efficiently trains sparse Bayesian neural networks with minimal compression and performance loss.
problem Efficiently training Bayesian neural networks with uncertainty quantification.
method SSVI optimizes a sparse subspace basis selection and its parameters alternately, guided by weight distribution statistics.
result SSVI achieves significant compression (10-20x model size reduction) with minimal performance drop (under 3%) and FLOPs reduction (up to 20x) compared to dense Variational Inference.
Meta-learn Bayesian inference for task-specific BNNs using amortised inference.
problem Efficiently learning Bayesian inference for small-scale probabilistic meta-learning.
method Replace global inducing points with actual data to create a set of approximate likelihoods, train a meta-model to learn these parameters across related datasets.
result Meta-learned inference can be applied to task-specific BNNs, improving efficiency and scalability.
Binarized Neural Network (BNN) removes bitwidth redundancy in classical CNN by using a single bit (-1/+1) for network parameters and intermediate representations, which has greatly reduced the off-chip data transfer and storage overhead. However, a large amount of computation redundancy still exists in BNN inference. B…
Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learningstat.ML
We propose Radial Bayesian Neural Networks (BNNs): a variational approximate posterior for BNNs which scales well to large models while maintaining a distribution over weight-space with full support. Other scalable Bayesian deep learning methods, like MC dropout or deep ensembles, have discrete support-they assign zero…
New PDMP samplers improve BNN inference with accelerated computation.
problem Inference on Bayesian Neural Networks violates independence and posterior assumptions.
method Piecewise Deterministic Markov Process (PDMP) with adaptive thinning for inhomogenous Poisson Process (IPPs) sampling.
result PDMP samplers accelerate inference in BNNs, improving accuracy and mixing performance.
Bayesian optimization uses BNNs as efficient surrogate models for expensive function evaluations.
problem Optimizing expensive objective functions using Gaussian process surrogates.
method Study of Bayesian neural networks (BNNs) as alternatives to standard Gaussian process (GP) surrogates for optimization.
result Infinite-width BNNs are particularly promising, especially in high dimensions.
Wide BNNs with odd activations fail to approximate data under mean-field inference.
problem Theoretical limitations of mean-field variational inference in wide, deep Bayesian neural networks.
method Analysis of mean-field variational inference in fully-connected BNNs with odd activation functions and Gaussian likelihood.
result The optimal mean-field variational posterior predictive distribution converges to the prior predictive distribution as network width increases.
Bayesian neural network (BNN) priors are defined in parameter space, making it hard to encode prior knowledge expressed in function space. We formulate a prior that incorporates functional constraints about what the output can or cannot be in regions of the input space. Output-Constrained BNNs (OC-BNN) represent an int…
Bayesian neural networks incorporate domain knowledge through variational inference.
problem Specifying priors for Bayesian neural networks that capture domain knowledge is challenging.
method Proposes a framework for integrating domain knowledge into BNN priors through variational inference.
result BNNs with proposed domain knowledge priors outperform those with standard priors, achieving better predictive performance.
Credit risk modelling is an integral part of the global financial system. While there has been great attention paid to neural network models for credit default prediction, such models often lack the required interpretation mechanisms and measures of the uncertainty around their predictions. This work develops and compa…
Bayesian Neural Networks (BNNs) have been proposed to address the problem of model uncertainty in training and inference. By introducing weights associated with conditioned probability distributions, BNNs are capable of resolving the overfitting issue commonly seen in conventional neural networks and allow for small-da…
An Empirical Analysis of the Advantages of Finite- v.s. Infinite-Width Bayesian Neural Networksstat.ML
Empirical study compares finite- and infinite-width BNNs, revealing performance differences under model mismatch.
problem Comparing BNNs with different widths due to conflicting model properties and inference intractability.
method Empirical comparison of finite- and infinite-width BNNs, analyzing performance under model mismatch.
result Increasing width can hurt BNN performance when the model is mis-specified, and finite-width BNNs generalize better under model mismatch.
We introduce a probabilistic robustness measure for Bayesian Neural Networks (BNNs), defined as the probability that, given a test point, there exists a point within a bounded set such that the BNN prediction differs between the two. Such a measure can be used, for instance, to quantify the probability of the existence…
Paper addresses variational inference issues in Bayesian neural networks.
problem Negative infinite ELBO for function-space priors in BNNs.
method Regularized KL divergence for well-defined function-space variational inference.
result Method provides competitive uncertainty estimates for BNNs.
The paper develops scalable variational inference for Bayesian neural networks under model and parameter uncertainty.
problem Combining structural and parameter uncertainties in scalable Bayesian neural networks.
method Adapted variational inference with reparametrization for model space constraints.
result Comparable accuracy with sparse inference compared to ordinary BNNs.
Bayesian neural networks (BNNs) have recently regained a significant amount of attention in the deep learning community due to the development of scalable approximate Bayesian inference techniques. There are several advantages of using Bayesian approach: Parameter and prediction uncertainty become easily available, fac…
BayesAdapter turns pre-trained NNs into reliable BNNs with minimal overhead.
problem Scalability, accessibility, and reliability of Bayesian neural networks.
method Bayesian fine-tuning of pre-trained deterministic NNs to variational BNNs.
result BayesAdapter produces more reliable posteriors with less training overhead.
Favour speeds up variance estimation in BNNs, making them practical for performance-critical tasks.
problem Slow inference in Bayesian Neural Networks (BNNs) hinders their adoption.
method A fast algorithm for updating a diagonal-plus-low-rank matrix approximation under various operations.
result Favour is as fast as 2-3 inference samples but matches the performance of 10-100 samples.
Analog BNNs perform similarly regardless of noise distribution shape.
problem Difficulty in precisely controlling noise distribution shape in analog devices.
method Used real device noise as the variational distribution in MFVI training.
result Predictive distributions converge to the same distribution regardless of noise shape.
While Bayesian neural networks (BNNs) hold the promise of being flexible, well-calibrated statistical models, inference often requires approximations whose consequences are poorly understood. We study the quality of common variational methods in approximating the Bayesian predictive distribution. For single-hidden laye…
Bayesian neural networks integrate uncertainty into neural networks for improved performance.
problem Overconfidence, lack of interpretability, and adversarial attacks in neural networks.
method Integrates Bayesian inference into neural networks to address limitations.
result BNNs improve model performance and provide uncertainty estimates.
Deep kernel processesstat.ML
Deep kernel processes unify various models using Gram matrices and kernel functions.
problem Unified representation of various deep learning models.
method Defining deep kernel processes with progressively transformed Gram matrices and sampling from inverse Wishart distributions.
result Deep Gaussian processes, BNNs, infinite BNNs, and infinite BNNs with bottlenecks can all be written as deep kernel processes.
PFP-BNNs offer a fast, deterministic approach to Bayesian neural networks.
problem Limited uncertainty handling in traditional neural networks restricts their use in safety-critical settings.
method Probabilistic Forward Pass (PFP) approximates Stochastic Variational Inference (SVI) for efficient BNNs.
result PFP-BNNs achieve up to 4200x speedup over SVI-BNNs while maintaining similar accuracy and uncertainty.
This study explores training Bayesian neural networks at scale using high-performance computing.
problem Challenges in training Bayesian neural networks at scale due to computational overhead.
method High-performance computing with distributed training, network pruning.
result Pruning up to 80% of the network can reduce inference time by 7.0% without significant accuracy loss.
Bayesian neural networks learn weights with closed-form updates.
problem Efficiently learning Bayesian neural networks with closed-form updates.
method Closed-form Bayesian inference for online learning of Gaussian-weighted BNNs.
result Closed-form expressions for sequential/online training of BNNs.
This work connects BNNs to GPs, providing scalable inference and identifying key properties.
problem Scaling and inference challenges in Bayesian neural networks.
method General convergence from BNNs to GPs, new covariance function, and scalable Nyström approximation.
result Established a scalable maximum a posterior (MAP) training and prediction procedure.
This paper improves Bayesian neural nets by using local linearization.
problem Underfitting in Bayesian neural networks.
method Local linearization of Bayesian neural networks to create a generalized linear model (GLM) for predictions.
result The GLM predictive resolves common underfitting problems of the Laplace approximation.
Vulnerability to adversarial attacks is one of the principal hurdles to the adoption of deep learning in safety-critical applications. Despite significant efforts, both practical and theoretical, the problem remains open. In this paper, we analyse the geometry of adversarial attacks in the large-data, overparametrized …
The paper examines VI for overparameterized BNNs, revealing a trade-off between likelihood and KL terms.
problem Critical issue in mean-field VI training for overparameterized BNNs.
method Theoretical and empirical study of overparameterized two-layer BNNs using VI.
result A trade-off between likelihood and KL terms in overparameterized regime, with KL scaling crucial.
This paper introduces GLT for better input data representation in BNN and proposes a compact topology with block pruning.
problem Improving input data representation for Binary Neural Networks (BNN).
method Generic Learned Thermometer (GLT) for encoding, block pruning and Knowledge Distillation for compact topology.
result Significant accuracy gains and lightweight fully-binarized models with limited accuracy degradation.
R2D2-Net improves Bayesian neural networks by preventing over-shrinkage of important weights.
problem Bayesian neural networks struggle with choosing appropriate priors, leading to over-shrinkage or poor predictive performance.
method Proposes R2D2-Net with an R^2-induced Dirichlet Decomposition prior and variational Gibbs inference algorithm.
result R2D2-Net effectively shrinks irrelevant coefficients while preventing key features from over-shrinkage.
A new method improves Bayesian deep learning by balancing scalability and accuracy.
problem Scalability issues in Bayesian neural networks.
method Collapsed inference scheme that performs Bayesian model averaging using collapsed samples.
result Significant improvements over existing methods in predictive performance and uncertainty estimation.
Paper presents a method to summarize HMC samples for neural networks, providing meaningful uncertainty estimates.
problem Lack of interpretable summary statistics for HMC samples in neural networks due to permutation symmetry.
method Introducing a transpositions metric to quantify permutations and using rebasin method to summarize HMC samples.
result Compact representation of HMC samples provides meaningful uncertainty estimates for each weight in a neural network.
Investigating maximum likelihood based training of infinite mixtures for uncertainty quantificationcs.LG
This work investigates training infinite mixtures with maximum likelihood for improved uncertainty quantification.
problem Improving uncertainty quantification in neural networks.
method Investigates training infinite mixtures with maximum likelihood instead of variational inference.
result The proposed method leads to stochastic networks with increased predictive variance, improved robustness, and higher entropy on out-of-distribution data.
We prove, under two sufficient conditions, that idealised models can have no adversarial examples. We discuss which idealised models satisfy our conditions, and show that idealised Bayesian neural networks (BNNs) satisfy these. We continue by studying near-idealised BNNs using HMC inference, demonstrating the theoretic…
Bayesian model averaging fails under covariate shift, affecting neural networks' performance.
problem Bayesian model averaging's failure in neural networks under covariate shift.
method Explained the issue and proposed novel priors to improve robustness.
result Bayesian model averaging is problematic under covariate shift, especially with linear feature dependencies.
Bayesian neural networks (BNNs) have developed into useful tools for probabilistic modelling due to recent advances in variational inference enabling large scale BNNs. However, BNNs remain brittle and hard to train, especially: (1) when using deep architectures consisting of many hidden layers and (2) in situations wit…
Leaky ReLU activations improve the calibration of Bayesian neural networks.
problem Bayesian neural networks struggle with mean-field variational inference for ReLU activations.
method Investigated the effect of activation functions on the calibration of Bayesian neural networks.
result Leaky ReLU activations lead to more Gaussian-like weight posteriors and lower expected calibration error.
A new method for efficient probabilistic meta-learning.
problem High-quality predictions with well-calibrated uncertainty estimates require large amounts of data.
method Amortised Inference in Bayesian Neural Networks (APOVI-BNN)
result The APOVI-BNN produces high-quality predictions with well-calibrated uncertainty estimates using significantly less data.