Quantum neural tangent kernels help understand variational quantum circuits in machine learning.
problem Designing and predicting performance of variational quantum circuits.
method Using quantum neural tangent kernels and dynamical equations for loss functions.
result Analytical solutions for training dynamics in variational quantum circuits.
Variational Bayes simplifies Bayesian neural networks for uncertainty quantification.
problem Quantifying uncertainty in neural networks' outputs.
method Approximates intractable Bayesian integrals using variational methods.
result Comparison of various approximation methods in literature.
Proposes a new method for continual learning in neural networks.
problem Challenges in applying sequential Bayesian inference to neural networks.
method Sequential function-space variational inference.
result Neural networks trained with the proposed method achieve better predictive accuracy.
SFSVI uses Gaussian mixtures to approximate neural network outputs for continual learning.
problem Learning new tasks without forgetting old ones in neural networks.
method Sequential function-space variational inference with Gaussian mixture approximation.
result Gaussian mixture SFSVI outperforms other methods in continual learning.
NVGD uses neural networks to infer distributions without kernel choices.
problem Challenges in choosing kernel functions for SVGD.
method NVGD parameterizes the witness function of the Stein discrepancy with a neural network.
result NVGD achieves good performance on various inference problems.
Advances variational Bayesian neural networks using singular learning theory.
problem Discrepancies between predictive performance and variational objective in BNNs.
method Corrected asymptotic form of singular posterior distributions to inform variational family design.
result Improvements in variational free energy and generalization error with proposed normalizing flow.
New Bayesian neural network model improves performance and compresses networks.
problem Improving predictive performance and compressing neural networks.
method Proposes a new variational family with radial and directional components.
result Improves predictive performance and yields compressed networks.
New framework explains deep neural networks using variational spline theory.
problem Understanding functions learned by deep neural networks.
method Developed a variational framework and function space.
result Deep ReLU networks are solutions to regularized data fitting problems over the proposed function space.
Elvet solves differential equations and variational problems with neural networks.
problem Solving complex differential and variational equations with arbitrary conditions.
method Machine learning, specifically neural networks, to represent and solve equations.
result Elvet can solve a wide range of differential and variational problems.
IVON optimizes large neural networks, matching or outperforming Adam.
problem The inefficacy of variational learning in large neural networks.
method Improved Variational Online Newton (IVON) optimizer.
result IVON consistently matches or outperforms Adam for large networks.
Improved uncertainty estimation in neural networks with VBLL.
problem Improving uncertainty estimation in neural networks.
method Deterministic variational formulation for training Bayesian last layer neural networks.
result Improves predictive accuracy, calibration, and out-of-distribution detection.
Variational Laplace improves Bayesian neural networks performance.
problem Improving Bayesian neural networks performance.
method Develops variational Laplace for BNNs, exploiting curvature of likelihood.
result Variational Laplace outperforms other inference methods.
We develop a method to learn neural network activations with controlled Lipschitz constant.
problem Increase neural network capacity while controlling Lipschitz constant.
method Variational framework to learn activation functions with piecewise-linear constraints.
result Proves existence of solutions with continuous and piecewise-linear activations.
Variational Laplace improves Bayesian neural network performance without sampling.
problem Improving Bayesian neural network performance and calibration.
method Develops a new variational Laplace method for BNNs, exploiting curvature of likelihood.
result Variational Laplace outperforms standard VI methods in test performance and calibration.
Study efficient neural operator learning using variation spaces.
problem Operator learning using encoder-decoder neural networks.
method Introduce variation space for nonlinear operators, establish approximation bounds.
result Algebraic approximation and learning rates for polynomially decaying input and output encoding errors.
BBNN improves neural network accuracy and uncertainty quantification.
problem Overfitting and lack of interpretability in probabilistic neural networks.
method Boosted Bayesian Neural Networks (BBNN) using Boosting Variational Inference (BVI).
result BBNN achieves ~5% higher accuracy and superior uncertainty quantification.
Paper develops efficient variational inference for sparse deep learning with theoretical guarantees.
problem Sparse deep learning's challenge of huge storage consumption and sparse structure recovery.
method Bayesian treatment with spike-and-slab priors and continuous relaxation of Bernoulli distribution for computationally efficient variational inferences.
result Provides variational posterior contraction rate, justifying consistency of the proposed method.
Unified theory for training neural networks with binary synapses.
problem Discrete nature of synapses and complex interactions in neural networks.
method Variational mean-field theory decomposing learning into maximization and expectation steps.
result Unified framework for unsupervised learning in neural networks.
VANO uses neural operators for unsupervised learning of functional data.
problem Learning operators between infinite dimensional spaces for functional data.
method Variational Autoencoding Neural Operators (VANO) approach.
result VANO can learn and reconstruct functional data without supervision.
Bayesian neural networks ignore data in infinite units limit.
problem Pathological behavior of posterior in over-parameterized networks.
method Mean-field variational inference in infinite hidden units limit.
result Posterior mean converges to zero, ignoring data.
NCV uses neural networks to improve Monte Carlo integration.
problem Improving variance reduction in parametric Monte Carlo integration.
method NCV combines a normalizing flow and a neural network to approximate the integrand and solve the integral equation, with a neural importance sampler to estimate the difference.
result NCV achieves state-of-the-art performance in light transport simulation with reduced noise and negligible bias.
VASE uses Bayesian neural networks to improve exploration in sparse reward environments.
problem Exploration in environments with continuous control and sparse rewards.
method VASE uses a Bayesian neural network model of the environment dynamics and variational inference to alternately update the model's accuracy and policy.
result VASE outperforms other surprise-based exploration techniques in continuous control sparse reward environments.
Bayesian method learns neural network architecture parameters.
problem Estimating optimal neural network architecture parameters.
method Bayesian learning of concrete distributions over layer size and network depth.
result Regular networks with learnt structure generalize better on small datasets, while stochastic networks are more robust to initialisation.
In statistics and machine learning, approximation of an intractable integration is often achieved by using the unbiased Monte Carlo estimator, but the variances of the estimation are generally high in many applications. Control variates approaches are well-known to reduce the variance of the estimation. These control v…
A new method infers neural trajectories in real-time, improving experimental design.
problem Real-time inference of neural trajectories for immediate feedback.
method Exponential family variational Kalman filter (eVKF) for online learning.
result eVKF achieves competitive performance on synthetic and real-world data.
GWI combines deep neural networks with Gaussian processes for better predictive performance and uncertainty quantification.
problem Combining deep learning with Gaussian process uncertainty quantification.
method Gaussian Wasserstein inference (GWI) using Wasserstein distance between Gaussian measures.
result GWI achieves state-of-the-art performance on benchmark datasets.
Proposes MOPED method for choosing priors in Bayesian DNNs.
problem Challenges in specifying meaningful priors for deep neural networks.
method Two-stage hierarchical modeling with empirical Bayes.
result MOPED enables scalable variational inference and reliable uncertainty quantification.
Paper introduces vector-valued variation spaces for multi-output neural networks.
problem Understanding and optimizing multi-output neural networks.
method Development of vector-valued variation spaces and representer theorem.
result Novel bounds for layer widths in deep networks and a convex optimization method for compression.
Hidden Markov Neural Networks balance adaptation and forgetting in time-series data.
problem Balancing adaptation to new data and forgetting outdated information in time-series forecasting.
method Modeling weights as hidden states of a Hidden Markov model, using a filtering algorithm for learning a variational approximation of the posterior distribution over weights, and employing sequential Bayes by Backprop with variational DropConnect for regularization.
result Achieves strong predictive performance and effective uncertainty quantification on various tasks.
We develop a variational framework for SDEs driven by fractional noise.
problem Capturing long-term dependencies in SDEs driven by fractional noise.
method Markov approximation of fractional Brownian motion, variational inference, neural networks.
result Efficient variational inference of posterior path measures for neural-SDEs.
VHVM models financial time series with varying volatility.
problem Modeling heteroscedastic behavior in multivariate financial time series.
method Variational autoencoder and recurrent neural network for capturing relationships and temporal dynamics.
result VHVM outperforms GARCH and SV models on FX datasets.
Combines neural networks with variational inference for better uncertainty quantification.
problem Overconfident predictions from traditional neural networks and time-consuming Bayesian optimization.
method VIFO (Variational Inference on the Final-Layer Output) using neural networks to learn mean and variance.
result VIFO provides a good tradeoff in run time and uncertainty quantification, especially for out of distribution data.
Max-Entropy approach improves variational inference for complex posterior distributions.
problem Efficient inference with simple families vs. accuracy in variational inference.
method Greedy approximation of the posterior distribution with Max-Entropy approach.
result Demonstrated ability to capture complex multimodal posterior distributions.
New method tightens variational representations of divergences for faster learning.
problem Improving tightness of variational representations of divergences for faster statistical estimation.
method Improved objective functionals constructed via an auxiliary optimization problem, leveraging neural network approximation.
result Tighter variational representations can result in significantly faster learning and more accurate estimation of divergences.
New algorithms accelerate SVGD convergence using deep unfolding.
problem Improving the speed of SVGD convergence.
method Integrating deep unfolding into SVGD for parameter learning.
result Proposed algorithms achieve faster convergence in various tasks.
Neural SDEs reduce variance in stochastic simulations.
problem Efficiency of Monte Carlo simulations in finance.
method Use neural SDEs with control variates parameterized by neural networks.
result Prove optimality conditions for variance reduction in SDEs with infinite activity.
Improves continual learning with Variational Continual Learning framework.
problem Avoiding catastrophic forgetting and efficient model capacity use in sequential tasks.
method Mean-field variational Bayesian neural networks approach.
result Significantly improved results on continual learning benchmarks.
D-VAE generates valid DAGs for neural architecture search and Bayesian network learning.
problem Generating valid DAGs for machine learning models.
method Proposes a novel DAG variational autoencoder (D-VAE) using graph neural networks and asynchronous message passing.
result Demonstrates the effectiveness of D-VAE through neural architecture search and Bayesian network structure learning.
Bayesian neural networks improve deep learning's accuracy and uncertainty estimation.
problem Overconfident predictions, adversarial attacks, and variability underestimation in deep models.
method Stochastic relaxation of feed-forward rectified neural networks with sparsity-promoting priors and Polya-Gamma data augmentation.
result Improved scalability and robustness to architectural design through approximate variational inference.
Alternative neural network training using monotone variational inequality.
problem Training neural networks efficiently and with guarantees.
method Using monotone variational inequality to solve non-convex problems efficiently.
result Our approach leads to fast convergence and competitive performance compared to traditional methods.
New Variational InfoMax objective improves neural network performance.
problem Optimizing neural networks using Bayesian Inference and Information Bottleneck.
method Derive Variational InfoMax (VIM) objective that maximizes InfoMax directly.
result VIM improves model performance in accuracy, robustness, and representation quality.
Paper shows variational inference works well for sparse deep learning models.
problem Generalization of sparse deep learning models using variational inference.
method Theoretical analysis linking variational inference to Bayesian and nonparametric regression.
result Near-minimax rates of convergence for Hölder smooth functions in sparse deep learning.
This thesis disentangles Gauss-Newton and variational approximations in Bayesian deep learning.
problem Understanding the interplay between the Gauss-Newton method and variational approximations in Bayesian deep learning.
method Analysis of the Gauss-Newton method and Laplace/Gaussian variational approximations for neural networks.
result The combination of the Gauss-Newton method with approximate inference can be cast as inference in a linear or Gaussian process model.
A new framework for lightweight BNNs learns heteroscedastic uncertainties efficiently.
problem Learning heteroscedastic uncertainties from BNNs for lightweight networks.
method Embedding heteroscedastic variances into BNN parameters and using moment propagation for inference.
result Improves predictive performance for lightweight BNNs without increasing parameter count.
Derives formulae for general permutation equivariant layers and presents a second order graph variational encoder.
problem Tackles the limitation of previous equivariant neural networks by considering permutations of matrices.
method Derives formulae for general permutation equivariant layers, including matrix permutations. Presents a second order graph variational encoder.
result Latent distribution of equivariant generative models must be exchangeable.
Probabilistic deep learning uses neural networks and models to handle uncertainty.
problem Handling uncertainty in deep learning models.
method Two approaches: probabilistic neural networks and deep probabilistic models.
result TensorFlow Probability library supports both approaches.
A new framework for neural network classification using vector quantization.
problem Learning a neural network classifier under the IB principle.
method Aggregated Learning framework, combining vector quantization and variational techniques.
result The effectiveness of Aggregated Learning verified through experiments.
GraphVRNN generates graphs with latent variables and node attributes.
problem Generating diverse and complex graph structures.
method Probabilistic autoregressive model for graph generation.
result GraphVRNN can model complicated distributions and generate plausible structures and node attributes.