A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Bayesian method improves neural net convergence for character recognition.
SDE-Net quantifies uncertainty in deep nets using stochastic dynamics.
Neural Ordinary Differential Equations (N-ODEs) are a powerful building block for learning systems, which extend residual networks to a continuous-time dynamical system. We propose a Bayesian version of N-ODEs that enables well-calibrated quantification of prediction uncertainty, while maintaining the expressive power …
Neural surrogate predicts SPN rates from token trajectories.
Bayesian ReLU nets fix asymptotic overconfidence with infinite features.
Deterministic neural nets have been shown to learn effective predictors on a wide range of machine learning problems. However, as the standard approach is to train the network to minimize a prediction loss, the resultant model remains ignorant to its prediction confidence. Orthogonally to Bayesian neural nets that indi…
R2D2-Net improves Bayesian neural networks by preventing over-shrinkage of important weights.
This paper improves Bayesian neural nets by using local linearization.
MPHD transfers knowledge across different domains for Bayesian optimization.
We develop a scalable method for Bayesian neural networks with stochastic differential equations.
We propose a new Bayesian Neural Net formulation that affords variational inference for which the evidence lower bound is analytically tractable subject to a tight approximation. We achieve this tractability by (i) decomposing ReLU nonlinearities into the product of an identity and a Heaviside step function, (ii) intro…
Improves model accuracy for neural nets in stochastic dynamics with partial prior knowledge.
We describe Bayesian Layers, a module designed for fast experimentation with neural network uncertainty. It extends neural network libraries with drop-in replacements for common layers. This enables composition via a unified abstraction over deterministic and stochastic functions and allows for scalability via the unde…
Proposes a new generalization bound for Bayesian deep nets without strict assumptions.
Two models predict net loan losses using Bayesian and frequentist regression.
New bound relaxes uniform gradient norm assumptions for PAC-Bayesian bounds.
Rank-1 BNNs improve efficiency and scalability of Bayesian neural nets.
PNCs balance tractability and expressiveness in probabilistic modeling.
We propose a novel method for closed-form predictive distribution modeling with neural nets. In quantifying prediction uncertainty, we build on Evidential Deep Learning, which has been impactful as being both simple to implement and giving closed-form access to predictive uncertainty. We employ it to model aleatoric un…
This paper improves ensemble learning for vision tasks by encouraging diversity in predictions.
Large multilayer neural networks trained with backpropagation have recently achieved state-of-the-art results in a wide range of problems. However, using backprop for neural net learning still has some disadvantages, e.g., having to tune a large number of hyperparameters to the data, lack of calibrated probabilistic pr…
This thesis builds theoretical foundations for deep learning, proving complexity theorems and training algorithms.
Hierarchical Bayesian networks and neural networks with stochastic hidden units are commonly perceived as two separate types of models. We show that either of these types of models can often be transformed into an instance of the other, by switching between centered and differentiable non-centered parameterizations of …
TyXe enables flexible Bayesian neural networks in Pytorch.
Effective training of deep neural networks suffers from two main issues. The first is that the parameter spaces of these models exhibit pathological curvature. Recent methods address this problem by using adaptive preconditioning for Stochastic Gradient Descent (SGD). These methods improve convergence by adapting to th…
Bayesian neural networks predict stress fields and uncertainty in materials.
BCD Nets use variational inference to estimate DAGs with uncertainty.
An important class of distance metrics proposed for training generative adversarial networks (GANs) is the integral probability metric (IPM), in which the neural net distance captures the practical GAN training via two neural networks. This paper investigates the minimax estimation problem of the neural net distance ba…
We present a new algorithm to train a robust neural network against adversarial attacks. Our algorithm is motivated by the following two ideas. First, although recent work has demonstrated that fusing randomness can improve the robustness of neural networks (Liu 2017), we noticed that adding noise blindly to all the la…
New method solves high-dimensional Bayesian inverse problems efficiently.
Despite the phenomenal success of deep learning in recent years, there remains a gap in understanding the fundamental mechanics of neural nets. More research is focussed on handcrafting complex and larger networks, and the design decisions are often ad-hoc and based on intuition. Some recent research has aimed to demys…
In this paper, we introduce transformations of deep rectifier networks, enabling the conversion of deep rectifier networks into shallow rectifier networks. We subsequently prove that any rectifier net of any depth can be represented by a maximum of a number of functions that can be realized by a shallow network with a …
Stochastic neural net weights are used in a variety of contexts, including regularization, Bayesian neural nets, exploration in reinforcement learning, and evolution strategies. Unfortunately, due to the large number of weights, all the examples in a mini-batch typically share the same weight perturbation, thereby limi…
A scalable method for Bayesian inference in large linear models.
Kernel methods outperform neural nets in operator learning tasks.
GIT-Net uses neural networks to approximate PDE operators efficiently.
DP-Net uses dynamic programming for efficient deep neural network compression.
Proposes HDBEN for heteroscedastic regression with improved sparsity and variance modeling.
The vast majority of current machine learning algorithms are designed to predict single responses or a vector of responses, yet many types of response are more naturally organized as matrices or higher-order tensor objects where characteristics are shared across modes. We present a new machine learning algorithm BaTFLE…
We challenge the longstanding assumption that the mean-field approximation for variational inference in Bayesian neural networks is severely restrictive, and show this is not the case in deep networks. We prove several results indicating that deep mean-field variational weight posteriors can induce similar distribution…
Study measures impact of data and neural net similarity on transferability in restaurant sales data.
Learning to Optimize is a recently proposed framework for learning optimization algorithms using reinforcement learning. In this paper, we explore learning an optimization algorithm for training shallow neural nets. Such high-dimensional stochastic optimization problems present interesting challenges for existing reinf…
This paper considers the power of deep neural networks (deep nets for short) in realizing data features. Based on refined covering number estimates, we find that, to realize some complex data features, deep nets can improve the performances of shallow neural networks (shallow nets for short) without requiring additiona…
DNF-Net tackles tabular data challenges with neural architecture.
We derive generalization and excess risk bounds for neural nets using a family of complexity measures based on a multilevel relative entropy. The bounds are obtained by introducing the notion of generated hierarchical coverings of neural nets and by using the technique of chaining mutual information introduced in Asadi…
fSDE-Net generates time series with long-term memory using neural networks.
New algorithm reduces neural net error in contextual bandits.
Proposes -Nets, polynomial neural networks, for improved representation power.