A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Bayesian priors for neural networks are improved by incorporating weight correlations and tail behavior.
Proposes MOPED method for choosing priors in Bayesian DNNs.
This paper introduces hierarchical Gaussian process priors for neural networks to capture weight correlations and inductive biases.
Bayesian weight priors improve neural network learning of identity relations.
Bayesian inference is known to provide a general framework for incorporating prior knowledge or specific properties into machine learning models via carefully choosing a prior distribution. In this work, we propose a new type of prior distributions for convolutional neural networks, deep weight prior (DWP), that exploi…
R2D2-Net improves Bayesian neural networks by preventing over-shrinkage of important weights.
This work explores how overparametrization and priors affect Bayesian neural network posteriors.
Existing Bayesian treatments of neural networks are typically characterized by weak prior and approximate posterior distributions according to which all the weights are drawn independently. Here, we consider a richer prior distribution in which units in the network are represented by latent variables, and the weights b…
Improves feature selection in high-dimensional data using LLM-generated weights.
Variational dropout (VD) is a generalization of Gaussian dropout, which aims at inferring the posterior of network weights based on a log-uniform prior on them to learn these weights as well as dropout rate simultaneously. The log-uniform prior not only interprets the regularization capacity of Gaussian dropout in netw…
Learning probability distributions on the weights of neural networks (NNs) has recently proven beneficial in many applications. Bayesian methods, such as Stein variational gradient descent (SVGD), offer an elegant framework to reason about NN model uncertainty. However, by assuming independent Gaussian priors for the i…
Prior-weighted logistic regression has become a standard tool for calibration in speaker recognition. Logistic regression is the optimization of the expected value of the logarithmic scoring rule. We generalize this via a parametric family of proper scoring rules. Our theoretical analysis shows how different members of…
Study on Bayesian transformers finds issues with weight-space inference and prior specification.
Bayesian deep learning with heavy-tailed weights achieves near-optimal performance.
We propose a novel approach for nonlinear regression using a two-layer neural network (NN) model structure with sparsity-favoring hierarchical priors on the network weights. We present an expectation propagation (EP) approach for approximate integration over the posterior distribution of the weights, the hierarchical s…
Study connects Gaussian processes and regularization for sequence-function mappings.
This paper solves aggregation of Pareto optimal models by using Bayesian priors and weighted averaging.
Bayesian optimization (BO) is a widely-used method for optimizing expensive (to evaluate) problems. At the core of most BO methods is the modeling of the objective function using a Gaussian Process (GP) whose covariance is selected from a set of standard covariance functions. From a weight-space view, this models the o…
Paper addresses variational inference issues in Bayesian neural networks.
New method improves BLL models for complex datasets.
Unified framework for Bayesian PDE-constrained inversion using physics-informed neural networks.
Prior knowledge can be used to improve predictive performance of learning algorithms or reduce the amount of data required for training. The same goal is pursued within the learning using privileged information paradigm which was recently introduced by Vapnik et al. and is aimed at utilizing additional information avai…
Hybrid Bayesian neural networks use function uncertainty for probabilistic inference.
We investigate deep Bayesian neural networks with Gaussian weight priors and a class of ReLU-like nonlinearities. Bayesian neural networks with Gaussian priors are well known to induce an L2, "weight decay", regularization. Our results characterize a more intricate regularization effect at the level of the unit activat…
Method identifies change points in high-dimensional models using sample weights.
Boltzmann machines are powerful distributions that have been shown to be an effective prior over binary latent variables in variational autoencoders (VAEs). However, previous methods for training discrete VAEs have used the evidence lower bound and not the tighter importance-weighted bound. We propose two approaches fo…
A new method for Bayesian neural networks using probabilistic backpropagation.
We study Generalised Restricted Boltzmann Machines with generic priors for units and weights, interpolating between Boolean and Gaussian variables. We present a complete analysis of the replica symmetric phase diagram of these systems, which can be regarded as Generalised Hopfield models. We underline the role of the r…
Structured sparsity has recently emerged in statistics, machine learning and signal processing as a promising paradigm for learning in high-dimensional settings. All existing methods for learning under the assumption of structured sparsity rely on prior knowledge on how to weight (or how to penalize) individual subsets…
Theory of learning with weight-distribution constraints.
No regularization needed for InLDL, achieving efficient and effective model.
New method learns priors for Bayesian neural networks from datasets.
Bayesian models that mix multiple Dirichlet prior parameters, called Multi-Dirichlet priors (MD) in this paper, are gaining popularity. Inferring mixing weights and parameters of mixed prior distributions seems tricky, as sums over Dirichlet parameters complicate the joint distribution of model parameters. This paper s…
We consider a Gaussian process formulation of the multiple kernel learning problem. The goal is to select the convex combination of kernel matrices that best explains the data and by doing so improve the generalisation on unseen data. Sparsity in the kernel weights is obtained by adopting a hierarchical Bayesian approa…
Bayesian neural networks with dependent weights converge to Gaussian mixtures.
Paper improves compressed sensing with prior probability information.
Bayesian PROCOVA uses AI to adjust for covariates in RCTs.
Variational Bayesian neural networks (BNNs) perform variational inference over weights, but it is difficult to specify meaningful priors and approximate posteriors in a high-dimensional weight space. We introduce functional variational Bayesian neural networks (fBNNs), which maximize an Evidence Lower BOund (ELBO) defi…
Unified analysis of privacy leakage in correlated data considering prior knowledge.
Adaptive ensemble improves flu forecasts with minimal data.
A Bayesian nonparametric approach for continual learning using neural networks.
We propose a novel method for compressed sensing recovery using untrained deep generative models. Our method is based on the recently proposed Deep Image Prior (DIP), wherein the convolutional weights of the network are optimized to match the observed measurements. We show that this approach can be applied to solve any…
We consider multi-task regression models where the observations are assumed to be a linear combination of several latent node functions and weight functions, which are both drawn from Gaussian process priors. Driven by the problem of developing scalable methods for forecasting distributed solar and other renewable powe…
Researchers derive exact priors for finite Bayesian neural networks.
The paper proposes a method to integrate prior information into penalized regression.
Sharp asymptotics derived for phase retrieval and compressed sensing with random generative priors.
In the classic sparsity-driven problems, the fundamental L-1 penalty method has been shown to have good performance in reconstructing signals for a wide range of problems. However this performance relies on a good choice of penalty weight which is often found from empirical experiments. We propose an algorithm called t…
Method reweights auxiliary tasks to reduce data need for main task.