arXiv research

RP-WNO extends WNO with uncertainty quantification, useful for scientists and engineers.

We consider the problem of compressed sensing and of (real-valued) phase retrieval with random measurement matrix. We derive sharp asymptotics for the information-theoretically optimal performance and for the best known polynomial algorithm for an ensemble of generative priors consisting of fully connected deep neural …

Unified framework for Bayesian PDE-constrained inversion using physics-informed neural networks.

The weights of a neural network are typically initialized at random, and one can think of the functions produced by such a network as having been generated by a prior over some function space. Studying random networks, then, is useful for a Bayesian understanding of the network evolution in early stages of training. In…

Develops methods for constructing likelihoods and priors for Bayesian networks.

G-Net constructs binary neural networks with high accuracy using randomized binary embeddings.

Gaussian processes are ubiquitous in nature and engineering. A case in point is a class of neural networks in the infinite-width limit, whose priors correspond to Gaussian processes. Here we perturbatively extend this correspondence to finite-width neural networks, yielding non-Gaussian processes as priors. The methodo…

Randomly trained neural networks can generalize well if there's a simpler underlying teacher model.

This paper proposes a framework for certifying neural network defenses against data poisoning attacks.

Given an ensemble of randomized regression trees, it is possible to restructure them as a collection of multilayered neural networks with particular connection weights. Following this principle, we reformulate the random forest method of Breiman (2001) into a neural network setting, and in turn propose two new hybrid p…

The deep image prior was recently introduced as a prior for natural images. It represents images as the output of a convolutional network with random inputs. For "inference", gradient descent is performed to adjust network parameters to make the output match observations. This approach yields good performance on a rang…

One of the fundamental problems in supervised classification and in machine learning in general, is the modelling of non-parametric invariances that exist in data. Most prior art has focused on enforcing priors in the form of invariances to parametric nuisance transformations that are expected to be present in data. Le…

The likelihood model of high dimensional data $X_n$ can often be expressed as $p(X_n|Z_n,θ)$, where $θ\mathrel{\mathop:}=(θ_k)_{k\in[K]}$ is a collection of hidden features shared across objects, indexed by $n$, and $Z_n$ is a non-negative factor loading vector with $K$ entries where $Z_{nk}$ indicates the strength of …

Sparse random networks reduce communication in federated learning.

Bayesian neural networks with dependent weights converge to Gaussian mixtures.

Efficiently quantifies uncertainty in DeepONets for function spaces.

RODE-Net learns ODEs from data with random parameters using neural networks and GANs.

We extend the Deep Image Prior (DIP) framework to one-dimensional signals. DIP is using a randomly initialized convolutional neural network (CNN) to solve linear inverse problems by optimizing over weights to fit the observed measurements. Our main finding is that properly tuned one-dimensional convolutional architectu…

Purpose: Conventional automated segmentation of the head anatomy in MRI distinguishes different brain and non-brain tissues based on image intensities and prior tissue probability maps (TPM). This works well for normal head anatomies, but fails in the presence of unexpected lesions. Deep convolutional neural networks l…

Deep Gaussian Processes are reinterpreted as deep trigonometric networks for tractable inference.

R2D2-Net improves Bayesian neural networks by preventing over-shrinkage of important weights.

A new model trains prior and encoder/decoder networks simultaneously for efficient generation.

Bayesian neural networks use temperature adjustments to improve predictive performance.

Enhances MMSB for complex graph structures with HL-MRF priors.

Dealing with uncertainty is essential for efficient reinforcement learning. There is a growing literature on uncertainty estimation for deep learning from fixed datasets, but many of the most popular approaches are poorly-suited to sequential decision problems. Other methods, such as bootstrap sampling, have no mechani…

This paper characterizes how randomized neural networks generalize well in multi-dimensional tasks.

Study characterizes training and test risks for MAP regression with Gaussian priors.

Proposes scale mixture of NNGPs for more flexible stochastic processes.

Bayesian optimization improved for high-dimensional outputs using randomized priors.

Bayesian deep learning with heavy-tailed weights achieves near-optimal performance.

Enhances GNN robustness during inference using Conditional Random Fields.

Generative models for graphs have been typically committed to strong prior assumptions concerning the form of the modeled distributions. Moreover, the vast majority of currently available models are either only suitable for characterizing some particular network properties (such as degree distribution or clustering coe…

Polynomial-time algorithm learns ReLU networks without assumptions.

Fine-tuning normalization layers can reconstruct smaller networks.

New model detects hidden group structures in criminal networks.

New method certifies neural network robustness under random input noise.

Bayesian deep learning uses function-space priors to improve model uncertainty and robustness.

Study compares random and learned features in deep Bayesian linear models.

We combine Riemannian geometry with the mean field theory of high dimensional chaos to study the nature of signal propagation in generic, deep neural networks with random weights. Our results reveal an order-to-chaos expressivity phase transition, with networks in the chaotic phase computing nonlinear functions whose g…

Random neural networks with ReLU activations are non-Gaussian processes.

DP-RandP improves privacy-utility tradeoff in DP-SGD by learning priors from random processes.

We propose a network structure discovery model for continuous observations that generalizes linear causal models by incorporating a Gaussian process (GP) prior on a network-independent component, and random sparsity and weight matrices as the network-dependent parameters. This approach provides flexible modeling of net…

A new method predicts links better across various networks.

We prove that the binary classifiers of bit strings generated by random wide deep neural networks with ReLU activation function are biased towards simple functions. The simplicity is captured by the following two properties. For any given input bit string, the average Hamming distance of the closest input bit string wi…

New algorithm samples neural network posteriors efficiently.

The paper improves bandit algorithms by incorporating random-effect models.

We consider a network of agents that aim to learn some unknown state of the world using private observations and exchange of beliefs. At each time, agents observe private signals generated based on the true unknown state. Each agent might not be able to distinguish the true state based only on her private observations.…

Deep generative priors offer powerful models for complex-structured data, such as images, audio, and text. Using these priors in inverse problems typically requires estimating the input and/or hidden signals in a multi-layer deep neural network from observation of its output. While these approaches have been successful…