arXiv research

We consider the problem of compressed sensing and of (real-valued) phase retrieval with random measurement matrix. We derive sharp asymptotics for the information-theoretically optimal performance and for the best known polynomial algorithm for an ensemble of generative priors consisting of fully connected deep neural …

Study characterizes training and test risks for MAP regression with Gaussian priors.

RP-WNO extends WNO with uncertainty quantification, useful for scientists and engineers.

DP-RandP improves privacy-utility tradeoff in DP-SGD by learning priors from random processes.

The paper improves bandit algorithms by incorporating random-effect models.

New PAC-Bayes method updates priors without losing confidence information.

This paper formulates an utility indifference pricing model for investors trading in a discrete time financial market under non-dominated model uncertainty. The investors preferences are described by strictly increasing concave random functions defined on the positive axis. We prove that under suitable conditions the m…

We consider learning on graphs, guided by kernels that encode similarity between vertices. Our focus is on random walk kernels, the analogues of squared exponential kernels in Euclidean spaces. We show that on large, locally treelike, graphs these have some counter-intuitive properties, specifically in the limit of lar…

This paper analyzes and guarantees convergence of prior-guided ZO algorithms.

Investor optimizes investment strategy under model uncertainty and random utility.

A parametrization of hypergraphs based on the geometry of points in $\mathbf{R}^d$ is developed. Informative prior distributions on hypergraphs are induced through this parametrization by priors on point configurations via spatial processes. This prior specification is used to infer conditional independence models or M…

Method learns SDEs from one trajectory using GP priors and randomized cross-validation.

We present a nonparametric prior over reversible Markov chains. We use completely random measures, specifically gamma processes, to construct a countably infinite graph with weighted edges. By enforcing symmetry to make the edges undirected we define a prior over random walks on graphs that results in a reversible Mark…

The paper analyzes sparse high-dimensional linear regression with random design and unknown error variance, providing adaptiveness and concentration rates.

The likelihood model of high dimensional data $X_n$ can often be expressed as $p(X_n|Z_n,θ)$, where $θ\mathrel{\mathop:}=(θ_k)_{k\in[K]}$ is a collection of hidden features shared across objects, indexed by $n$, and $Z_n$ is a non-negative factor loading vector with $K$ entries where $Z_{nk}$ indicates the strength of …

Statistical physics approaches can be used to derive accurate predictions for the performance of inference methods learning from potentially noisy data, as quantified by the learning curve defined as the average error versus number of training examples. We analyse a challenging problem in the area of non-parametric inf…

This paper studies an optimal trading problem that incorporates the trader's market view on the terminal asset price distribution and uninformative noise embedded in the asset price dynamics. We model the underlying asset price evolution by an exponential randomized Brownian bridge (rBb) and consider various prior dist…

Master algorithm fails to detect non-stationarity in practical settings.

Unified framework for Bayesian PDE-constrained inversion using physics-informed neural networks.

Bayesian optimization improved for high-dimensional outputs using randomized priors.

Normalized compound random measures are flexible nonparametric priors for related distributions. We consider building general nonparametric regression models using normalized compound random measure mixture models. Posterior inference is made using a novel pseudo-marginal Metropolis-Hastings sampler for normalized comp…

The weights of a neural network are typically initialized at random, and one can think of the functions produced by such a network as having been generated by a prior over some function space. Studying random networks, then, is useful for a Bayesian understanding of the network evolution in early stages of training. In…

Prior design is one of the most important problems in both statistics and machine learning. The cross validation (CV) and the widely applicable information criterion (WAIC) are predictive measures of the Bayesian estimation, however, it has been difficult to apply them to find the optimal prior because their mathematic…

Two EM algorithms estimate prior distributions in mixture of linear regressions.

We show that Entropy-SGD (Chaudhari et al., 2017), when viewed as a learning algorithm, optimizes a PAC-Bayes bound on the risk of a Gibbs (posterior) classifier, i.e., a randomized classifier obtained by a risk-sensitive perturbation of the weights of a learned classifier. Entropy-SGD works by optimizing the bound's p…

Randomly trained neural networks can generalize well if there's a simpler underlying teacher model.

Develops methods for constructing parameter priors in DAG models.

This paper proposes a framework for certifying neural network defenses against data poisoning attacks.

Efficiently quantifies uncertainty in DeepONets for function spaces.

We consider the correlated multiarmed bandit (MAB) problem in which the rewards associated with each arm are modeled by a multivariate Gaussian random variable, and we investigate the influence of the assumptions in the Bayesian prior on the performance of the upper credible limit (UCL) algorithm and a new correlated U…

AI-driven Bayesian inference improves decision-making uncertainty.

Gaussian processes are ubiquitous in nature and engineering. A case in point is a class of neural networks in the infinite-width limit, whose priors correspond to Gaussian processes. Here we perturbatively extend this correspondence to finite-width neural networks, yielding non-Gaussian processes as priors. The methodo…

Develops methods for constructing likelihoods and priors for Bayesian networks.

Dealing with uncertainty is essential for efficient reinforcement learning. There is a growing literature on uncertainty estimation for deep learning from fixed datasets, but many of the most popular approaches are poorly-suited to sequential decision problems. Other methods, such as bootstrap sampling, have no mechani…

Study explores geometric structure and prior for beta-logistic distribution.

Study tests financial market efficiency using random number generator tests.

We extend the Deep Image Prior (DIP) framework to one-dimensional signals. DIP is using a randomly initialized convolutional neural network (CNN) to solve linear inverse problems by optimizing over weights to fit the observed measurements. Our main finding is that properly tuned one-dimensional convolutional architectu…

A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, introduced by Johnstone, in which a prominent eigenvector (or "spike") is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughou…

G-Net constructs binary neural networks with high accuracy using randomized binary embeddings.

We present a consensus Monte Carlo algorithm that scales existing Bayesian nonparametric models for clustering and feature allocation to big data. The algorithm is valid for any prior on random subsets such as partitions and latent feature allocation, under essentially any sampling model. Motivated by three case studie…

Optimality of TS with noninformative priors proven for Pareto model.

New study shows limits to classifying brain activity from randomized EEG trials.

Bayesian deep learning uses function-space priors to improve model uncertainty and robustness.

New methods improve black-box adversarial attacks with fewer queries.

Bayesian model averaging (BMA) is the state of the art approach for overcoming model uncertainty. Yet, especially on small data sets, the results yielded by BMA might be sensitive to the prior over the models. Credal Model Averaging (CMA) addresses this problem by substituting the single prior over the models by a set …

The deep image prior was recently introduced as a prior for natural images. It represents images as the output of a convolutional network with random inputs. For "inference", gradient descent is performed to adjust network parameters to make the output match observations. This approach yields good performance on a rang…

Thompson sampling has impressive empirical performance for many multi-armed bandit problems. But current algorithms for Thompson sampling only work for the case of conjugate priors since these algorithms require to infer the posterior, which is often computationally intractable when the prior is not conjugate. In this …

Enhances CMS with Bayesian nonparametrics for better low-frequency token estimation.