arXiv research

Study of maximum likelihood under biased constraints reveals novel degeneracies and anomalous statistical behavior.

Paper compares Bayesian and de-biased estimators for low-rank matrix completion.

The paper addresses optimal control on Riemannian manifolds, introducing biased splines for robotic systems.

In this analytical study we derive the optimal unbiased value estimator (MVU) and compare its statistical risk to three well known value estimators: Temporal Difference learning (TD), Monte Carlo estimation (MC) and Least-Squares Temporal Difference Learning (LSTD). We demonstrate that LSTD is equivalent to the MVU if …

New oracles improve stochastic optimization with noisy or biased measurements.

Consider a process, stochastic or deterministic, obtained by using a numerical integration scheme, or from Monte-Carlo methods involving an approximation to an integral, or a Newton-Raphson iteration to approximate the root of an equation. We will assume that we can sample from the distribution of the process from time…

New method stabilizes machine learning for physics-informed inverse problems.

Biased mean regression estimates factors exceeding expected loss or radiation release severity.

The Whittle likelihood is a widely used and computationally efficient pseudo-likelihood. However, it is known to produce biased parameter estimates for large classes of models. We propose a method for de-biasing Whittle estimates for second-order stationary stochastic processes. The de-biased Whittle likelihood can be …

The paper proposes biased advantage estimates for reinforcement learning, improving efficiency and performance.

This note studies a method for the efficient estimation of a finite number of unknown parameters from linear equations, which are perturbed by Gaussian noise. In case the unknown parameters have only few nonzero entries, the proposed estimator performs more efficiently than a traditional approach. The method consists o…

A number of optimization approaches have been proposed for optimizing nonconvex objectives (e.g. deep learning models), such as batch gradient descent, stochastic gradient descent and stochastic variance reduced gradient descent. Theory shows these optimization methods can converge by using an unbiased gradient estimat…

Derives equations for deep learning biases and weights, showing data complexity reduction.

New variance-reduction methods solve stochastic composite inclusions.

Develops an accelerated algorithm for solving nonmonotone generalized equations.

Random forests are powerful non-parametric regression method but are severely limited in their usage in the presence of randomly censored observations, and naively applied can exhibit poor predictive performance due to the incurred biases. Based on a local adaptive representation of random forests, we develop its regre…

Develops variance-reduced methods for solving generalized equations.

Analyze SGD with biased gradients, improving convergence rates and accuracy.

Strong inductive biases prevent harmless interpolation in overparameterized models.

The paper analyzes time-dependent streaming data with biased gradient estimates and proposes improved stochastic optimization methods.

How do we learn from biased data? Historical datasets often reflect historical prejudices; sensitive or protected attributes may affect the observed treatments and outcomes. Classification algorithms tasked with predicting outcomes accurately from these datasets tend to replicate these biases. We advocate a causal mode…

We introduce a simple method for nearly simultaneous computation of all moments needed for quasi maximum likelihood estimation of parameters in discretely observed stochastic differential equations commonly seen in finance. The method proposed in this papers is not restricted to any particular dynamics of the different…

Develops a method to estimate quantiles in censored data using random forests.

The paper tackles bandit problems with biased offline data by using causal methods.

A new biased gradient descent method for conditional stochastic optimization.

Estimates social network structure from random walk subgraphs.

In many applications, different populations are compared using data that are sampled in a biased manner. Under sampling biases, standard methods that estimate the difference between the population means yield unreliable inferences. Here we develop an inference method that is resilient to sampling biases and is able to …

Biased sampling and missing data complicates statistical problems ranging from causal inference to reinforcement learning. We often correct for biased sampling of summary statistics with matching methods and importance weighting. In this paper, we study nearest neighbor matching (NNM), which makes estimates of populati…

Noise in imputed values corrects biases in machine learning models.

RS-PINN uses randomized smoothing to speed up high-dimensional PDE simulations without sacrificing accuracy.

The paper introduces a new model to correct bias in treatment effect estimates due to sample selection.

This paper analyzes SGD with biased gradients for deep learning models.

New method estimates nested expectations with biased and antithetic sampling.

Improved estimator reduces bias in statistical learning models.

Previous algorithms for constructing regression tree models for longitudinal and multiresponse data have mostly followed the CART approach. Consequently, they inherit the same selection biases and computational difficulties as CART. We propose an alternative, based on the GUIDE approach, that treats each longitudinal d…

New method for estimating treatment effects without complex propensity models.

The paper proposes methods to estimate MCMC quality with couplings, bounding Wasserstein distance.

While the interpretability of machine learning models is often equated with their mere syntactic comprehensibility, we think that interpretability goes beyond that, and that human interpretability should also be investigated from the point of view of cognitive science. The goal of this paper is to discuss to what exten…

NeuralRBMLE tackles explore-exploit trade-offs in contextual bandits with neural networks.

This study examines biases in flow matching samplers using finite-sample estimation.

Extract symbolic models from deep learning with inductive biases.

We study high-dimensional Gaussian mixture classification using statistical physics methods.

We study the convergence rate of stochastic optimization of exact (NP-hard) objectives, for which only biased estimates of the gradient are available. We motivate this problem in the context of learning the structure and parameters of Ising models. We first provide a convergence-rate analysis of deterministic errors fo…

Contrastive Divergence (CD) and Persistent Contrastive Divergence (PCD) are popular methods for training the weights of Restricted Boltzmann Machines. However, both methods use an approximate method for sampling from the model distribution. As a side effect, these approximations yield significantly different biases and…

Helps visually impaired users make better decisions by adjusting their observations.

The paper corrects biases in estimating intrinsic dimension and differential entropy.

Algorithm identifies best arm with biased proxy and selective ground truth audits.

Neural networks solve high-dimensional HJB PDEs with asymptotic guarantees.