arXiv research

This work studies the statistical performance of Sinkhorn iterations in estimating Schrödinger bridges.

A method for estimating parameters from entangled single-sample distributions, robust to high-noise data.

Faster WIND accelerates iterative BOND for LLM alignment.

New online method estimates OT distances from sample streams.

Adaptive SAA solves large-scale stochastic linear programs efficiently.

Exchangeable graphs arise via a sampling procedure from measurable functions known as graphons. A natural estimation problem is how well we can recover a graphon given a single graph sampled from it. One general framework for estimating a graphon uses step-functions obtained by partitioning the nodes of the graph accor…

The paper develops a theory for iterative self-improvement of models, proving conditions for better performance with easy-to-hard curricula.

iRDM selects unlabeled samples for regression without labels, improving model accuracy.

New privacy bounds for DP-SGD's last iterate, even with cyclic sampling.

This paper introduces Tree-Pyramidal Adaptive Importance Sampling (TP-AIS), a novel iterated sampling method that outperforms state-of-the-art approaches like deterministic mixture population Monte Carlo (DM-PMC), mixture population Monte Carlo (M-PMC) and layered adaptive importance sampling (LAIS). TP-AIS iteratively…

We introduce Bayesian least-squares policy iteration (BLSPI), an off-policy, model-free, policy iteration algorithm that uses the Bayesian least-squares temporal-difference (BLSTD) learning algorithm to evaluate policies. An online variant of BLSPI has been also proposed, called randomised BLSPI (RBLSPI), that improves…

DAC enhances exploration in reinforcement learning with entropy regularization.

Improved private learning of halfspaces with reduced sample complexity.

We present two sampled quasi-Newton methods (sampled LBFGS and sampled LSR1) for solving empirical risk minimization problems that arise in machine learning. Contrary to the classical variants of these methods that sequentially build Hessian or inverse Hessian approximations as the optimization progresses, our proposed…

A new algorithm reconstructs population dynamics from coarse samples.

Cubic regularization (CR) is an optimization method with emerging popularity due to its capability to escape saddle points and converge to second-order stationary solutions for nonconvex optimization. However, CR encounters a high sample complexity issue for finite-sum problems with a large data size. %Various inexact …

The paper shows how training with synthetic data can lead to model improvement, not degradation, under certain conditions.

New algorithm speeds up diffusion model sampling 4-14 times.

FVI method calculates bicausal OT with neural networks, outperforming other methods.

This paper resolves BIHT convergence, showing normalization is not necessary in noiseless settings but crucial for robustness.

Iterative method 'Concent' corrects spectrum bias in covariance matrices.

Polyak step size GD reaches final radius of convergence after log iterations.

IDBM solves Schrödinger bridge problems with iterative sampling.

In this paper, we propose to adopt the diffusion approximation tools to study the dynamics of Oja's iteration which is an online stochastic gradient descent method for the principal component analysis. Oja's iteration maintains a running estimate of the true principal component from streaming data and enjoys less tempo…

LARA forecasts financial asset trends by refining noisy labels and extracting profitable samples.

Optimal ridge regularization computed iteratively from generative parameters.

Paper improves worst-case regret bounds for RLSVI in reinforcement learning.

Extends JKO scheme for iterative algorithms with unknown parameters.

Gaussianization flows transform any random vector into a Gaussian, enabling efficient computation and sample generation.

Directed latent variable models that formulate the joint distribution as $p(x,z) = p(z) p(x \mid z)$ have the advantage of fast and exact sampling. However, these models have the weakness of needing to specify $p(z)$, often with a simple fixed prior that limits the expressiveness of the model. Undirected latent variabl…

The paper develops AMP theory for sparse and robust regression with polynomial iterations.

iDEM generates samples from Boltzmann densities without data.

Given a linear regression setting, Iterative Least Trimmed Squares (ILTS) involves alternating between (a) selecting the subset of samples with lowest current loss, and (b) re-fitting the linear model only on that subset. Both steps are very fast and simple. In this paper we analyze ILTS in the setting of mixed linear …

Support Vector Data Description (SVDD) is a popular outlier detection technique which constructs a flexible description of the input data. SVDD computation time is high for large training datasets which limits its use in big-data process-monitoring applications. We propose a new iterative sampling-based method for SVDD…

Generative model improves sample quality on ImageNet32.

Several approximate policy iteration schemes without value functions, which focus on policy representation using classifiers and address policy learning as a supervised learning problem, have been proposed recently. Finding good policies with such methods requires not only an appropriate classifier, but also reliable e…

Stochastic Gradient Descent or SGD is the most popular optimization algorithm for large-scale problems. SGD estimates the gradient by uniform sampling with sample size one. There have been several other works that suggest faster epoch-wise convergence by using weighted non-uniform sampling for better gradient estimates…

Derives error bounds for stochastic iterative algorithms using Stein's method.

We present an iterative Markov chainMonte Carlo algorithm for computingreference priors and minimax risk forgeneral parametric families. Ourapproach uses MCMC techniques based onthe Blahut-Arimoto algorithm forcomputing channel capacity ininformation theory. We give astatistical analysis of the algorithm,bounding the n…

Paper achieves $ε^{-2}$ sample complexity for actor-critic methods with minimal assumptions.

PDDS samples from unnormalized densities using iterative particle scheme.

Enhanced Sampling Scheme improves masked generative modeling.

We present a novel analysis of the dynamics of tensor power iterations in the overcomplete regime where the tensor CP rank is larger than the input dimension. Finding the CP decomposition of an overcomplete tensor is NP-hard in general. We consider the case where the tensor components are randomly drawn, and show that …

Scaled gradient descent improves matrix recovery for ill-conditioned matrices with optimal sampling complexity.

We consider the problem of supervised learning with convex loss functions and propose a new form of iterative regularization based on the subgradient method. Unlike other regularization approaches, in iterative regularization no constraint or penalization is considered, and generalization is achieved by (early) stoppin…

Enhances LMC for log-concave sampling, reducing computational cost.

In this paper, we propose a stochastic Primal-Dual Hybrid Gradient (PDHG) approach for solving a wide spectrum of regularized stochastic minimization problems, where the regularization term is composite with a linear function. It has been recognized that solving this kind of problem is challenging since the closed-form…

LIIT uses large deviations to train neural networks faster.