arXiv research

The paper analyzes the risk of CV-tuned regularized estimators and connects it to SURE.

New method uses SURE to denoise signals, outperforming NPMLE.

Paper establishes convergence rates and concentration bounds for stochastic approximation and reinforcement learning with Markovian noise.

Paper proves convergence of SA algorithm via martingale and converse Lyapunov methods.

The paper analyzes convergence rates for stochastic approximation and reinforcement learning.

The purpose of this paper is to provide a sharp analysis on the asymptotic behavior of the Durbin-Watson statistic. We focus our attention on the first-order autoregressive process where the driven noise is also given by a first-order autoregressive process. We establish the almost sure convergence and the asymptotic n…

A new hybrid Newton algorithm improves convergence in logistic regression.

We show that gradient descent converges to a local minimizer, almost surely with random initialization. This is proved by applying the Stable Manifold Theorem from dynamical systems theory.

The paper analyzes convergence rates for SGD and SHB methods.

This paper formalizes $Q$-learning and linear TD convergence using Lean 4.

o1Neuro neural network approximates complex functions and converges quickly.

SGD converges almost surely in non-convex problems, avoiding saddle points and accelerating convergence.

New algorithm solves saddle point problems in Banach spaces.

The purpose of this paper is to provide further understanding into the structure of the sequential allocation ("stochastic multi-armed bandit", or MAB) problem by establishing probability one finite horizon bounds and convergence rates for the sample (or "pseudo") regret associated with two simple classes of allocation…

We propose a unified and systematic framework for performing online nonnegative matrix factorization in the presence of outliers. Our framework is particularly suited to large-scale data. We propose two solvers based on projected gradient descent and the alternating direction method of multipliers. We prove that the se…

Unrolled neural networks emerged recently as an effective model for learning inverse maps appearing in image restoration tasks. However, their generalization risk (i.e., test mean-squared-error) and its link to network design and train sample size remains mysterious. Leveraging the Stein's Unbiased Risk Estimator (SURE…

In this paper, we prove some convergence results of a special case of optimistic policy iteration algorithm for stochastic shortest path problem. We consider both Monte Carlo and $TD(λ)$ methods for the policy evaluation step under the condition that the termination state will eventually be reached almost surely.

Adam is a popular variant of stochastic gradient descent for finding a local minimizer of a function. In the constant stepsize regime, assuming that the objective function is differentiable and non-convex, we establish the convergence in the long run of the iterates to a stationary point under a stability condition. Th…

TSAW improves MCMC integral estimation with faster convergence.

We extend some properties of random walks on hyperbolic groups to random walks on convergence groups. In particular we prove that if a convergence group $G$ acts on a compact metrizable space $M$ with the convergence property then we can provide $G\cup M$ with a compact topology such that random walks on $G$ converge a…

In this work we construct an optimal linear shrinkage estimator for the covariance matrix in high dimensions. The recent results from the random matrix theory allow us to find the asymptotic deterministic equivalents of the optimal shrinkage intensities and estimate them consistently. The developed distribution-free es…

C-SURE improves complex-valued deep learning models by shrinking estimates, outperforming MLE and SurReal.

Using integration by parts on Gaussian space we construct a Stein Unbiased Risk Estimator (SURE) for the drift of Gaussian processes using their local and occupation times. By almost-sure minimization of the SURE risk of shrinkage estimators we derive an estimation and de-noising procedure for an input signal perturbed…

We present an actor-critic framework for MDPs where the objective is the variance-adjusted expected return. Our critic uses linear function approximation, and we extend the concept of compatible features to the variance-adjusted setting. We present an episodic actor-critic algorithm and show that it converges almost su…

New algorithm tackles complex optimization problems with inexact and stochastic methods.

We present a distributed (non-Bayesian) learning algorithm for the problem of parameter estimation with Gaussian noise. The algorithm is expressed as explicit updates on the parameters of the Gaussian beliefs (i.e. means and precision). We show a convergence rate of $O(1/k)$ with the constant term depending on the numb…

Develops new algorithms for solving root-finding problems in large-scale settings.

New method solves root-finding problems with faster convergence.

Paper analyzes convergence rates of SGD for non-convex functions under various assumptions.

Motivated by a recent result of Daskalakis et al. 2018, we analyze the population version of Expectation-Maximization (EM) algorithm for the case of \textit{truncated} mixtures of two Gaussians. Truncated samples from a $d$-dimensional mixture of two Gaussians $\frac{1}{2} \mathcal{N}(\vecμ, \vecΣ)+ \frac{1}{2} \mathca…

We study a tower of normal coverings over a compact Kähler manifold with holomorphic line bundles. When the line bundle is sufficiently positive, we obtain an effective estimate, which implies the Bergman stability. As a consequence, we deduce the equidistribution for zero currents of random holomorphic sections. Furth…

Paper analyzes convergence of two time-scale stochastic approximation using martingale approach.

Develops accelerated fixed-point methods with delayed oracles for scientific computing.

Develops an accelerated algorithm for solving nonmonotone generalized equations.

MSGD outperforms SGD in overparametrized settings with faster convergence rates.

We describe a simple and efficient procedure for approximating the Lévy measure of a $\text{Gamma}(α,1)$ random variable. We use this approximation to derive a finite sum-representation that converges almost surely to Ferguson's representation of the Dirichlet process based on arrivals of a homogeneous Poisson process.…

Paper proves convergence of measure transfer schemes using slicing and matching.

We determine the long-time asymptotic behavior of a relativistic diffusion taking values in the unitary tangent bundle of a Robertson-Walker space-time. We prove in particular that when approaching the explosion time of the diffusion, its projection on the base manifold almost surely converges to a random point of the …

Let G be a countable group which acts by isometries on a separable, but not necessarily proper, Gromov hyperbolic space X. We say the action of G is weakly hyperbolic if G contains two independent hyperbolic isometries. We show that a random walk on such G converges to the Gromov boundary almost surely. We apply the co…

In this paper, we study regression problems over a separable Hilbert space with the square loss, covering non-parametric regression over a reproducing kernel Hilbert space. We investigate a class of spectral/regularized algorithms, including ridge regression, principal component regression, and gradient methods. We pro…

Develops variance-reduced methods for solving generalized equations.

New SAGA algorithm with decreasing step for stochastic optimization.

Develops a new essential supremum concept for financial models.

New proof shows local wealth condensation in economic models with biases.

Testing-by-betting strategies almost surely go bankrupt under null hypotheses.

Kernel-based mean-field games use MMD penalties for interaction and target costs.

Kaimanovich and Masur showed that a random walk on the mapping class group for an initial distribution with finite first moment and whose support generates a non-elementary subgroup, converges almost surely to a point in the space PMF of projective measured foliations on the surface. This defines a harmonic measure on …

Algorithm estimates parameters over time-varying graphs without special assumptions.