arXiv research

Exponential rate of convergence for harmonic heat flow maps.

Near-Exponential Convergence Rates for kNN Classification

Study on convergence rate of $Q$-curvature flow in 6 dimensions.

Paper shows SVM can achieve super fast convergence rates.

MSTGD optimizes gradient descent with stratified sampling for faster convergence.

MSGD outperforms SGD in overparametrized settings with faster convergence rates.

We prove that for analytic functions in low dimension, the convergence rate of the deep neural network approximation is exponential.

This paper presents a Bayesian optimization method with exponential convergence without the need of auxiliary optimization and without the delta-cover sampling. Most Bayesian optimization methods require auxiliary optimization: an additional non-convex global optimization problem, which can be time-consuming and hard t…

Boosting with tempered exponential measures improves AdaBoost's convergence rate.

Optimization rates improved for manifolds with bounded geometry.

Study on Langevin dynamics convergence rates and their application to GAN training.

Estimates the rate of convergence of mean curvature flow solutions.

We estimate from above the rate at which a solution to the normalized Ricci flow on a closed manifold may converge to a limit soliton. Our main result implies that any solution which converges modulo diffeomorphisms to a soliton faster than any fixed exponential rate must itself be self-similar.

We consider stochastic gradient descent and its averaging variant for binary classification problems in a reproducing kernel Hilbert space. In the traditional analysis using a consistency property of loss functions, it is known that the expected classification error converges more slowly than the expected risk even whe…

A new topology improves decentralized learning efficiency and accuracy.

We consider the action of a pseudo-Anosov mapping class on $\mathcal{PML}(S)$. This action has north-south dynamics and so, under iteration, laminations converge exponentially to the stable lamination. We study the rate of this convergence and give examples of families of pseudo-Anosov mapping classes where the rate go…

We consider binary classification problems with positive definite kernels and square loss, and study the convergence rates of stochastic gradient methods. We show that while the excess testing loss (squared loss) converges slowly to zero as the number of observations (and thus iterations) goes to infinity, the testing …

Paper proposes an algorithm to recover full supervision from weakly labeled data.

Kernel estimator optimally recovers function from noisy exponential Radon transform.

EM algorithm converges in KL divergence for exponential families via mirror descent.

Proves convergence of mean curvature flow on cylinders with unique continuation.

New stability bounds for Sinkhorn's algorithm in entropic optimal transport.

The MAP estimate's log-likelihood sub-optimality is hard to bound in general.

Paper analyzes faster convergence rates for reinforcement learning from offline data.

This paper studies stability of the exponential utility maximization when there are small variations on agent's utility function. Two settings are considered. First, in a general semimartingale model where random endowments are present, a sequence of utilities defined on R converges to the exponential utility. Under a …

Study shows exponential error reduction in multiclass classification without bias-variance trade-off.

We study the behavior of the critical price of an American put option near maturity in the exponential Lévy model when the underlying stock pays dividends at a continuous rate. In particular, we prove that, in situations where the limit of the critical price is equal to the stock price, the rate of convergence to the l…

Adapts SGD to noise and problem specifics for faster convergence.

Study proves uniqueness of asymptotic limits for specific manifolds.

Adaptive importance samplers are adaptive Monte Carlo algorithms to estimate expectations with respect to some target distribution which \textit{adapt} themselves to obtain better estimators over a sequence of iterations. Although it is straightforward to show that they have the same $\mathcal{O}(1/\sqrt{N})$ convergen…

We provide non-asymptotic bounds for the well-known temporal difference learning algorithm TD(0) with linear function approximators. These include high-probability bounds as well as bounds in expectation. Our analysis suggests that a step-size inversely proportional to the number of iterations cannot guarantee optimal …

Although kernel methods are widely used in many learning problems, they have poor scalability to large datasets. To address this problem, sketching and stochastic gradient methods are the most commonly used techniques to derive efficient large-scale learning algorithms. In this study, we consider solving a binary class…

Paper improves deep learning convergence rates for low-dimensional data.

Paper establishes convergence rates and concentration bounds for stochastic approximation and reinforcement learning with Markovian noise.

Improved averaging method for noisy observations converges strongly.

Gradient methods converge exponentially in concave network games.

This work studies clustering in transformer models, proving exponential convergence to a single token state.

We study the Heston-Cox-Ingersoll-Ross++ stochastic-local volatility model in the context of foreign exchange markets and propose a Monte Carlo simulation scheme which combines the full truncation Euler scheme for the stochastic volatility component and the stochastic domestic and foreign short interest rates with the …

Improved convergence rates for saddle-point optimization algorithms.

This paper studies a class of exponential family models whose canonical parameters are specified as linear functionals of an unknown infinite-dimensional slope function. The optimal minimax rates of convergence for slope function estimation are established. The estimators that achieve the optimal rates are constructed …

AdamNX improves Adam's stability by adjusting its learning rate.

New loss function improves convergence rate for neural networks.

The AdaBoost algorithm was designed to combine many "weak" hypotheses that perform slightly better than random guessing into a "strong" hypothesis that has very low error. We study the rate at which AdaBoost iteratively converges to the minimum of the "exponential loss." Unlike previous work, our proofs do not require …

In our recent paper, we showed that in exponential family, contrastive divergence (CD) with fixed learning rate will give asymptotically consistent estimates \cite{wu2016convergence}. In this paper, we establish consistency and convergence rate of CD with annealed learning rate $η_t$. Specifically, suppose CD-$m$ gener…

Paper extends MLFD to signed measures via bilevel approach.

Geometric analysis improves convergence of variational inference.

Gradient descent slows significantly in over-parameterized single neuron learning.

We study the problem of sampling from a distribution $\target$ using the Langevin Monte Carlo algorithm and provide rate of convergences for this algorithm in terms of Wasserstein distance of order $2$. Our result holds as long as the continuous diffusion process associated with the algorithm converges exponentially fa…