Paper explores weighted averaging schemes for SGD, achieving asymptotic normality and optimality.
problem Improving convergence of SGD in various settings.
method Develops a general weighted averaging scheme for SGD and establishes asymptotic normality.
result Establishes asymptotic normality and optimality of weighted averaged SGD solutions.
The time average of geometric Brownian motion plays a crucial role in the pricing of Asian options in mathematical finance. In this paper we consider the asymptotics of the discrete-time average of a geometric Brownian motion sampled on uniformly spaced times in the limit of a very large number of averaging time steps.…
Study on stochastic approximation with Polyak-Ruppert averaging for linear systems.
problem Understanding the asymptotic and non-asymptotic properties of stochastic approximation procedures.
method Detailed analysis of linear stochastic approximation with Polyak-Ruppert averaging, focusing on asymptotic and non-asymptotic properties.
result Proves CLT and non-asymptotic concentration inequality for averaged iterates, providing refined understanding of linear stochastic approximation.
The study calculates the average genus of 2-bridge knots based on their crossing numbers.
problem Determining the average genus of 2-bridge knots with a given crossing number.
method Analytical approach focusing on the properties of 2-bridge knots.
result Obtained the oblique asymptote of the average genus as crossing numbers increase.
We introduce a simple algorithm, True Asymptotic Natural Gradient Optimization (TANGO), that converges to a true natural gradient descent in the limit of small learning rates, without explicit Fisher matrix estimation. For quadratic models the algorithm is also an instance of averaged stochastic gradient, where the par…
SGDM accelerates faster than SGD with large batch sizes and permits broader learning rates.
problem Understanding the role of momentum in SGDM and its convergence rates.
method Analysis of SGDM convergence rates under strongly convex settings, including finite-sample rates and asymptotic normality of the averaged estimator.
result SGDM converges faster than SGD with large batch sizes and permits broader learning rates.
The paper develops time-uniform inference methods for stochastic approximation parameters.
problem Statistical inference for parameters in stochastic approximation problems.
method Analysis of averaged iterates convergence rates and construction of asymptotic confidence sequences.
result Valid asymptotic confidence sequences for parameters in stochastic approximation problems.
Study Vassiliev invariants and periodic orbits of Axiom A flows.
problem Calculating Vassiliev invariants and writhe for periodic orbits of Axiom A flows.
method Asymptotic analysis of Vassiliev invariants and writhe.
result Obtained asymptotics for Vassiliev invariants and writhe of periodic orbits.
This work proposes a model averaging method for SVM that avoids redundant covariates and achieves asymptotic optimality.
problem Redundant covariates impair SVM performance in high-dimensional settings.
method Frequentist model averaging procedure for SVM using cross-validation to select optimal weights.
result The proposed method achieves asymptotic optimality in SVM model averaging.
Paper develops bounds for stochastic approximation with averaging.
problem Establish high-probability bounds for averaged stochastic approximation.
method Develops a general framework for non-asymptotic concentration bounds.
result Derives sharp bounds for averaged iterates and tightens existing results.
New model shows average genus of 2-bridge knots grows linearly with crossing number.
problem Understanding the growth of Seifert genus for 2-bridge knots.
method Billiard table model for 2-bridge knots.
result Average genus of a 2-bridge knot with crossing number c asymptotically approaches c/4 + 1/12.
Paper introduces stability in model averaging and proposes a L2-penalty method.
problem Theoretical properties of model averaging from stability perspective.
method Introduces stability, defines asymptotic empirical risk minimizer, and proposes L2-penalty model averaging method.
result Proposed L2-penalty method ensures stability and consistency under reasonable conditions.
We extend a recent synchronization analysis of exact finite-state sources to nonexact sources for which synchronization occurs only asymptotically. Although the proof methods are quite different, the primary results remain the same. We find that an observer's average uncertainty in the source state vanishes exponential…
Stochastic algo learns from evolving data, achieving optimal performance.
problem Performative prediction and multiplayer extensions.
method Stochastic approximation with decision-dependent distributions.
result Asymptotic normality and optimality of the algorithm's performance.
We provide non-asymptotic convergence rates of the Polyak-Ruppert averaged stochastic gradient descent (SGD) to a normal random vector for a class of twice-differentiable test functions. A crucial intermediate step is proving a non-asymptotic martingale central limit theorem (CLT), i.e., establishing the rates of conve…
This paper introduces time-uniform CLT-based confidence intervals for statistical inference.
problem Developing valid statistical inference methods for sequential data.
method Time-uniform central limit theory and strong invariance principles.
result Asymptotic confidence sequences (CSs) that are uniformly valid over time.
Paper approximates risk measures using SGD with Langevin dynamics.
problem Approximating arbitrary law invariant risk measures.
method Stochastic Gradient Langevin Dynamics (SGD-Langevin) for general risk measures.
result Non-asymptotic convergence rates of the approximation algorithm.
We propose and analyze a variant of the classic Polyak-Ruppert averaging scheme, broadly used in stochastic gradient methods. Rather than a uniform average of the iterates, we consider a weighted average, with weights decaying in a geometric fashion. In the context of linear least squares regression, we show that this …
In this paper, we study the Edgeworth expansion for a pre-averaging estimator of quadratic variation in the framework of continuous diffusion models observed with noise. More specifically, we obtain a second order expansion for the joint density of the estimators of quadratic variation and its asymptotic variance. Our …
The study of 2-bridge knots reveals a linear average braid index as crossing number increases.
problem Understanding the distribution of braid indices in 2-bridge knots.
method Analyzing the asymptotic behavior of braid indices for fixed crossing numbers.
result The average braid index of 2-bridge knots of crossing number c is asymptotically $ rac{c}{3}+ rac{11}{9}$. Optimizes prediction error method for time-varying models.
problem Achieving optimal prediction error rates for time-varying models.
method Nonlinear least squares method for time-varying parametric models.
result First rate-optimal non-asymptotic analysis for time-varying models.
The study analyzes when Bayesian averaging over decision trees is reliable.
problem When do Bayesian model averaging weights over decision trees provide reliable information?
method Closed-form solution for Bayesian decision trees with Catalan-exponential priors.
result Established a complete non-asymptotic theory of rational commitment thresholds.
Proves a quantum invariant conjecture for specific three-manifolds.
problem Proving a conjecture about quantum invariants for certain three-manifolds.
method Gives an integral representation and resurgent asymptotic expansion for the average of GPPV invariants.
result Proves the Costantino--Geer--Patureau-Mirand invariant for negative definite plumbed three-manifolds.
We generalise the average asymptotic linking number of a pair of divergence-free vector fields on homology three-spheres by considering the linking of a divergence-free vector field on a manifold of arbitrary dimension with a codimension two foliation endowed with an invariant transverse measure. We prove that the aver…
Study Q-learning with averaging for reinforcement learning, proving efficient inference and error bounds.
problem Efficient inference and error bounds for Q-learning with averaging.
method Functional central limit theorem and asymptotic linear estimator for optimal Q-value function.
result Standardized partial-sum process converges weakly to a rescaled Brownian motion, matching instance-dependent lower bound for error.
The paper offers precise bounds for averaged LSA iterates in linear systems.
problem Computing approximate solutions of linear systems with noisy observations.
method Finite-time analysis of LSA algorithms with Polyak-Ruppert averaging.
result Sharp high-probability bounds for averaged LSA iterates.
Distributed statistical inference has recently attracted enormous attention. Many existing work focuses on the averaging estimator. We propose a one-step approach to enhance a simple-averaging based distributed estimator. We derive the corresponding asymptotic properties of the newly proposed estimator. We find that th…
This paper introduces sample-averaged Q-learning for better RL performance.
problem Improving reinforcement learning algorithms by managing uncertainty.
method Integrates statistical inference into Q-learning through sample averaging and functional central limit theorem.
result Establishes a unified theoretical foundation for sample-averaged Q-learning.
We formulate and study a general family of (continuous-time) stochastic dynamics for accelerated first-order minimization of smooth convex functions. Building on an averaging formulation of accelerated mirror descent, we propose a stochastic variant in which the gradient is contaminated by noise, and study the resultin…
This paper studies robust estimation methods in high dimensions, comparing model-averaged and composite quantile estimators.
problem Understanding robustness in high-dimensional regularized estimation.
method Optimal weights are determined by minimizing the asymptotic mean squared error, incorporating regularization effects without perfect selection.
result Model-averaged and composite quantile estimators often outperform least-squares methods in prediction quality.
This work analyzes nonexpansive stochastic approximations with Markovian noise, proving convergence in reinforcement learning.
problem Applying stochastic approximation to reinforcement learning settings with nonexpansive operators.
method Investigates nonexpansive stochastic approximations with Markovian noise, providing asymptotic and finite sample analysis.
result First-time proof of convergence for classical tabular average reward temporal difference learning.
A framework to compare federated learning algorithms in high-dimensional settings.
problem Comparing the performance of federated learning algorithms in high-dimensional settings.
method Formulating federated learning as a multi-criterion objective and analyzing a linear regression model.
result Federated Averaging with simple client fine-tuning achieves the same asymptotic risk as more intricate approaches and outperforms without personalization.
We develop a new method for the calculation of the heat trace asymptotics of the Laplacian on symmetric spaces that is based on a representation of the heat semigroup in form of an average over the Lie group of isometries and obtain a generating function for the whole sequence of all heat invariants.
We find the entropy's infinite-size behavior in complex manifold sections.
problem Determining entropy behavior in complex manifold sections.
method Analyzing entanglement entropy in tensor powers of hermitian line bundles.
result Asymptotic formula for expected entanglement entropy.
We calculate the asymptotic average rate at which a generic geodesic on a finite area hyperbolic 2-orbifold returns to an embedded disc on the surface, as well as the average amount of time it spends in the disc during each visit. This includes the case where the center of the disc is a cone point.
Gromov proposed an averaged version of the Dehn function and claimed that in many cases it should be subasymptotic to the Dehn function. Using results on random walks in nilpotent groups, we confirm this claim for most nilpotent groups. In particular, if a nilpotent group satisfies the isoperimetric inequality $δ(l)<Cl…
New streaming methods improve convergence rates for optimization problems.
problem Optimizing large-scale, sequential data problems.
method Time-varying mini-batches and Polyak-Ruppert averaging for gradient-based algorithms.
result Time-varying mini-batches and averaging achieve optimal convergence and variance reduction.
New algorithms improve estimation of treatment effects.
problem Estimating Average Treatment Effect (ATE) in adaptive settings.
method Optimistic algorithms for adaptive estimation using AIPW estimator.
result Significant theoretical and empirical gains over prior methods.
Optimal model averaging for conditional generative models improves performance across various data types.
problem Multiple plausible generators for conditional distributions can vary in performance.
method Sample-based maximum mean discrepancy, static model averaging, and mixture-of-experts model averaging.
result MoEMA improves over competing baselines across various data types.
We present a rigorous study of the short maturity asymptotics for Asian options with continuous-time averaging, under the assumption that the underlying asset follows the Constant Elasticity of Variance (CEV) model. We present an analytical approximation for the Asian options prices which has the appropriate short matu…
Paper introduces new estimator for continuous treatment effects.
problem Estimating the average dose-response function of continuous treatments.
method Utilizes ADML and DML tools, with a novel debiasing method.
result Proves asymptotic normality and shows good performance in simulations.
Consider a family of portfolio strategies with the aim of achieving the asymptotic growth rate of the best one. The idea behind Cover's universal portfolio is to build a wealth-weighted average which can be viewed as a buy-and-hold portfolio of portfolios. When an optimal portfolio exists, the wealth-weighted average c…
Paper proves CLTs for Q-learning with asynchronous updates.
problem Establishing convergence rates for Q-learning algorithms.
method Polyak-Ruppert averaging, non-asymptotic and functional CLTs.
result Convergence rates in Wasserstein distance for Q-learning.
Paper improves Bayesian inference in federated learning with new algorithm VR-FALD*.
problem Bayesian inference in federated learning with communication bottlenecks and statistical heterogeneity.
method Federated Averaging Langevin Dynamics (FALD) and VR-FALD*.
result VR-FALD* corrects client drift due to statistical heterogeneity, improving convergence.
We consider a finite simplicial complex K together with its successive barycentric subdivisions Sdd(K),d≥0, and study the expected topology of a random subcomplex in Sdd(K),d≫0. We get asymptotic upper and lower bounds for the expected Betti numbers of those subcomplexes, together with the average Morse …
Quantum walks blend patterns into splines when averaged.
problem Understanding the asymptotic patterns of quantum random walks.
method Averaging over quantum coins using the Haar measure.
result Patterns blend into splines, showing a unified behavior.
We study the asymptotic behavior of distribution densities arising in stock price models with stochastic volatility. The main objects of our interest in the present paper are the density of time averages of the squared volatility process and the density of the stock price process in the Stein-Stein and the Heston model…
New invariant for square-free integers derived from kei theory.
problem Developing numerical invariants for square-free integers.
method Defining a kei for each square-free integer and calculating a coloring invariant.
result Conjecture and proof of asymptotic average order for coloring invariant.