arXiv research

Improved Sparse Polyak for high-dimensional M-estimation with sparser solutions.

Sparse Polyak improves high-dimensional statistical estimation.

Formulae for Vassiliev invariants derived from Kauffman polynomial.

In smooth strongly convex optimization, knowledge of the strong convexity parameter is critical for obtaining simple methods with accelerated rates. In this work, we study a class of methods, based on Polyak steps, where this knowledge is substituted by that of the optimal value, $f_*$. We first show slightly improved …

Polyak-Ruppert CLT for SA-Adam with momentum and non-convergent adaptive preconditioning

The paper improves convergence for linear systems using entropic mirror descent with Polyak stepsizes.

Proposes a new adaptive learning rate for SGD.

Polyak's momentum accelerates training of neural networks.

Polyak step size GD reaches final radius of convergence after log iterations.

New adaptive scheduler improves SAM for better model training.

The paper provides statistical guarantees for SGD and ASGD in high-dimensional settings.

We describe the space of arrow diagram formulas for virtual knot diagrams in the annulus as the kernel of a linear map, inspired from a conjecture due to M. Polyak. As a main application, we slightly improve Grishanov-Vassiliev's theorem for planar chain invariants.

Improved SGD methods converge faster for nonconvex optimization.

In this paper, it is shown that there are no nonconstant Goussarov-Polyak-Viro finite-type invariants that are invariant under the virtualization move. As an immediate corollary, we obtain the theorem which states none of the Birman coefficients of the Jones-Kauffman polynomial are of GPV finite type.

Study on stochastic approximation with Polyak-Ruppert averaging for linear systems.

We describe the Polyak-Viro arrow diagram formulas for the coefficients of the Conway polynomial. As a consequence, we obtain the Conway polynomial as a state sum over some subsets of the crossings of the knot diagram. It turns out to be a simplification of a special case of Jaeger's state model for the HOMFLY polynomi…

We enhance the biquandle counting invariant using elements of truncated biquandle-labeled Polyak algebras. These finite type enhancements reduce to the finite type enhancements defined by Goussarov, Polyak and Viro for the trivial biquandle of one element and determine (but are not determined by) the biquandle counting…

New step-size methods improve SHB convergence for stochastic optimization.

We present a new method to produce simple formulas for 1-cocycles of knots over the integers, inspired by Polyak-Viro's formulas for finite-type knot invariants. We conjecture that these formulas always represent finite-type cohomology classes in the sense of Vassiliev. An example of degree 3 is studied, and shown to c…

Formula for Milnor triple linking number in link diagrams with multiple crossings.

Although it is known that the dimension of the Vassiliev invariants of degree three of long virtual knots is seven, the complete list of seven distinct Gauss diagram formulas have been unknown explicitly, where only one known formula was revised without proof. In this paper, we give seven Gauss diagram formulas to pres…

Proposes momentum methods for Lie groups, improving on classical algorithms.

TD(0) with Polyak-Ruppert averaging achieves robust and fast convergence rates

Gradient descent with biased rounding errors converges faster under certain conditions.

The paper analyzes time-dependent streaming data with biased gradient estimates and proposes improved stochastic optimization methods.

Paper proves CLTs for Q-learning with asynchronous updates.

Study Q-learning with averaging for reinforcement learning, proving efficient inference and error bounds.

New streaming methods improve convergence rates for optimization problems.

Goussarov, Polyak, and Viro proved that finite type invariants of knots are ``finitely multi-local'', meaning that on a knot diagram, sums of quantities, defined by local information, determine the value of the knot invariant. The result implies the existence of Gauss diagram combinatorial formulas for finite type inva…

Large deviations theory applied to policy gradient methods.

New formulas classify higher-dimensional knots and links.

In 1963, Polyak proposed a simple condition that is sufficient to show a global linear convergence rate for gradient descent. This condition is a special case of the Łojasiewicz inequality proposed in the same year, and it does not require strong convexity (or even convexity). In this work, we show that this much-older…

The paper refines optimization algorithms using Lyapunov functions and differential equations.

Gradient method converges locally linearly for overparameterized Gaussian mixtures.

Minimal sets of moves for isotopic knots and trivalent graphs identified.

New SPS variant improves non-smooth optimization without small gradients.

Stochastic GD converges linearly for CV@R learning under certain conditions.

Paper improves confidence intervals for LSA with multiplier bootstrap.

Gradient descent with large momentum finds flatter minima.

Gradient descent converges linearly for overparameterized linear networks.

Study of non-convex potential functions in deep learning with Poincaré inequality.

The paper offers precise bounds for averaged LSA iterates in linear systems.

Paper analyzes normal approximation for two-timescale stochastic algorithms, revealing interaction between fast and slow timescales.

Paper improves confidence set construction for SGD using multiplier bootstrap.

Two new polynomial invariants for long virtual knots.

Paper develops bounds for stochastic approximation with averaging.

Analyze SGD with biased gradients, improving convergence rates and accuracy.

Polyak proved that the set $\{\Omega1a,\Omega1b,\Omega2a,\Omega3a\}$ is a minimal generating set of oriented Reidemeister moves. One may distinguish between forward and backward moves, obtaining $32$ different types of moves, which we call directed oriented Reidemeister moves. In this article we prove that the set of $…