We deduce from the work of Chen, that the restriction morphism from closed free iterated integrals to closed iterated integrals on loops is onto. We use this to show that the module of higher order invariants of smooth functions is generated by free closed iterated integrals.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “iteration order”
Value iteration is a fixed point iteration technique utilized to obtain the optimal value function and policy in a discounted reward Markov Decision Process (MDP). Here, a contraction operator is constructed and applied repeatedly to arrive at the optimal solution. Value iteration is a first order method and therefore …
OptEx accelerates first-order optimization with parallelized iterations.
problem Inefficiencies in first-order optimization algorithms for complex tasks.
method Approximately parallelized iterations using kernelized gradient estimation.
result OptEx achieves substantial efficiency improvements with an effective acceleration rate of .
We construct bi-invariant total orderings of residually torsion-free nilpotent groups by using Chen's iterated integrals. This construction can be seen as a generalization of the Magnus ordering of the free groups, and equivalent to the classical construction which uses an iteration of central extensions. Our geometric…
We observe that the iterated tangent group of a Lie group may be realized as a double cross product of the 2nd order tangent group, with the Lie algebra of the base Lie group. Based on this observation, we derive the 2nd order Euler-Lagrange equations on the 2nd order tangent group from the 1st order Euler-Lagrange equ…
Paper develops a TR-SSQP method for noisy optimization with heavy-tailed noise.
problem Optimization problems with stochastic objectives and heavy-tailed noise.
method Trust-Region Stochastic Sequential Quadratic Programming (TR-SSQP) method.
result Achieves high-probability first-order and second-order stationarity bounds for heavy-tailed noise.
Many problems in machine learning and game theory can be formulated as saddle-point problems, for which various first-order methods have been developed and proven efficient in practice. Under the general convex-concave assumption, most first-order methods only guarantee an ergodic convergence rate, that is, the uniform…
Higher-order methods for convex-concave min-max optimization and monotone variational inequalitiesmath.OC
Improved algorithms for convex-concave min-max optimization and monotone variational inequalities.
problem Efficiently solving constrained convex-concave min-max problems and monotone variational inequalities.
method Higher-order methods achieving iteration complexities of O(1/T^{ rac{p+1}{2}}) for p-th order derivatives.
result Achieved improved convergence rates for min-max and monotone variational inequalities.
This work's purpose is to understand the dynamics of limit order books in order-driven markets. We try to illustrate a dynamical trading mechanism attached to the microstructure of limit order markets. We capture the iterative nature of trading processes, which is critical in the dynamics of bid-ask pairs and the switc…
We propose a mixed integer programming (MIP) model and iterative algorithms based on topological orders to solve optimization problems with acyclic constraints on a directed graph. The proposed MIP model has a significantly lower number of constraints compared to popular MIP models based on cycle elimination constraint…
Neural networks learn higher-order derivatives for physics problems.
problem Lack of higher-order derivatives in neural networks for theoretical physics.
method Graph-theoretical approach to assign diagrams to partial derivatives, iterative NN perturbation theory.
result NNs can learn higher-order derivatives, improving machine-learned approximations.
New method accelerates steepest descent for convex optimization.
problem Achieving acceleration for general smooth functions.
method Primal-dual iterate sequences with differing norms, implicitly determined interpolation parameter.
result Improves iteration complexity to for norm smooth problems.
Derivative-free Alternating Projection Algorithms for General Nonconvex-Concave Minimax Problemsmath.OC
New algorithms solve complex minimax problems without needing derivatives.
problem Solving nonconvex-concave minimax problems efficiently.
method Zeroth-order alternating and proximal gradient algorithms.
result Iteration complexity and function value estimation bounds established.
Two new algorithms solve nonconvex-strongly concave problems efficiently.
problem Solving nonconvex-strongly concave minimax problems.
method Proposed MINIMAX-TR and MINIMAX-TRACE algorithms.
result Find -second order stationary points within iterations.
First-order stochastic methods are the state-of-the-art in large-scale machine learning optimization owing to efficient per-iteration complexity. Second-order methods, while able to provide faster convergence, have been much less explored due to the high cost of computing the second-order information. In this paper we …
New algorithms solve complex minimax problems efficiently.
problem Nonconvex-strongly concave minimax problems in machine learning.
method Gradient norm regularized trust-region (GRTR) and Levenberg-Marquardt (LMNegCur) algorithms.
result Proved iteration complexities matching best known results.
In this paper we consider successive iterations of the first-order differential operations in space
Basic Inequalities for First-Order Optimization with Applications to Statistical Risk Analysismath.ST
New inequalities help optimize first-order algorithms for statistical risk analysis.
problem Optimizing first-order iterative algorithms for statistical risk analysis.
method Introducing basic inequalities that connect implicit and explicit regularization.
result The basic inequalities translate the number of iterations into an effective regularization coefficient.
Unified bounds for iterative algorithms with Gaussian data matrices.
problem Establishing non-asymptotic bounds for iterative algorithms with Gaussian data.
method Explicit coupling between iterates and Gaussian process with deterministic covariance.
result Tight, dimension-free bounds for generalized first-order methods.
SGD's performance improves with critical batch size, minimizing SFO complexity.
problem Optimizing SGD's performance with batch size and learning rate.
method Analysis of SGD using constant and decaying learning rates, focusing on batch size effects.
result SGD with critical batch size minimizes SFO complexity.
Improved solver maintains positivity and accuracy across all time steps.
problem Linear second-order schemes for Fokker-Planck equation cannot preserve positivity.
method Flux-Corrected Diagonal Frog (FCDF) framework using nonlinear extension and iterative limiter.
result FCDF schemes are unconditionally positive across all time steps and maintain second-order accuracy.
Improved Optimization of Finite Sums with Minibatch Stochastic Variance Reduced Proximal Iterationsmath.OC
We present novel minibatch stochastic optimization methods for empirical risk minimization problems, the methods efficiently leverage variance reduced first-order and sub-sampled higher-order information to accelerate the convergence speed. For quadratic objectives, we prove improved iteration complexity over state-of-…
A new iterative K-FAC algorithm reduces training time and memory usage.
problem Training deep learning models efficiently.
method Uses conjugate gradient to approximate Fisher information matrix without generating the matrix or factors.
result Time and memory complexity of iterative CG-FAC is less than standard K-FAC.
New method shows how order of gradient updates impacts stability and convergence in deep learning.
problem Training deep learning models can be unstable and computationally expensive.
method Theoretical analysis and experiments with backward-SGD.
result The order of gradient updates affects stability and convergence, leading to improved performance.
Paper analyzes iterative learning for concept classes and learns half-spaces.
problem Learning concept classes efficiently with iterative learners.
method Analyzes various settings of iterative learning and provides a constructive algorithm for half-spaces.
result Constructive iterative algorithm for learning half-spaces from informant.
FLOP algorithm speeds up causal structure learning for linear models.
problem Efficiently learning causal structures from discrete data.
method FLOP algorithm combines fast parent selection and iterative score updates.
result FLOP finds highly accurate causal structures with near-perfect recovery.
Improved matching for multiple objects using a novel reweighting method.
problem Current multi-object matching methods have limitations and are not robust.
method Proposes a novel iterative reweighting strategy using the graph connection Laplacian.
result Demonstrates superior performance over state-of-the-art methods.
First order discretizations of Langevin diffusion can achieve better generalization error with additional smoothness assumptions.
problem Analyzing generalization error for first order discretizations of Langevin diffusion.
method Providing a sufficient smoothness condition to show that first order methods can achieve arbitrarily runtime complexity for a given expected generalization error.
result First order methods can achieve arbitrarily runtime complexity with additional smoothness assumptions.
Optimal algorithms for Riemannian optimization with reduced complexity.
problem Stochastic optimization on Riemannian manifolds with limited data.
method Zeroth-order Riemannian Averaging Stochastic Approximation algorithms using Riemannian moving-average estimators and novel geometric conditions.
result Achieves optimal sample complexities for generating approximate first-order stationary solutions.
AdaSub optimizes with second-order info in low-dims subspace.
problem Efficiently use second-order optimization methods with low computational cost.
method Adaptive subspace selection for second-order optimization.
result AdaSub outperforms other stochastic optimizers in time and iterations.
Simple gradient descent algorithm escapes saddle points efficiently.
problem Escaping saddle points in nonconvex optimization.
method Gradient-based algorithm with polynomial iterations.
result Outputs ε-approximate second-order stationary points efficiently.
A new hybrid-ordered SGD method reduces communication and complexity for non-convex optimization.
problem Balancing communication, computational complexity, and convergence rate in distributed non-convex optimization.
method Hybrid-ordered distributed SGD with pre-shared scalers and periodic vector communication.
result Order-wise faster convergence compared to existing methods.
New algorithm improves understanding of decentralized SBO transient iteration complexity.
problem Limited understanding of how network topology, data heterogeneity, and nested structures affect SBO.
method D-SOBA framework with two variants: D-SOBA-SO and D-SOBA-FO, providing non-asymptotic convergence analysis and transient iteration complexity.
result First theoretical understanding of how network topology, data heterogeneity, and nested structures influence decentralized SBO.
LocalNewton reduces communication in distributed learning.
problem Communication bottleneck in distributed optimization.
method LocalNewton is a distributed second-order algorithm with local averaging, updating models locally and communicating once every few iterations.
result LocalNewton reduces communication rounds and end-to-end running time compared to state-of-the-art algorithms.
New method improves DAG learning by using large coefficients for higher-order terms.
problem Recovering DAG structures from observational data is challenging due to combinatorial optimization.
method Proposes truncated matrix power iteration to approximate DAG constraints efficiently.
result Empirically outperforms previous methods by a factor of 3 or more in structural Hamming distance.
We propose and analyze a new type of stochastic first order method: gradient descent with compressed iterates (GDCI). GDCI in each iteration first compresses the current iterate using a lossy randomized compression technique, and subsequently takes a gradient step. This method is a distillation of a key ingredient in t…
COMRADE is a communication-efficient, Byzantine-resilient second-order optimization algorithm.
problem Byzantine failures in distributed optimization.
method COMRADE is a communication-efficient, second-order optimization algorithm that uses a simple norm-based thresholding rule to filter out Byzantine workers.
result COMRADE achieves linear-quadratic convergence and is robust against Byzantine workers.
This paper accelerates TV regularization algorithms by unrolling proximal gradient descent.
problem Solving Total Variation (TV) regularized problems with iterative algorithms.
method Unrolling proximal gradient descent solvers to learn their parameters.
result Two approaches to compute derivatives through proximal operators improve performance.
Paper analyzes LSA algorithm bias and error bounds with RR extrapolation.
problem Analyzing bias and high-order error bounds of LSA with Markovian noise.
method Polyak-Ruppert averaging, linearization, Richardson-Romberg extrapolation.
result RR extrapolation effectively cancels the leading bias term.
Zeroth-order optimization methods lack inherent privacy guarantees.
problem Ensuring differential privacy in zeroth-order optimization methods.
method Analyzing ZO-GD with and without random initialization for convex and strongly convex objectives.
result ZO-GD is not differentially private for strongly convex objectives and can have superlinear privacy loss.
We introduce a novel class of rotation invariants of two dimensional curves based on iterated integrals. The invariants we present are in some sense complete and we describe an algorithm to calculate them, giving explicit computations up to order six. We present an application to online (stroke-trajectory based) charac…
An important result from psycholinguistics (Griffiths & Kalish, 2005) states that no language can be learned iteratively by rational agents in a self-sustaining manner. We show how to modify the learning process slightly in order to achieve self-sustainability. Our work is in two parts. First, we characterize iterated …
In the present paper we study interval identification systems of order three. We prove that the Rauzy induction preserves symmetry: for any symmetric interval identification system of order three after finitely many iterations of the Rauzy induction we always obtain a symmetric system. We also provide an example of sym…
Stochastic Second-order Methods for Non-convex Optimization with Inexact Hessian and Gradientmath.OC
Trust region and cubic regularization methods have demonstrated good performance in small scale non-convex optimization, showing the ability to escape from saddle points. Each iteration of these methods involves computation of gradient, Hessian and function value in order to obtain the search direction and adjust the r…
We propose and analyze a new parallel coordinate descent method---`NSync---in which at each iteration a random subset of coordinates is updated, in parallel, allowing for the subsets to be chosen non-uniformly. We derive convergence rates under a strong convexity assumption, and comment on how to assign probabilities t…
Two algorithms solve nonconvex minimax problems with linear constraints, achieving complexity guarantees.
problem Nonconvex minimax problems with coupled linear constraints.
method Zeroth-order primal-dual alternating projected gradient (ZO-PDAPG) and zeroth-order regularized momentum primal-dual projected gradient (ZO-RMPDPG) algorithms.
result Iteration complexity guarantees for solving nonconvex-(strongly) concave minimax problems with coupled linear constraints.
This paper proposes low-complexity algorithms for finding approximate second-order stationary points (SOSPs) of problems with smooth non-convex objective and linear constraints. While finding (approximate) SOSPs is computationally intractable, we first show that generic instances of the problem can be solved efficientl…
QMME balances cost and speed in convex optimization.
problem Slow convergence of first-order methods and high cost of second-order methods.
method Minimizing quadratic majorants with fixed curvature at each iteration.
result QMME framework achieves sequential convergence under standard assumptions.