Holonomy groups of metric connections converge in a monotonic way.
problem Monotonicity of holonomy groups under convergence of metric connections.
method Proving the monotonicity of holonomy groups for sequences of metric connections converging in C0. result The holonomy group of the limit connection is contained in the holonomy group of the initial connections.
Non-affine aggregation rules cannot preserve monotonicity in convex learning.
problem Designing non-affine aggregation rules that maintain monotonicity in convex learning.
method Proving that monotonicity of aggregated gradients is preserved only if the aggregation rule is positively affine.
result Non-affine aggregation prevents steady convergence and substantially degrades algorithmic stability.
Improved analysis of extragradient methods for structured VIPs.
problem Efficiently solving large-scale VIPs with weaker conditions.
method Single-call stochastic extragradient methods with expected residual condition.
result Convergence guarantees for quasi-strongly monotone and weak Minty VIPs.
GD monotonically decreases GFS sharpness in neural networks and scalar models.
problem Oscillatory behavior of loss in GD training.
method Analysis of GFS sharpness and empirical validation.
result GFS sharpness decreases monotonically during GD training.
This paper analyzes and improves monotonic accelerated algorithms like M-NAG and M-FISTA.
problem Establishing linear convergence of M-NAG and M-FISTA under strong convexity.
method Lyapunov analysis and modified Lyapunov functions.
result Linear convergence of M-NAG and M-FISTA is guaranteed without full NAG iterates.
In this paper, we consider first-order convergence theory and algorithms for solving a class of non-convex non-concave min-max saddle-point problems, whose objective function is weakly convex in the variables of minimization and weakly concave in the variables of maximization. It has many important applications in mach…
Researchers propose a non-monotone quantum natural gradient for quantum systems.
problem Applying natural gradient methods to quantum systems without monotonicity.
method Introducing a non-monotone quantum natural gradient (QNG) and demonstrating its superiority over conventional QNG.
result Non-monotone QNG outperforms conventional QNG in terms of convergence speed.
Monotone adversarial corruptions degrade optimal learning algorithms.
problem Optimal learning algorithms' reliance on exchangeability and independence is challenged.
method Introduces a monotone adversarial corruption model where an adversary adds monotone corruptions to a clean dataset.
result Optimal learning algorithms achieve suboptimal expected error on new test points.
Paper establishes NE existence and efficient algorithms for weakly monotone GMFGs.
problem Existence and efficient learning of Nash Equilibrium in λ-regularized GMFGs. method Establishes existence of NE for any λ-regularized GMFGs. Proposes efficient algorithms for weakly monotone GMFGs. result Efficient algorithms for weakly monotone GMFGs with provable convergence.
BCD algorithm finds global minima in neural networks.
problem Training deep neural networks to find global minima.
method Block coordinate descent with skip connections and non-negative projection.
result Proves convergence to global minima for strictly monotonic and ReLU activations.
New model outperforms Neural ODEs while being more efficient.
problem Stable convergence and existence guarantees for implicit-depth models.
method Developed Monotone Operator Equilibrium Network (monDEQ) based on monotone operator theory.
result MonDEQ models outperform Neural ODEs and are more computationally efficient.
Improved algorithms for convex-concave min-max optimization and monotone variational inequalities.
problem Efficiently solving constrained convex-concave min-max problems and monotone variational inequalities.
method Higher-order methods achieving iteration complexities of O(1/T^{ rac{p+1}{2}}) for p-th order derivatives.
result Achieved improved convergence rates for min-max and monotone variational inequalities.
Improved convergence for VIPs with SEG-RR, a variant of SEG with random reshuffling.
problem Solving variational inequality problems (VIPs) in machine learning.
method Stochastic Extragradient with Random Reshuffling (SEG-RR).
result SEG-RR achieves faster convergence rates than with-replacement variants for certain VIP classes.
We consider the learning algorithms under general source condition with the polynomial decay of the eigenvalues of the integral operator in vector-valued function setting. We discuss the upper convergence rates of Tikhonov regularizer under general source condition corresponding to increasing monotone index function. T…
We prove three new monotonicity formulas for manifolds with a lower Ricci curvature bound and show that they are connected to rate of convergence to tangent cones. In fact, we show that the derivative of each of these three monotone quantities is bounded from below in terms of the Gromov-Hausdorff distance to the neare…
Study shows convergence of Lagrangian submanifolds under certain metrics.
problem Understanding convergence of Lagrangian submanifolds under specific metrics.
method Proves convergence to an embedded Lagrangian submanifold using a monotonicity lemma applied on a carefully-chosen metric ball.
result Convergence to an embedded Lagrangian submanifold implies convergence in the Hausdorff metric for a class of metrics.
Proves flows of two-convex Lagrangians are regular, global, and converge.
problem Proves regularity, global existence, and convergence of Lagrangian mean curvature flows in the two-convex case.
method Uses a newly discovered monotone quantity to control two-convexity.
result Proves results for the mean curvature flow of area-decreasing Lagrangian submanifolds.
Survey on extragradient methods for solving nonlinear equations and inclusions.
problem Approximating solutions of nonlinear equations and inclusions.
method Unified convergence analysis of extragradient and its variants.
result Sublinear convergence rates for different classes of algorithms.
A new stochastic primal--dual algorithm for solving a composite optimization problem is proposed. It is assumed that all the functions/operators that enter the optimization problem are given as statistical expectations. These expectations are unknown but revealed across time through i.i.d. realizations. The proposed al…
Paper proposes a quasi-Newton method for nonlinear equations with global convergence guarantees.
problem Solving smooth and monotone nonlinear equations efficiently and globally.
method Hybrid proximal extragradient framework combined with online learning for Jacobian approximation.
result First global convergence results showing quasi-Newton method's advantage over extragradient method.
Unified view of monotonicity formulas for inverse mean curvature flow and p-capacitary potentials.
problem Understanding monotonicity formulas for various geometric flows and potentials.
method Refined analysis of p-capacitary potentials and their level sets. result Strong convergence of p-capacitary potentials to inverse mean curvature flow and curvature varifolds. Based on a study of the coupling by reflection of diffusion processes, a new monotonicity in time of a time-dependent transportation cost between heat distribution is shown under Bakry-Emery's curvature-dimension condition on a Riemannian manifold. The cost function comes from the total variation between heat distribut…
New algorithms solve monotone inclusions and convex-concave minimax problems.
problem Solving maximally monotone equations and inclusions.
method Developed new accelerated algorithms based on Halpern-type fixed-point iteration and Popov's past extra-gradient method.
result Achieved O(1/k) convergence rates for various problems. The paper examines the unexpected losses and risk ratios for co-monotonic alternatives in large portfolios.
problem Understanding the unexpected losses and risk ratios for large portfolios with co-monotonic alternatives.
method Analyzes the asymptotic behavior of unexpected losses and risk ratios for co-monotonic alternatives using monotone cash-additive risk measures and Choquet insurance premia.
result Unexpected losses of large weighted portfolios are of order o(nλn), where λn is the average weight. New method for optimizing risk in financial models using Fourier transforms.
problem Optimizing risk in financial models with multi-period mean-CVaR.
method Strictly monotone 2D integration scheme via Fourier-trained transition kernels.
result Established robust and accurate optimization method for financial models.
The paper proves smoothness of transition layers in the Allen-Cahn equation.
problem Proving uniform C2,α regularity for transition layers. method Utilizes Allen-Cahn monotonicity formula, Lipschitz approximation, and blowups.
result Shows uniform C2,α regularity for transition layers converging to smooth mean curvature flows. GradaGrad adapts learning rate non-monotonically, overcoming AdaGrad's step size decrease.
problem Fixed learning rate in AdaGrad leads to step size decrease over time.
method Introduces GradaGrad, which grows or shrinks the learning rate based on a different accumulation in the denominator.
result GradaGrad achieves similar convergence rates as AdaGrad and demonstrates non-monotone adaptation.
Equivalence of convex optimization, saddle-point problems, and variational inequalities is a well-established concept. The variational inequality (VI) is a static problem which is studied under dynamical settings using a framework called the projected dynamical system, whose stationary points coincide with the static s…
A one-parameter family of coupled flows depending on a parameter κ>0 is introduced which reduces when κ=1 to the coupled flow of a metric ω with a (1,1)-form α due recently to Y. Li, Y. Yuan, and Y. Zhang. It is shown in particular that, for κ=1, estimates for derivatives of all orders would follow from…
Develops multifactor approximations for SVEs with completely monotone kernels.
problem Approximating SVEs with kernels of completely monotone type.
method Multifactor approximation, Euler discretization, L2-estimation, convergence analysis. result New multifactor Euler scheme reduces computational cost and outperforms SVEs for option pricing.
Developed a monotone numerical method for MV portfolio optimization under jump-diffusion models.
problem Efficiently optimizing portfolios with jump-diffusion dynamics and investment constraints.
method Strictly monotone numerical integration method using Fourier transforms and composite quadrature rules.
result Proven to be ℓ∞-stable and pointwise consistent, converging to the MV optimization solution. Study tackles nonlinear factor models with unknown monotone links from incomplete and noisy data.
problem Learning nonlinear factor models with unknown monotone links from incomplete and noisy data.
method Formulated as joint recovery of low-rank factors, loadings, and nonlinear link function; proposed BCD algorithm with regularization.
result Established convergence guarantees and sublinear regret bounds for link-function updates.
Monotonic Linear Interpolation property in neural networks persists despite non-convexity.
problem Understanding the geometric properties of neural network loss landscapes.
method Tools from differential geometry to analyze the monotonicity of neural network weights.
result Sufficient conditions for the Monotonic Linear Interpolation property under mean squared error.
New ODE models show saddle-point optimization methods converge differently, with last-iterate convergence for OGDA.
problem Analyzing convergence properties of saddle-point optimization methods.
method High-Resolution Differential Equations (HRDEs) to design differential equation models for saddle-point optimization methods.
result HRDEs reveal last-iterate convergence for Optimistic Gradient Descent Ascent (OGDA) in bilinear games.
In this article, we introduce a new type of mean curvature flow for bounded star-shaped domains in space forms and prove its longtime existence, exponential convergence without any curvature assumption. Along this flow, the enclosed volume is a constant and the surface area evolves monotonically. Moreover, for a bounde…
New algorithms solve DR-submodular maximization with faster convergence.
problem Maximizing monotone DR-submodular functions under convex constraints.
method Introduced strongly DR-submodular functions and proposed SDRFW and PGA algorithms.
result SDRFW achieves optimal approximation ratio after fewer iterations.
ThiopheneIV is a new solver for implied volatility with proven monotonicity.
problem Efficiently solving implied volatility in financial models.
method Monotone core with Euler-Chebyshev and Halley steps, exact arithmetic proof, practical boundary handling.
result ThiopheneIV agrees closely with multiprecision Black reference prices at low latency.
In this article, we study the convergence of Mirror Descent (MD) and Optimistic Mirror Descent (OMD) for saddle point problems satisfying the notion of coherence as proposed in Mertikopoulos et al. We prove convergence of OMD with exact gradients for coherent saddle point problems, and show that monotone convergence on…
The extragradient method fails for hypomonotone variational inequalities.
problem The convergence of the extragradient method for hypomonotone variational inequalities.
method Application of the extragradient method to hypomonotone linear operators.
result The extragradient method diverges for hypomonotone variational inequalities.
New deficit functions link elliptic and parabolic inequalities, proving log Sobolev.
problem Proving log Sobolev inequality using deficit functions.
method Introducing two deficit functions, one elliptic and one parabolic, and showing their pointwise convergence and equations.
result Elliptic deficit converges to parabolic deficit, leading to an elliptic proof of log Sobolev inequality.
We consider differentiable games where the goal is to find a Nash equilibrium. The machine learning community has recently started using variants of the gradient method (GD). Prime examples are extragradient (EG), the optimistic gradient method (OG) and consensus optimization (CO), which enjoy linear convergence in cas…
Unified framework for sparse logistic regression with nonconvex regularization.
problem Sparse logistic regression with nonconvex regularization.
method Unified framework, line search criteria for nonconvex terms.
result Effective classification and feature selection at lower computational cost.
Stochastic Gradient Descent (SGD) is a central tool in machine learning. We prove that SGD converges to zero loss, even with a fixed (non-vanishing) learning rate - in the special case of homogeneous linear classifiers with smooth monotone loss functions, optimized on linearly separable data. Previous works assumed eit…
Alternative neural network training using monotone variational inequality.
problem Training neural networks efficiently and with guarantees.
method Using monotone variational inequality to solve non-convex problems efficiently.
result Our approach leads to fast convergence and competitive performance compared to traditional methods.
A new family of momentum coefficients improves the convergence rate of accelerated algorithms.
problem Improving the convergence rate of accelerated gradient methods for strongly convex functions.
method Introducing a family of controllable momentum coefficients for forward-backward accelerated methods.
result Established a controllable $O\left(1/k^{2α}
ight)$ convergence rate for the NAG-α method. The monotonic linear interpolation in deep networks often leads to plateaus, revealing biases in optimization.
problem Plateaus in the optimization landscape of deep networks during monotonic linear interpolation.
method Investigated monotonic linear interpolation on deep neural networks, focusing on biases in weights and biases.
result Interpolating weights and biases differently can lead to significant differences in loss and accuracy, revealing biases in optimization.
We study the formation of singularities for the mean curvature flow of monotone Lagrangians in $\C^n$. More precisely, we show that if singularities happen before a critical time then the tangent flow can be decomposed into a finite union of area-minimizing Lagrangian cones (Slag cones). When n=2, we can improve this…
It is common to encounter large-scale monotone inclusion problems where the objective has a finite sum structure. We develop a general framework for variance-reduced forward-backward splitting algorithms for this problem. This framework includes a number of existing deterministic and variance-reduced algorithms for fun…