Saddle-point optimization problems are an important class of optimization problems with applications to game theory, multi-agent reinforcement learning and machine learning. A majority of the rich literature available for saddle-point optimization has focused on the offline setting. In this paper, we study nonstationar…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “saddle-point problem”
New method solves saddle-point problems faster than existing methods.
problem Large-scale saddle-point problems in optimization.
method Sequential subspace optimization with proximal regularization.
result Significantly better convergence compared to first-order methods.
FeDualEx tackles saddle point optimization in federated learning with composite objectives.
problem Saddle point optimization with constraints and non-smooth regularization in federated learning.
method Federated Dual Extrapolation (FeDualEx) algorithm for saddle point optimization and composite objectives.
result FeDualEx effectively solves saddle point optimization problems with composite objectives in federated learning.
Gradient-based optimization methods are the most popular choice for finding local optima for classical minimization and saddle point problems. Here, we highlight a systemic issue of gradient dynamics that arise for saddle point problems, namely the presence of undesired stable stationary points that are no local optima…
New algorithm solves saddle point problems in Banach spaces.
problem Solving saddle point problems in real reflexive Banach spaces.
method Stochastic Bregman Primal-Dual Splitting Algorithm with relative smoothness and strong convexity assumptions.
result Almost sure convergence to saddle points under various conditions.
A new method helps escape saddle points in non-convex optimization.
problem Escaping saddle points in non-convex optimization problems.
method CNC-SCSG method using a separate SGD step to help escape from strict saddle points.
result The method converges to a second-order stationary point with a rate of .
Equivalence of convex optimization, saddle-point problems, and variational inequalities is a well-established concept. The variational inequality (VI) is a static problem which is studied under dynamical settings using a framework called the projected dynamical system, whose stationary points coincide with the static s…
In this paper we study the smooth convex-concave saddle point problem. Specifically, we analyze the last iterate convergence properties of the Extragradient (EG) algorithm. It is well known that the ergodic (averaged) iterates of EG converge at a rate of (Nemirovski, 2004). In this paper, we show that the last…
In this paper we consider solving saddle point problems using two variants of Gradient Descent-Ascent algorithms, Extra-gradient (EG) and Optimistic Gradient Descent Ascent (OGDA) methods. We show that both of these algorithms admit a unified analysis as approximations of the classical proximal point method for solving…
Optimal privacy-preserving algorithm for solving saddle point problems.
problem Solving convex-concave stochastic saddle point problems under differential privacy constraints.
method Recursive regularization technique repurposed for saddle point problems, achieving strong gap rate of O(1/√n + √d/nε).
result Achieves nearly optimal strong gap rate of O(1/√n + √d/nε) with gradient complexity O(min{n^2ε^(1.5)/√d, n^(3/2)}).
On the modes of convergence of Stochastic Optimistic Mirror Descent (OMD) for saddle point problemsmath.OC
In this article, we study the convergence of Mirror Descent (MD) and Optimistic Mirror Descent (OMD) for saddle point problems satisfying the notion of coherence as proposed in Mertikopoulos et al. We prove convergence of OMD with exact gradients for coherent saddle point problems, and show that monotone convergence on…
We consider saddle point problems which objective functions are the average of strongly convex-concave individual components. Recently, researchers exploit variance reduction methods to solve such problems and achieve linear-convergence guarantees. However, these methods have a slow convergence when the condition n…
A distributed optimization method solves saddle point problems with strong concavity and convexity.
problem Solving saddle point problems with distributed and heterogeneous data.
method GT-GDA, a distributed first-order method using gradient tracking and consensus over coupling matrices.
result GT-GDA converges linearly to the unique saddle point solution under specific conditions.
We reformulate LIPs as min-max problems for easier solution.
problem Recovering signals from few linear measurements.
method Proposed a min-max reformulation of LIPs.
result Saddle points characterize solutions to LIPs.
We consider convex-concave saddle point problems with a separable structure and non-strongly convex functions. We propose an efficient stochastic block coordinate descent method using adaptive primal-dual updates, which enables flexible parallel optimization for large-scale problems. Our method shares the efficiency an…
This paper extends Newton's method to distributed learning, avoiding saddle points and handling Byzantine workers.
problem Avoiding saddle points in distributed non-convex optimization, especially in the presence of Byzantine workers.
method Extends cubic-regularized Newton method to distributed framework, addressing communication bottlenecks and Byzantine attacks.
result The method achieves improved iteration complexity compared to first-order methods, with a 25% improvement in experiments.
We extend the Frank-Wolfe (FW) optimization algorithm to solve constrained smooth convex-concave saddle point (SP) problems. Remarkably, the method only requires access to linear minimization oracles. Leveraging recent advances in FW optimization, we provide the first proof of convergence of a FW-type saddle point solv…
We consider the problem of computing optimal policies in average-reward Markov decision processes. This classical problem can be formulated as a linear program directly amenable to saddle-point optimization methods, albeit with a number of variables that is linear in the number of states. To address this issue, recent …
Houdini finds high-dimensional saddle points under few constraints.
problem Escaping from saddle points in high-dimensional spaces with constraints.
method Gradient descent methods under logarithmic inequality constraints.
result Polynomial time algorithms for escaping saddle points under constraints.
GenFlow optimizes faster, avoiding saddle points in fixed time.
problem Designing efficient optimization algorithms for convex and non-convex functions.
method Introduces GenFlow and momentum variants with fixed-time convergence guarantees.
result GenFlow and momentum variants converge to optimal solutions in fixed time for PL functions and evade saddle points uniformly.
We consider a generic convex-concave saddle point problem with separable structure, a form that covers a wide-ranged machine learning applications. Under this problem structure, we follow the framework of primal-dual updates for saddle point problems, and incorporate stochastic block coordinate descent with adaptive st…
Nonconvex optimization problems such as the ones in training deep neural networks suffer from a phenomenon called saddle point proliferation. This means that there are a vast number of high error saddle points present in the loss function. Second order methods have been tremendously successful and widely adopted in the…
New method stabilizes saddle-point optimization with unbounded gradients.
problem Stochastic saddle-point optimization faces instability due to large gradients.
method Proposes a regularization technique to stabilize iterates.
result Yields meaningful performance guarantees even with unbounded gradients.
Perturbed Proximal Descent to Escape Saddle Points for Non-convex and Non-smooth Objective Functionscs.LG
We consider the problem of finding local minimizers in non-convex and non-smooth optimization. Under the assumption of strict saddle points, positive results have been derived for first-order methods. We present the first known results for the non-smooth case, which requires different analysis and a different algorithm…
A generalized optimistic method for saddle point problems with improved complexity.
problem Solving convex-concave saddle point problems efficiently.
method Proposes a generalized optimistic method that includes the optimistic gradient method as a special case, handling constrained saddle point problems with composite objective functions and arbitrary norms.
result Best-known global iteration complexity bounds for first-, second-, and higher-order methods.
We study the online saddle point problem, an online learning problem where at each iteration a pair of actions need to be chosen without knowledge of the current and future (convex-concave) payoff functions. The objective is to minimize the gap between the cumulative payoffs and the saddle point value of the aggregate …
New method simplifies optimization landscapes by transforming saddle points.
problem Saddle points hinder non-convex optimization in machine learning.
method Variable elimination algorithms, like VarPro, are compared to reveal geometric insights.
result Variable elimination reshapes critical point structure, creating local maxima from saddle points.
We solve a complex optimization problem for Wasserstein barycenters using stochastic methods.
problem Optimizing the average of multiple probability distributions in a streaming data setting.
method We reformulate the problem as a convex-concave saddle-point problem and propose a stochastic optimization algorithm.
result Our algorithm has better complexity than existing methods for arbitrary distributions.
Auto-conditioned primal-dual hybrid gradient method and alternating direction method of multipliersmath.OC
New methods solve saddle point problems without line search.
problem Solving saddle point problems efficiently and adaptively.
method Auto-conditioned primal-dual hybrid gradient (AC-PDHG) and auto-conditioned ADMM (AC-ADMM) methods.
result Methods achieve optimal complexity and convergence guarantees.
New method for causal effect estimation with hidden confounders.
problem Estimating causal effects in the presence of hidden confounders.
method Singular value decomposition of a conditional expectation operator followed by saddle-point optimization.
result Our method outperforms existing methods on common benchmarks.
Optimizes bond portfolios to avoid worst-case losses.
problem Finding the worst-case value of a bond portfolio over a range of yield curves and spreads.
method Solves a convex-concave saddle point optimization problem to find the worst-case value and construct a robust portfolio.
result Constructs a bond portfolio that includes the worst-case value, ensuring robustness against market uncertainties.
A commonly used heuristic in non-convex optimization is Normalized Gradient Descent (NGD) - a variant of gradient descent in which only the direction of the gradient is taken into account and its magnitude ignored. We analyze this heuristic and show that with carefully chosen parameters and noise injection, this method…
A central challenge to many fields of science and engineering involves minimizing non-convex error functions over continuous, high dimensional spaces. Gradient descent or quasi-Newton methods are almost ubiquitously used to perform such minimizations, and it is often thought that a main source of difficulty for these l…
Although adversarial examples and model robustness have been extensively studied in the context of linear models and neural networks, research on this issue in tree-based models and how to make tree-based models robust against adversarial examples is still limited. In this paper, we show that tree based models are also…
We consider the convex-concave saddle point problem where is smooth and convex and is smooth and strongly convex. We prove that if the coupling matrix has full column rank, the vanilla primal-dual gradient method can achieve linear convergence even if is not stron…
Gradient descent can take exponentially long to escape saddle points in 2D.
problem Worst-case inefficiency of gradient descent in non-convex optimization.
method Analysis of gradient descent's performance on 2D functions.
result Gradient descent can take exponentially long to escape saddle points.
New methods help escape strict saddle points in nonsmooth optimization.
problem Escaping strict saddle points in nonsmooth optimization.
method An inexact stochastically perturbed gradient method applied to the Moreau envelope.
result A variety of algorithms for nonsmooth optimization can efficiently escape strict saddle points of the Moreau envelope.
Survey of methods for solving smooth stochastic variational inequalities.
problem Solving smooth (strongly) monotone stochastic variational inequalities.
method Deterministic foundation, general stochastic formulation, finite sum setup, recent advances.
result Review of various methods for solving smooth stochastic variational inequalities.
This paper concerns the recursive utility maximization problem under partial information. We first transform our problem under partial information into the one under full information. When the generator of the recursive utility is concave, we adopt the variational formulation of the recursive utility which leads to a s…
A new algorithm trains deep neural networks by adding neurons greedily.
problem Training deep neural networks efficiently and effectively.
method Neuron Pursuit (NP) algorithm, which alternates between neuron addition and loss minimization.
result The algorithm can train deep neural networks efficiently and effectively.
Nonconvex optimization algorithms with random initialization have attracted increasing attention recently. It has been showed that many first-order methods always avoid saddle points with random starting points. In this paper, we answer a question: can the nonconvex heavy-ball algorithms with random initialization avoi…
Efficient algorithm converges to Nash equilibrium in bilinear problems with bandit feedback.
problem Learning dynamics in bilinear saddle-point problems with bandit feedback.
method Uncoupled learning algorithm combining experimental design and FTRL with a tailored regularizer.
result Last-iterate convergence rate of in high probability.
Study on scalar curvature minimizability loss and saddle point solutions.
problem Loss of minimizability in prescribing scalar curvature.
method Variational approach, focusing on saddle point solutions.
result Existence of saddle point solutions despite minimizability loss.
Last-Iterate Convergence of Saddle-Point Optimizers via High-Resolution Differential Equationsmath.OC
New ODE models show saddle-point optimization methods converge differently, with last-iterate convergence for OGDA.
problem Analyzing convergence properties of saddle-point optimization methods.
method High-Resolution Differential Equations (HRDEs) to design differential equation models for saddle-point optimization methods.
result HRDEs reveal last-iterate convergence for Optimistic Gradient Descent Ascent (OGDA) in bilinear games.
Two new algorithms solve privacy-constrained SVI and SSP problems.
problem Privacy-constrained stochastic variational inequality and saddle-point problems.
method Proposed Noisy Stochastic Extragradient (NSEG) and Noisy Inexact Stochastic Proximal Point (NISPP) algorithms.
result Optimal risk bounds for weak gap function with sampling with replacement.
Extends saddle-point method for large-time volatility smiles.
problem Analyzing large-time volatility smiles in financial models.
method Saddle-point approach to derive large-time model-implied volatility smiles.
result Provides theoretical foundation and wide class of arbitrage-free parametrizations.
SGD converges almost surely in non-convex problems, avoiding saddle points and accelerating convergence.
problem Understanding convergence of SGD in non-convex optimization problems.
method Analysis of SGD trajectories, focusing on boundedness, convergence to strict saddle points, and rate of convergence.
result SGD converges almost surely to a minimizer in non-convex problems, avoiding strict saddle points.
In this paper, we focus on solving a class of constrained non-convex non-concave saddle point problems in a decentralized manner by a group of nodes in a network. Specifically, we assume that each node has access to a summand of a global objective function and nodes are allowed to exchange information only with their n…