A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New algorithm solves non-convex, non-differentiable min-max games.
We reformulate LIPs as min-max problems for easier solution.
L2R learns to denoise images without needing noise distribution knowledge.
Survey of advances in non-convex min-max optimization for applications.
In this paper, we consider first-order convergence theory and algorithms for solving a class of non-convex non-concave min-max saddle-point problems, whose objective function is weakly convex in the variables of minimization and weakly concave in the variables of maximization. It has many important applications in mach…
A new method solves bilevel optimization problems in competitive Markov games.
Study introduces statistical mechanics for min-max problems.
New algorithm solves min-max optimization problems in a decentralized manner.
New methods solve min-max problems on manifolds using Riemannian Hamiltonians.
Recent applications that arise in machine learning have surged significant interest in solving min-max saddle point games. This problem has been extensively studied in the convex-concave regime for which a global equilibrium solution can be computed efficiently. In this paper, we study the problem in the non-convex reg…
FeDualEx tackles saddle point optimization in federated learning with composite objectives.
New algorithms reduce variance in solving complex mathematical problems.
Riemannian algorithms converge at Euclidean rates for geodesically convex-concave problems.
Motivated by applications in Game Theory, Optimization, and Generative Adversarial Networks, recent work of Daskalakis et al \cite{DISZ17} and follow-up work of Liang and Stokes \cite{LiangS18} have established that a variant of the widely used Gradient Descent/Ascent procedure, called "Optimistic Gradient Descent/Asce…
PWGF escapes saddle points in nonconvex optimization.
New algorithm solves structured nonconvex-nonconcave min-max problems.
Recent successes of game-theoretic formulations in ML have caused a resurgence of research interest in differentiable games. Overwhelmingly, that research focuses on methods and upper bounds on their speed of convergence. In this work, we approach the question of fundamental iteration complexity by providing lower boun…
Recent focus on robustness to adversarial attacks for deep neural networks produced a large variety of algorithms for training robust models. Most of the effective algorithms involve solving the min-max optimization problem for training robust models (min step) under worst-case attacks (max step). However, they often s…
New saddle network architectures preserve convex-concave geometry in optimization problems.
The paper proposes a novel MKL approach for OCC using -norm constraints.
Adaptive momentum method solves non-convex min-max problems.
A generalized optimistic method for saddle point problems with improved complexity.
We consider the problem of finding local minimizers in non-convex and non-smooth optimization. Under the assumption of strict saddle points, positive results have been derived for first-order methods. We present the first known results for the non-smooth case, which requires different analysis and a different algorithm…
New methods reduce communication in distributed training for variational inequalities.
Paper tackles fast convergence for non-convex strongly-concave min-max problems.
New ODE models show saddle-point optimization methods converge differently, with last-iterate convergence for OGDA.
Paper tackles multi-block min-max optimization with applications in deep AUC maximization.
Although adversarial examples and model robustness have been extensively studied in the context of linear models and neural networks, research on this issue in tree-based models and how to make tree-based models robust against adversarial examples is still limited. In this paper, we show that tree based models are also…
The min-max problem, also known as the saddle point problem, is a class of optimization problems which minimizes and maximizes two subsets of variables simultaneously. This class of problems can be used to formulate a wide range of signal processing and communication (SPCOM) problems. Despite its popularity, most exist…
GenFlow optimizes faster, avoiding saddle points in fixed time.
New algorithm solves saddle point problems in Banach spaces.
In this paper, we focus on solving a class of constrained non-convex non-concave saddle point problems in a decentralized manner by a group of nodes in a network. Specifically, we assume that each node has access to a summand of a global objective function and nodes are allowed to exchange information only with their n…
In this paper we prove that, given a compact four dimensional smooth Riemannian manifold (M,g) with smooth boundary there exists a metric conformal to g with constant T-curvature, zero Q-curvature and zero mean curvature under generic and conformally invariant assumptions. The problem amounts to solving a fourth order …
We consider saddle point problems which objective functions are the average of strongly convex-concave individual components. Recently, researchers exploit variance reduction methods to solve such problems and achieve linear-convergence guarantees. However, these methods have a slow convergence when the condition n…
We study the iteration complexity of the optimistic gradient descent-ascent (OGDA) method and the extra-gradient (EG) method for finding a saddle point of a convex-concave unconstrained min-max problem. To do so, we first show that both OGDA and EG can be interpreted as approximate variants of the proximal point method…
Optimizes bond portfolios to avoid worst-case losses.
We provide larger step-size restrictions for which gradient descent based algorithms (almost surely) avoid strict saddle points. In particular, consider a twice differentiable (non-convex) objective function whose gradient has Lipschitz constant L and whose Hessian is well-behaved. We prove that the probability of init…
With the growth of adversarial attacks against machine learning models, several concerns have emerged about potential vulnerabilities in designing deep neural network-based intrusion detection systems (IDS). In this paper, we study the resilience of deep learning-based intrusion detection systems against adversarial at…
We propose a general theory for studying the \xl{landscape} of nonconvex \xl{optimization} with underlying symmetric structures \tz{for a class of machine learning problems (e.g., low-rank matrix factorization, phase retrieval, and deep linear neural networks)}. In specific, we characterize the locations of stationary …
AMI framework improves text generation by optimizing mutual information between source and target.
New method simplifies optimization landscapes by transforming saddle points.
In this paper, we study the problem of constrained robust (min-max) optimization ina black-box setting, where the desired optimizer cannot access the gradients of the objective function but may query its values. We present a principled optimization framework, integrating a zeroth-order (ZO) gradient estimator with an a…
Epoch gradient descent method (a.k.a. Epoch-GD) proposed by Hazan and Kale (2011) was deemed a breakthrough for stochastic strongly convex minimization, which achieves the optimal convergence rate of with iterative updates for the {\it objective gap}. However, its extension to solving stochastic min-max pr…
Equivalence of convex optimization, saddle-point problems, and variational inequalities is a well-established concept. The variational inequality (VI) is a static problem which is studied under dynamical settings using a framework called the projected dynamical system, whose stationary points coincide with the static s…
Lower bounds found for nonconvex-strongly-concave min-max optimization problems.
We consider discriminative dictionary learning in a distributed online setting, where a network of agents aims to learn a common set of dictionary elements of a feature space and model parameters while sequentially receiving observations. We formulate this problem as a distributed stochastic program with a non-convex o…
Improved convergence rates for saddle-point optimization algorithms.
Bayesian optimization for function-valued responses, addressing worst case deviations.