A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New algorithms tackle machine learning problems using manifold proximal point methods.
Stochastic proximal point algorithm with momentum converges faster and is more stable than standard methods.
Introduces PPMM algorithm for nonconvex robust regression problems.
Develops a new SPP algorithm with variance reduction for weakly convex optimization.
We consider optimization problems over the Stiefel manifold whose objective function is the summation of a smooth function and a nonsmooth function. Existing methods for solving this kind of problems can be classified into three classes. Algorithms in the first class rely on information of the subgradients of the objec…
Two new methods solve nonsmooth optimization on Riemannian Stiefel manifold.
A new algorithm solves signed Fréchet regression on manifolds with bounded curvature.
Improves RL algorithms with two techniques.
In this paper we consider solving saddle point problems using two variants of Gradient Descent-Ascent algorithms, Extra-gradient (EG) and Optimistic Gradient Descent Ascent (OGDA) methods. We show that both of these algorithms admit a unified analysis as approximations of the classical proximal point method for solving…
Proximal algorithms applied to current deformation into cycles.
In this paper we develop proximal methods for statistical learning. Proximal point algorithms are useful in statistics and machine learning for obtaining optimization solutions for composite functions. Our approach exploits closed-form solutions of proximal operators and envelope representations based on the Moreau, Fo…
A new optimization method, BPM, converges linearly in non-convex, non-smooth problems.
New findings show Bregman proximal algorithms can get stuck near non-stationary points.
Gradient boosting is a prediction method that iteratively combines weak learners to produce a complex and accurate model. From an optimization point of view, the learning procedure of gradient boosting mimics a gradient descent on a functional variable. This paper proposes to build upon the proximal point algorithm, wh…
Riemannian Proximal Sampler improves sampling on manifold data.
A new method reformulates Optimal Transport Conditional Flow Matching using proximal operators.
We consider saddle point problems which objective functions are the average of strongly convex-concave individual components. Recently, researchers exploit variance reduction methods to solve such problems and achieve linear-convergence guarantees. However, these methods have a slow convergence when the condition n…
Two new algorithms improve federated optimization under second-order similarity.
Improved sampling guarantees for weakly log-concave distributions.
We introduce a proximal subdifferential and develop a calculus for nonsmooth functions defined on any Riemannian manifold . We give several applications of this theory, concerning: 1) differentiability and geometrical properties of the distance function to a closed subset of ; 2) solvability and implicit func…
Paper proposes a new method for supervised manifold learning using random forest proximities.
The paper analyzes convergence properties of NGA and PAMe for -norm PCA.
A method for estimating the median of gradients in stochastic optimization.
As the most successful variant and improvement for Trust Region Policy Optimization (TRPO), proximal policy optimization (PPO) has been widely applied across various domains with several advantages: efficient data utilization, easy implementation, and good parallelism. In this paper, a first-order gradient reinforcemen…
Nonconvex and nonsmooth problems have recently attracted considerable attention in machine learning. However, developing efficient methods for the nonconvex and nonsmooth optimization problems with certain performance guarantee remains a challenge. Proximal coordinate descent (PCD) has been widely used for solving opti…
New model approximates sparse mean-CVaR portfolio optimization efficiently.
We propose a new proximal, path-following framework for a class of constrained convex problems. We consider settings where the nonlinear---and possibly non-smooth---objective part is endowed with a proximity operator, and the constraint set is equipped with a self-concordant barrier. Our approach relies on the followin…
A new method solves convex optimization on curved spaces.
Paper introduces a new reinforcement learning method with improved performance.
This paper studies the lower bound complexity for the optimization problem whose objective function is the average of individual smooth convex functions. We consider the algorithm which gets access to gradient and proximal oracle for each individual component. For the strongly-convex case, we prove such an algorith…
Many scientific and engineering applications feature nonsmooth convex minimization problems over convex sets. In this paper, we address an important instance of this broad class where we assume that the nonsmooth objective is equipped with a tractable proximity operator and that the convex constraint set affords a self…
New algorithms accelerate model-based optimization for stochastic problems.
Improves time series classification with forest proximities.
Wasserstein distance plays increasingly important roles in machine learning, stochastic programming and image processing. Major efforts have been under way to address its high computational complexity, some leading to approximate or regularized variations such as Sinkhorn distance. However, as we will demonstrate, regu…
Improved random forest proximities capture data geometry.
We propose an inexact variable-metric proximal point algorithm to accelerate gradient-based optimization algorithms. The proposed scheme, called QNing can be notably applied to incremental first-order methods such as the stochastic variance-reduced gradient descent algorithm (SVRG) and other randomized incremental opti…
We develop a family of accelerated stochastic algorithms that minimize sums of convex functions. Our algorithms improve upon the fastest running time for empirical risk minimization (ERM), and in particular linear least-squares regression, across a wide range of problem settings. To achieve this, we establish a framewo…
New method solves saddle-point problems faster than existing methods.
A new method for RLHF using proximal point Nash learning.
Large sectors of the recent optimization literature focused in the last decade on the development of optimal stochastic first order schemes for constrained convex models under progressively relaxed assumptions. Stochastic proximal point is an iterative scheme born from the adaptation of proximal point algorithm to nois…
In this paper, we study the proximal gradient algorithm with extrapolation for minimizing the sum of a Lipschitz differentiable function and a proper closed convex function. Under the error bound condition used in [19] for analyzing the convergence of the proximal gradient algorithm, we show that there exists a thresho…
PPOS improves PPO by smoothing the surrogate objective function.
New algorithm solves non-convex, non-differentiable min-max games.
Two algorithms find optimal points in decentralized optimization.
In this paper, we consider high-dimensional nonconvex square-root-loss regression problems and introduce a proximal majorization-minimization (PMM) algorithm for these problems. Our key idea for making the proposed PMM to be efficient is to develop a sparse semismooth Newton method to solve the corresponding subproblem…
In this paper we propose a primal-dual proximal extragradient algorithm to solve the generalized Dantzig selector (GDS) estimation problem, based on a new convex-concave saddle-point (SP) reformulation. Our new formulation makes it possible to adopt recent developments in saddle-point optimization, to achieve the optim…
New method reduces variance in stochastic optimization with high confidence.
This paper improves cross-domain learning using random forests for manifold alignment.