A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
The paper accelerates ISTA and FISTA algorithms for composite optimization problems.
Unified Lagrangian-based methods for nonsmooth nonconvex optimization.
We develop model-based methods for solving stochastic convex optimization problems, introducing the approximate-proximal point, or aProx, family, which includes stochastic subgradient, proximal point, and bundle methods. When the modeling approaches we propose are appropriately accurate, the methods enjoy stronger conv…
A new method solves convex optimization on curved spaces.
New algorithms accelerate model-based optimization for stochastic problems.
Inexact subgradient methods work well for semialgebraic functions with additive errors.
New algorithm solves phase retrieval with adaptive stopping criteria.
New insights show NAG and FISTA converge linearly without knowing strong convexity modulus.
New sampling methods for constrained and composite distributions.
We study a hybrid conditional gradient - smoothing algorithm (HCGS) for solving composite convex optimization problems which contain several terms over a bounded set. Examples of these include regularization problems with several norms as penalties and a norm constraint. HCGS extends conditional gradient methods to cas…
Unified framework for training neural networks with non-smooth, non-convex regularizers.
The paper analyzes convergence properties of NGA and PAMe for -norm PCA.
We consider the stochastic nested composition optimization problem where the objective is a composition of two expected-value functions. We proposed the stochastic ADMM to solve this complicated objective. In order to find an stationary point where the expected norm of the subgradient of corresponding augmented Lag…
Optimized method tackles convex optimization with heavy-tailed noise.
We consider optimization problems over the Stiefel manifold whose objective function is the summation of a smooth function and a nonsmooth function. Existing methods for solving this kind of problems can be classified into three classes. Algorithms in the first class rely on information of the subgradients of the objec…
The Alternating Direction Method of Multipliers (ADMM) has been studied for years. The traditional ADMM algorithm needs to compute, at each iteration, an (empirical) expected loss function on all training examples, resulting in a computational complexity proportional to the number of training examples. To reduce the ti…
Paper shows linear convergence of ISTA and FISTA for ill-conditioned images.
Two new methods solve nonsmooth optimization on Riemannian Stiefel manifold.
We study the problem of estimating high-dimensional regression models regularized by a structured sparsity-inducing penalty that encodes prior structural information on either the input or output variables. We consider two widely adopted types of penalties of this kind as motivating examples: (1) the general overlappin…
The paper addresses nonconvex penalized LAD estimation in partial linear models using DNNs.
New method bounds stochastic subgradient methods with heavy-tailed noise.
We study the problem of learning high dimensional regression models regularized by a structured-sparsity-inducing penalty that encodes prior structural information on either input or output sides. We consider two widely adopted types of such penalties as our motivating examples: 1) overlapping group lasso penalty, base…
New algorithms tackle machine learning problems using manifold proximal point methods.
The paper guarantees global stability for stochastic subgradient methods in nonsmooth nonconvex optimization.
A distributed subgradient method tackles non-convex optimization problems in networks.
Improved subgradient method tackles ill-conditioned composite optimization problems.
Paper proposes iLPA for solving DC composite optimization problems, with applications to matrix completion with outliers.
New Max-Plus neural network exploits subgradient sparsity for efficient training.
Novel coordinate descent (CD) methods are proposed for minimizing nonconvex functions consisting of three terms: (i) a continuously differentiable term, (ii) a simple convex term, and (iii) a concave and continuous term. First, by extending randomized CD to nonsmooth nonconvex settings, we develop a coordinate subgradi…
The paper derives subgradient estimates for a specific nonlinear subparabolic equation on pseudo-Hermitian manifolds.
We show that the Subgradient algorithm is universal for online learning on the simplex in the sense that it simultaneously achieves regret for adversarial costs and pseudo-regret for i.i.d costs. To the best of our knowledge this is the first demonstration of a universal algorithm on the simplex tha…
Proof of convergence for multi-objective optimization using inverse reinforcement learning.
New algorithms achieve high-probability parameter-free regret in online convex optimization with heavy-tailed data.
New adaptive methods solve weakly convex stochastic optimization problems.
Study proves convergence of subgradients for optimal transport-based objectives.
We describe novel subgradient methods for a broad class of matrix optimization problems involving nuclear norm regularization. Unlike existing approaches, our method executes very cheap iterations by combining low-rank stochastic subgradients with efficient incremental SVD updates, made possible by highly optimized and…
In this paper we study integer multiplicity rectifiable currents carried by the subgradient (subdifferential) graphs of semi-convex functions on a -dimensional convex domain, and show a weak continuity theorem with respect to pointwise convergence for such currents. As an application, the -Hessian measures are ca…
The conjugate gradient (CG) method is an efficient iterative method for solving large-scale strongly convex quadratic programming (QP). In this paper we propose some generalized CG (GCG) methods for solving the -regularized (possibly not strongly) convex QP that terminate at an optimal solution in a finite numb…
New algorithms solve large-scale convex regression problems.
Study robust recovery of low-rank matrices from corrupted measurements without rank prior.
We consider the problem of unconstrained online convex optimization (OCO) with sub-exponential noise, a strictly more general problem than the standard OCO. In this setting, the learner receives a subgradient of the loss functions corrupted by sub-exponential noise and strives to achieve optimal regret guarantee, witho…
In this note, we present a new averaging technique for the projected stochastic subgradient method. By using a weighted average with a weight of t+1 for each iterate w_t at iteration t, we obtain the convergence rate of O(1/t) with both an easy proof and an easy implementation. The new scheme is compared empirically to…
Composite convex optimization problems which include both a nonsmooth term and a low-rank promoting term have important applications in machine learning and signal processing, such as when one wishes to recover an unknown matrix that is simultaneously low-rank and sparse. However, such problems are highly challenging t…
Bayesian max-margin models have shown superiority in various practical applications, such as text categorization, collaborative prediction, social network link prediction and crowdsourcing, and they conjoin the flexibility of Bayesian modeling and predictive strengths of max-margin learning. However, Monte Carlo sampli…
Stochastic subgradient descent avoids critical points in definable functions.
Sparse methods for supervised learning aim at finding good linear predictors from as few variables as possible, i.e., with small cardinality of their supports. This combinatorial selection problem is often turned into a convex optimization problem by replacing the cardinality function by its convex envelope (tightest c…
This paper proves equivalences of portfolio optimization problems with negative expectile and omega ratio. We derive subgradients for the negative expectile as a function of the portfolio from a known dual representation of expectile and general theory about subgradients of risk measures. We also give an elementary der…
Paper presents an efficient algorithm for learning minimax risk classifiers with large-scale data.