A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New Max-Plus neural network exploits subgradient sparsity for efficient training.
The paper accelerates ISTA and FISTA algorithms for composite optimization problems.
This paper concerns dictionary learning, i.e., sparse coding, a fundamental representation learning problem. We show that a subgradient descent algorithm, with random initialization, can provably recover orthogonal dictionaries on a natural nonsmooth, nonconvex minimization formulation of the problem, under mi…
New method estimates sparse mean from noisy data without knowing sparsity level.
Study robust recovery of low-rank matrices from corrupted measurements without rank prior.
We show that the Subgradient algorithm is universal for online learning on the simplex in the sense that it simultaneously achieves regret for adversarial costs and pseudo-regret for i.i.d costs. To the best of our knowledge this is the first demonstration of a universal algorithm on the simplex tha…
We study a hybrid conditional gradient - smoothing algorithm (HCGS) for solving composite convex optimization problems which contain several terms over a bounded set. Examples of these include regularization problems with several norms as penalties and a norm constraint. HCGS extends conditional gradient methods to cas…
New algorithms achieve high-probability parameter-free regret in online convex optimization with heavy-tailed data.
A distributed subgradient method tackles non-convex optimization problems in networks.
New algorithms solve large-scale convex regression problems.
New method bounds stochastic subgradient methods with heavy-tailed noise.
Inexact subgradient methods work well for semialgebraic functions with additive errors.
We describe novel subgradient methods for a broad class of matrix optimization problems involving nuclear norm regularization. Unlike existing approaches, our method executes very cheap iterations by combining low-rank stochastic subgradients with efficient incremental SVD updates, made possible by highly optimized and…
AIHT improves online high-dimensional quantile regression by separating support discovery and refinement.
We consider the problem of unconstrained online convex optimization (OCO) with sub-exponential noise, a strictly more general problem than the standard OCO. In this setting, the learner receives a subgradient of the loss functions corrupted by sub-exponential noise and strives to achieve optimal regret guarantee, witho…
In this paper, we define the geometric median of a probability measure on a Riemannian manifold, give its characterization and a natural condition to ensure its uniqueness. In order to calculate the median in practical cases, we also propose a subgradient algorithm and prove its convergence as well as estimating the er…
Paper presents an efficient algorithm for learning minimax risk classifiers with large-scale data.
In this paper, we propose a convergent parallel best-response algorithm with the exact line search for the nondifferentiable nonconvex sparsity-regularized rank minimization problem. On the one hand, it exhibits a faster convergence than subgradient algorithms and block coordinate descent algorithms. On the other hand,…
The paper guarantees global stability for stochastic subgradient methods in nonsmooth nonconvex optimization.
Given a convex optimization problem and its dual, there are many possible first-order algorithms. In this paper, we show the equivalence between mirror descent algorithms and algorithms generalizing the conditional gradient method. This is done through convex duality, and implies notably that for certain problems, such…
We propose a randomized block-coordinate variant of the classic Frank-Wolfe algorithm for convex optimization with block-separable constraints. Despite its lower iteration cost, we show that it achieves a similar convergence rate in duality gap as the full Frank-Wolfe algorithm. We also show that, when applied to the d…
Improved subgradient method tackles ill-conditioned composite optimization problems.
This paper proposes exact and approximation algorithms for Sparse PCA, improving interpretability and scalability.
In this paper, a new theory is developed for first-order stochastic convex optimization, showing that the global convergence rate is sufficiently quantified by a local growth rate of the objective function in a neighborhood of the optimal solutions. In particular, if the objective function in the -sub…
In this paper we analyze boosting algorithms in linear regression from a new perspective: that of modern first-order methods in convex optimization. We show that classic boosting algorithms in linear regression, namely the incremental forward stagewise algorithm (FS) and least squares boosting (LS-Boost($…
New algorithms optimize spectral risk measures, improving interpolation between average and worst-case performance.
We consider optimization problems over the Stiefel manifold whose objective function is the summation of a smooth function and a nonsmooth function. Existing methods for solving this kind of problems can be classified into three classes. Algorithms in the first class rely on information of the subgradients of the objec…
Paper shows linear convergence of ISTA and FISTA for ill-conditioned images.
Gaussian Markov random fields (GMRFs) are useful in a broad range of applications. In this paper we tackle the problem of learning a sparse GMRF in a high-dimensional space. Our approach uses the l1-norm as a regularization on the inverse covariance matrix. We utilize a novel projected gradient method, which is faster …
Stochastic subgradient descent avoids critical points in definable functions.
The paper derives subgradient estimates for a specific nonlinear subparabolic equation on pseudo-Hermitian manifolds.
New algorithm reduces privacy loss in SGD without learning rate tuning.
Proof of convergence for multi-objective optimization using inverse reinforcement learning.
We present new algorithms to compute the mean of a set of empirical probability measures under the optimal transport metric. This mean, known as the Wasserstein barycenter, is the measure that minimizes the sum of its Wasserstein distances to each element in that set. We propose two original algorithms to compute Wasse…
The paper tackles efficient online learning by achieving minimal regret with respect to the best expert.
Unified Lagrangian-based methods for nonsmooth nonconvex optimization.
New method for optimization on Hadamard manifolds with curvature-independent guarantees.
Stochastic algorithm achieves sublinear convergence for bi-objective optimization.
Study proves convergence of subgradients for optimal transport-based objectives.
A new method solves convex optimization on curved spaces.
In this paper we study integer multiplicity rectifiable currents carried by the subgradient (subdifferential) graphs of semi-convex functions on a -dimensional convex domain, and show a weak continuity theorem with respect to pointwise convergence for such currents. As an application, the -Hessian measures are ca…
We propose graph-dependent implicit regularisation strategies for distributed stochastic subgradient descent (Distributed SGD) for convex problems in multi-agent learning. Under the standard assumptions of convexity, Lipschitz continuity, and smoothness, we establish statistical learning rates that retain, up to logari…
New algorithms accelerate model-based optimization for stochastic problems.
Linear encoding of sparse vectors is widely popular, but is commonly data-independent -- missing any possible extra (but a priori unknown) structure beyond sparsity. In this paper we present a new method to learn linear encoders that adapt to data, while still performing well with the widely used decoder. The …
We develop model-based methods for solving stochastic convex optimization problems, introducing the approximate-proximal point, or aProx, family, which includes stochastic subgradient, proximal point, and bundle methods. When the modeling approaches we propose are appropriately accurate, the methods enjoy stronger conv…
We study the problem of learning high dimensional regression models regularized by a structured-sparsity-inducing penalty that encodes prior structural information on either input or output sides. We consider two widely adopted types of such penalties as our motivating examples: 1) overlapping group lasso penalty, base…
We consider in this paper a class of composite optimization problems whose objective function is given by the summation of a general smooth and nonsmooth component, together with a relatively simple nonsmooth term. We present a new class of first-order methods, namely the gradient sliding algorithms, which can skip the…
In this note, we present a new averaging technique for the projected stochastic subgradient method. By using a weighted average with a weight of t+1 for each iterate w_t at iteration t, we obtain the convergence rate of O(1/t) with both an easy proof and an easy implementation. The new scheme is compared empirically to…