arXiv research

A new method solves convex optimization problems on manifolds efficiently.

Paper proposes a new method for sparse spectral clustering on Stiefel manifold.

New algorithm solves phase retrieval with adaptive stopping criteria.

Innovative method solves nonconvex optimization on manifolds.

In this work, we highlight a connection between the incremental proximal method and stochastic filters. We begin by showing that the proximal operators coincide, and hence can be realized with, Bayes updates. We give the explicit form of the updates for the linear regression problem and show that there is a one-to-one …

New algorithms tackle machine learning problems using manifold proximal point methods.

Generative flows learn distributions on low-dimensional manifolds robustly via Wasserstein proximals.

Paper proposes a new method for supervised manifold learning using random forest proximities.

Large sectors of the recent optimization literature focused in the last decade on the development of optimal stochastic first order schemes for constrained convex models under progressively relaxed assumptions. Stochastic proximal point is an iterative scheme born from the adaptation of proximal point algorithm to nois…

Graph embedding learns low-dimensional representations for nodes in a graph and effectively preserves the graph structure. Recently, a significant amount of progress has been made toward this emerging research area. However, there are several fundamental problems that remain open. First, existing methods fail to preser…

Riemannian Proximal Sampler improves sampling on manifold data.

New method finds linear relationships across multiple data blocks using proximal gradient descent with $\ell_1$ constraint.

A new algorithm solves signed Fréchet regression on manifolds with bounded curvature.

In this paper, we extend the geometric descent method recently proposed by Bubeck, Lee and Singh to tackle nonsmooth and strongly convex composite problems. We prove that our proposed algorithm, dubbed geometric proximal gradient method (GeoPG), converges with a linear rate $(1-1/\sqrtκ)$ and thus achieves the optimal …

A new method reformulates Optimal Transport Conditional Flow Matching using proximal operators.

Two new methods solve nonsmooth optimization on Riemannian Stiefel manifold.

The classical multi-set split feasibility problem seeks a point in the intersection of finitely many closed convex domain constraints, whose image under a linear mapping also lies in the intersection of finitely many closed convex range constraints. Split feasibility generalizes important inverse problems including con…

Stochastic proximal point algorithm with momentum converges faster and is more stable than standard methods.

In this paper, we discuss the problem of minimizing the sum of two convex functions: a smooth function plus a non-smooth function. Further, the smooth part can be expressed by the average of a large number of smooth component functions, and the non-smooth part is equipped with a simple proximal mapping. We propose a pr…

This paper improves cross-domain learning using random forests for manifold alignment.

A wide class of regularization problems in machine learning and statistics employ a regularization term which is obtained by composing a simple convex function ωwith a linear transformation. This setting includes Group Lasso methods, the Fused Lasso and other total variation methods, multi-task learning methods and man…

Curvature regularization prevents distortion in graph embeddings.

New method improves structure learning on sparse graphs.

The paper addresses nonconvex penalized LAD estimation in partial linear models using DNNs.

Extends multidimensional scaling to analyze three-way asymmetric proximities.

The paper analyzes PPM for nonconvex-nonconcave problems, identifying three regions with varying convergence guarantees.

In [19], a general, inexact, efficient proximal quasi-Newton algorithm for composite optimization problems has been proposed and a sublinear global convergence rate has been established. In this paper, we analyze the convergence properties of this method, both in the exact and inexact setting, in the case when the obje…

Nonconvex and nonsmooth problems have recently attracted considerable attention in machine learning. However, developing efficient methods for the nonconvex and nonsmooth optimization problems with certain performance guarantee remains a challenge. Proximal coordinate descent (PCD) has been widely used for solving opti…

We develop model-based methods for solving stochastic convex optimization problems, introducing the approximate-proximal point, or aProx, family, which includes stochastic subgradient, proximal point, and bundle methods. When the modeling approaches we propose are appropriately accurate, the methods enjoy stronger conv…

Proximal algorithms applied to current deformation into cycles.

Modern proximal and stochastic gradient descent (SGD) methods are believed to efficiently minimize large composite objective functions, but such methods have two algorithmic challenges: (1) a lack of fast or justified stop conditions, and (2) sensitivity to the objective function's conditioning. In response to the firs…

Paper improves a method for fast global and local convergence in optimization.

A new optimization method, BPM, converges linearly in non-convex, non-smooth problems.

Nonconvex optimization problems arise in different research fields and arouse lots of attention in signal processing, statistics and machine learning. In this work, we explore the accelerated proximal gradient method and some of its variants which have been shown to converge under nonconvex context recently. We show th…

Region-specific linear models are widely used in practical applications because of their non-linear but highly interpretable model representations. One of the key challenges in their use is non-convexity in simultaneous optimization of regions and region-specific models. This paper proposes novel convex region-specific…

We propose a new algorithm---Stochastic Proximal Langevin Algorithm (SPLA)---for sampling from a log concave distribution. Our method is a generalization of the Langevin algorithm to potentials expressed as the sum of one stochastic smooth term and multiple stochastic nonsmooth terms. In each iteration, our splitting t…

Paper analyzes convergence of proximal algorithm in metric spaces without geodesic convexity.

New algorithm accelerates single-pass SGD for generalized linear prediction.

Unified view connects CoCoA and ADMM for distributed ERM.

New insights show NAG and FISTA converge linearly without knowing strong convexity modulus.

Recovering matrices from compressive and grossly corrupted observations is a fundamental problem in robust statistics, with rich applications in computer vision and machine learning. In theory, under certain conditions, this problem can be solved in polynomial time via a natural convex relaxation, known as Compressive …

In 1963, Polyak proposed a simple condition that is sufficient to show a global linear convergence rate for gradient descent. This condition is a special case of the Łojasiewicz inequality proposed in the same year, and it does not require strong convexity (or even convexity). In this work, we show that this much-older…

We analyze stochastic gradient algorithms for optimizing nonconvex, nonsmooth finite-sum problems. In particular, the objective function is given by the summation of a differentiable (possibly nonconvex) component, together with a possibly non-differentiable but convex component. We propose a proximal stochastic gradie…

In machine learning research, the proximal gradient methods are popular for solving various optimization problems with non-smooth regularization. Inexact proximal gradient methods are extremely important when exactly solving the proximal operator is time-consuming, or the proximal operator does not have an analytic sol…

Paper introduces a new reinforcement learning method with improved performance.

Paper proposes iLPA for solving DC composite optimization problems, with applications to matrix completion with outliers.

Sparse coding consists in representing signals as sparse linear combinations of atoms selected from a dictionary. We consider an extension of this framework where the atoms are further assumed to be embedded in a tree. This is achieved using a recently introduced tree-structured sparse regularization norm, which has pr…

In this paper, we propose a new algorithm to speed-up the convergence of accelerated proximal gradient (APG) methods. In order to minimize a convex function $f(\mathbf{x})$, our algorithm introduces a simple line search step after each proximal gradient step in APG so that a biconvex function $f(θ\mathbf{x})$ is minimi…