In machine learning research, the proximal gradient methods are popular for solving various optimization problems with non-smooth regularization. Inexact proximal gradient methods are extremely important when exactly solving the proximal operator is time-consuming, or the proximal operator does not have an analytic sol…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “proximal gradient method”
Innovative method solves nonconvex optimization on manifolds.
problem Nonconvex optimization problems on Riemannian manifolds.
method Intrinsic Riemannian proximal gradient method.
result Converges for nonconvex or nonembedded problems.
Adaptive methods improve gradient descent and proximal gradient for convex optimization.
problem Improving efficiency of gradient descent and proximal gradient methods.
method Adaptive versions of GD and ProxGD using local curvature information.
result Proved convergence with local Lipschitz gradient assumptions.
We consider the problem of minimizing the sum of two convex functions: one is the average of a large number of smooth component functions, and the other is a general convex function that admits a simple proximal mapping. We assume the whole objective function is strongly convex. Such problems often arise in machine lea…
In this paper, we propose a probabilistic optimization method, named probabilistic incremental proximal gradient (PIPG) method, by developing a probabilistic interpretation of the incremental proximal gradient algorithm. We explicitly model the update rules of the incremental proximal gradient method and develop a syst…
Unified framework for training neural networks with non-smooth, non-convex regularizers.
problem Training neural networks with non-smooth, non-convex regularizers.
method ProxGen framework for stochastic proximal gradient descent.
result ProxGen framework achieves the same convergence rate as standard methods and outperforms subgradient-based approaches.
This paper converts ADMM to proximal gradient for efficient sparse estimation.
problem Sparse estimation problems like fused lasso and convex clustering.
method General method converting ADMM to proximal gradient, assuming Lipschitz continuity of derivative.
result Significant improvement in efficiency for sparse estimation problems.
A new method solves convex optimization problems on manifolds efficiently.
problem Optimization on Hadamard manifolds with convex objectives.
method Intrinsic Riemannian proximal gradient method.
result Sublinear and linear convergence rates for convex and strongly convex problems, respectively.
A relaxed proximal gradient descent algorithm for convergent plug-and-play with proximal denoiserstat.ML
New PnP algorithm converges with relaxed proximal gradient descent.
problem Convergence issues in PnP methods with deep denoisers.
method Relaxed proximal gradient descent for PnP with weakly convex regularization.
result Proposed PnP-PGD converges for a wider range of regularization parameters.
In this paper we consider solving saddle point problems using two variants of Gradient Descent-Ascent algorithms, Extra-gradient (EG) and Optimistic Gradient Descent Ascent (OGDA) methods. We show that both of these algorithms admit a unified analysis as approximations of the classical proximal point method for solving…
Gradient boosting is a prediction method that iteratively combines weak learners to produce a complex and accurate model. From an optimization point of view, the learning procedure of gradient boosting mimics a gradient descent on a functional variable. This paper proposes to build upon the proximal point algorithm, wh…
A method for estimating the median of gradients in stochastic optimization.
problem Robust gradient estimation in stochastic optimization for various applications.
method Stochastic Proximal Point Method for median gradient estimation.
result The proposed method can converge even under heavy-tailed, state-dependent noise.
In this paper, we investigate the attractive properties of the proximal gradient algorithm with inertia. Notably, we show that using alternated inertia yields monotonically decreasing functional values, which contrasts with usual accelerated proximal gradient methods. We also provide convergence rates for the algorithm…
Proximal gradient method has been playing an important role to solve many machine learning tasks, especially for the nonsmooth problems. However, in some machine learning problems such as the bandit model and the black-box learning problem, proximal gradient method could fail because the explicit gradients of these pro…
A regularized optimization problem over a large unstructured graph is studied, where the regularization term is tied to the graph geometry. Typical regularization examples include the total variation and the Laplacian regularizations over the graph. When applying the proximal gradient algorithm to solve this problem, t…
Improved bounds for proximal gradient algorithms with computational errors.
problem Analyzing convergence of proximal gradient algorithms with inaccuracies.
method Deriving new tighter deterministic and probabilistic bounds for convex composite problems.
result Probabilistic bounds are more robust and accurate for algorithm verification and performance guarantees.
We study a hybrid conditional gradient - smoothing algorithm (HCGS) for solving composite convex optimization problems which contain several terms over a bounded set. Examples of these include regularization problems with several norms as penalties and a norm constraint. HCGS extends conditional gradient methods to cas…
-norm constrained multi-block sparse canonical correlation analysis via proximal gradient descentstat.ME
New method finds linear relationships across multiple data blocks using proximal gradient descent with constraint.
problem Finding leading generalized eigenvectors for multi-block CCA.
method Proximal gradient descent with constraint.
result Rate-optimal solution under suitable assumptions.
New method improves structure learning on sparse graphs.
problem Structure learning on sparse directed acyclic graphs (DAGs).
method Bregman proximal gradient method to address non-convex, high-curvature problem.
result Significantly improved convergence and efficiency.
Paper introduces a new reinforcement learning method with improved performance.
problem Designing and analyzing efficient reinforcement learning algorithms.
method Proximal gradient temporal difference learning (GTD) with accelerated algorithm GTD2-MP.
result GTD algorithms have linear complexity and improved convergence rate.
We analyze Riemannian accelerated methods using a new framework.
problem Understanding Riemannian accelerated gradient methods.
method Riemannian A-HPE framework, focusing on Euclidean A-HPE insights and metric distortion control.
result Characterization of acceleration for various Riemannian methods.
Accelerated first-order methods for convex optimization with locally Lipschitz continuous gradientmath.OC
New methods for convex optimization with locally Lipschitz gradient, achieving faster convergence.
problem Optimization problems with locally Lipschitz continuous gradient.
method Accelerated proximal gradient (APG) methods and proximal augmented Lagrangian method.
result Achieved faster convergence rates for convex optimization problems with locally Lipschitz gradient.
Improved Last-Iterate Convergence of Shuffling Gradient Methods for Nonsmooth Convex Optimizationmath.OC
Improved shuffling gradient methods converge faster for nonsmooth convex optimization.
problem Improving convergence rates for nonsmooth convex optimization problems.
method Analysis of shuffling gradient methods, focusing on Random Reshuffle and Single Shuffle strategies.
result Shuffling gradient methods, particularly Random Reshuffle and Single Shuffle, converge faster than Proximal Gradient Descent for nonsmooth convex optimization.
This paper accelerates TV regularization algorithms by unrolling proximal gradient descent.
problem Solving Total Variation (TV) regularized problems with iterative algorithms.
method Unrolling proximal gradient descent solvers to learn their parameters.
result Two approaches to compute derivatives through proximal operators improve performance.
A new method solves l1-regularized optimization problems efficiently and sparsely.
problem l1-regularized optimization problems in machine learning.
method Orthant Based Proximal Stochastic Gradient Method (OBProx-SG)
result Promotes sparsity of solutions substantially and converges to global optimal solutions.
Stochastic proximal point algorithm with momentum converges faster and is more stable than standard methods.
problem Improving convergence and stability of stochastic optimization methods.
method Developed and analyzed the convergence and stability of the stochastic proximal point algorithm with momentum (SPPAM).
result SPPAM converges faster and is more stable than standard stochastic proximal point algorithm (SPPA) and stochastic gradient descent with momentum (SGDM).
Riemannian Stochastic Proximal Gradient Methods for Nonsmooth Optimization over the Stiefel Manifoldmath.OC
Two new methods solve nonsmooth optimization on Riemannian Stiefel manifold.
problem Optimization over nonsmooth, non-differentiable functions on Riemannian manifolds.
method R-ProxSGD and R-ProxSPB, generalizing proximal SGD and SpiderBoost.
result R-ProxSPB finds ε-stationary points with IFO complexity of Ø(ε^(-3)) in online and Ø(n + √nε^(-2)) in finite-sum cases.
Policy-gradient method controls multiple non-cohesive targets.
problem Controlling multiple non-cohesive targets in a decentralized manner.
method Proximal Policy Optimization for target selection and driving.
result Effective control of non-cohesive targets without prior dynamics knowledge.
In this paper, we propose a new algorithm to speed-up the convergence of accelerated proximal gradient (APG) methods. In order to minimize a convex function , our algorithm introduces a simple line search step after each proximal gradient step in APG so that a biconvex function is minimi…
Paper analyzes convergence of proximal algorithm in metric spaces without geodesic convexity.
problem Analyzing convergence of proximal algorithm in general metric spaces.
method Analysis of the Wasserstein proximal algorithm without geodesic convexity assumption.
result Establishes unbiased and linear convergence rate for proximal algorithm under natural Wasserstein inequality.
We propose a new optimization method for training feed-forward neural networks. By rewriting the activation function as an equivalent proximal operator, we approximate a feed-forward neural network by adding the proximal operators to the objective function as penalties, hence we call the lifted proximal operator machin…
Adaptive sampling method solves constrained and composite optimization problems.
problem Solving constrained optimization problems with stochastic objectives and deterministic constraints.
method Proximal gradient method with adaptive sampling to improve gradient approximation quality.
result Convergence results established for both strongly convex and general convex objectives.
We use differential equations based approaches to provide some {\it \textbf{physics}} insights into analyzing the dynamics of popular optimization algorithms in machine learning. In particular, we study gradient descent, proximal gradient descent, coordinate gradient descent, proximal coordinate gradient, and Newton's …
Develops a new SPP algorithm with variance reduction for weakly convex optimization.
problem Weakly convex, composite optimization problems.
method Inexact semismooth Newton framework with variance reduction for stochastic proximal point updates.
result Establishes convergence results for the proposed algorithm.
In this paper, we extend the geometric descent method recently proposed by Bubeck, Lee and Singh to tackle nonsmooth and strongly convex composite problems. We prove that our proposed algorithm, dubbed geometric proximal gradient method (GeoPG), converges with a linear rate and thus achieves the optimal …
Optimization is at the heart of machine learning, statistics and many applied scientific disciplines. It also has a long history in physics, ranging from the minimal action principle to finding ground states of disordered systems such as spin glasses. Proximal algorithms form a class of methods that are broadly applica…
Efficient algorithms solve joint graphical lasso problems.
problem Learning graphical models from sparse data.
method Proximal gradient procedures with ADMM backtracking option.
result Proposed algorithms achieve high accuracy and precision.
New algorithm speeds up sampling from complex distributions.
problem Efficiently sampling from non-log-concave distributions.
method Stochastic Proximal Samplers (SPS) based on SGLD and MALA.
result SPS-SGLD and SPS-MALA achieve faster sampling with reduced gradient complexity.
We analyze stochastic gradient algorithms for optimizing nonconvex, nonsmooth finite-sum problems. In particular, the objective function is given by the summation of a differentiable (possibly nonconvex) component, together with a possibly non-differentiable but convex component. We propose a proximal stochastic gradie…
Federated learning approach for binary matrix factorization.
problem Efficiently factorizing binary data distributed across stakeholders while maintaining privacy.
method Proximal optimization for federated learning of relaxed binary matrix factorization.
result Federated algorithm outperforms state-of-the-art methods in quality and efficacy.
The incremental aggregated gradient algorithm is popular in network optimization and machine learning research. However, the current convergence results require the objective function to be strongly convex. And the existing convergence rates are also limited to linear convergence. Due to the mathematical techniques, th…
In this paper, we consider a class of possibly nonconvex, nonsmooth and non-Lipschitz optimization problems arising in many contemporary applications such as machine learning, variable selection and image processing. To solve this class of problems, we propose a proximal gradient method with extrapolation and line sear…
Accelerates coordinate descent methods for machine learning problems.
problem Slowness of coordinate descent methods in machine learning.
method Extrapolation-based accelerated coordinate descent.
result Significant speed-up in practice compared to existing methods.
We present a distributed proximal-gradient method for optimizing the average of convex functions, each of which is the private local objective of an agent in a network with time-varying topology. The local objectives have distinct differentiable components, but they share a common nondifferentiable component, which has…
Stochastic Proximal Gradient Algorithm with Minibatches. Application to Large Scale Learning Modelscs.LG
Develops minibatch stochastic proximal gradient for large-scale learning models.
problem Finding optimal predictors with complex regularizers in large-scale learning models.
method Minibatch variants of stochastic proximal gradient algorithm for composite objective functions.
result Minibatch size after iterations achieves suboptimality in expected quadratic distance.
A new PGA algorithm ensures stable, robust, and noise-immune solutions for non-negative inverse problems.
problem Stable convergence and suboptimal solutions in inverse problems due to negative values and high sensitivity to hyperparameters.
method A novel multiplicative update proximal gradient algorithm (SSO-PGA) that enforces non-negativity and boundedness through a learnable sigmoid-based operator.
result Significantly surpasses traditional PGA and other state-of-the-art algorithms in performance and stability.
We study the problem of estimating high-dimensional regression models regularized by a structured sparsity-inducing penalty that encodes prior structural information on either the input or output variables. We consider two widely adopted types of penalties of this kind as motivating examples: (1) the general overlappin…
The paper accelerates ISTA and FISTA algorithms for composite optimization problems.
problem Improving convergence rates of ISTA and FISTA for composite optimization.
method Improved proximal subgradient norm minimization using Lyapunov function.
result Convergence rates of ISTA and FISTA are accelerated.