New inexact proximal gradient methods solve non-convex optimization problems.
problem Solving non-convex optimization problems with non-smooth regularization.
method Proposed three inexact proximal gradient algorithms, including basic and Nesterov's accelerated versions.
result Theoretical analysis shows convergence rates similar to exact methods.
Inertial proximal gradient algorithm shows monotonically decreasing values.
problem Optimizing functions with inertia.
method Proximal gradient algorithm with alternated inertia.
result Algorithm with alternated inertia achieves monotonically decreasing functional values.
Proposes a probabilistic optimization method for large-scale problems.
problem Large-scale regularized optimization problems.
method Develops a probabilistic interpretation of the incremental proximal gradient algorithm and uses Bayesian filtering.
result Makes it possible to solve large-scale problems using well-known Bayesian filters.
Innovative method solves nonconvex optimization on manifolds.
problem Nonconvex optimization problems on Riemannian manifolds.
method Intrinsic Riemannian proximal gradient method.
result Converges for nonconvex or nonembedded problems.
This paper converts ADMM to proximal gradient for efficient sparse estimation.
problem Sparse estimation problems like fused lasso and convex clustering.
method General method converting ADMM to proximal gradient, assuming Lipschitz continuity of derivative.
result Significant improvement in efficiency for sparse estimation problems.
Adaptive methods improve gradient descent and proximal gradient for convex optimization.
problem Improving efficiency of gradient descent and proximal gradient methods.
method Adaptive versions of GD and ProxGD using local curvature information.
result Proved convergence with local Lipschitz gradient assumptions.
Snake solves large graph optimization problems with fast proximal steps.
problem Optimization over large unstructured graphs with graph-specific regularization.
method Snake algorithm using random simple paths for proximal gradient steps.
result Convergence proven for the Snake algorithm.
A new method solves convex optimization problems on manifolds efficiently.
problem Optimization on Hadamard manifolds with convex objectives.
method Intrinsic Riemannian proximal gradient method.
result Sublinear and linear convergence rates for convex and strongly convex problems, respectively.
Improved bounds for proximal gradient algorithms with computational errors.
problem Analyzing convergence of proximal gradient algorithms with inaccuracies.
method Deriving new tighter deterministic and probabilistic bounds for convex composite problems.
result Probabilistic bounds are more robust and accurate for algorithm verification and performance guarantees.
Develops minibatch stochastic proximal gradient for large-scale learning models.
problem Finding optimal predictors with complex regularizers in large-scale learning models.
method Minibatch variants of stochastic proximal gradient algorithm for composite objective functions.
result Minibatch size N after O(Nε1) iterations achieves ε−suboptimality in expected quadratic distance. New method finds linear relationships across multiple data blocks using proximal gradient descent with ℓ1 constraint.
problem Finding leading generalized eigenvectors for multi-block CCA.
method Proximal gradient descent with ℓ1 constraint. result Rate-optimal solution under suitable assumptions.
In this paper, we propose a new algorithm to speed-up the convergence of accelerated proximal gradient (APG) methods. In order to minimize a convex function f(x), our algorithm introduces a simple line search step after each proximal gradient step in APG so that a biconvex function f(θx) is minimi…
New method speeds up optimization for complex functions.
problem Optimizing complex functions with stochastic composition.
method Accelerated stochastic compositional proximal gradient (ASC-PG) method.
result ASC-PG achieves faster convergence and optimal sample-error complexity.
New PnP algorithm converges with relaxed proximal gradient descent.
problem Convergence issues in PnP methods with deep denoisers.
method Relaxed proximal gradient descent for PnP with weakly convex regularization.
result Proposed PnP-αPGD converges for a wider range of regularization parameters. New method accelerates optimization for nonconvex problems in statistics.
problem Nonconvex optimization problems in statistics and machine learning.
method Adaptive momentum and block coordinate proximal gradient method with specific update rules.
result The method achieves provable local linear convergence in sparse linear regression problems.
Efficient algorithms solve joint graphical lasso problems.
problem Learning graphical models from sparse data.
method Proximal gradient procedures with ADMM backtracking option.
result Proposed algorithms achieve high accuracy and precision.
This paper accelerates TV regularization algorithms by unrolling proximal gradient descent.
problem Solving Total Variation (TV) regularized problems with iterative algorithms.
method Unrolling proximal gradient descent solvers to learn their parameters.
result Two approaches to compute derivatives through proximal operators improve performance.
New method solves sparse PCA and CCA with guaranteed convergence.
problem Sparse PCA and CCA for large-scale data analysis.
method Alternating manifold proximal gradient method.
result Unified convergence analysis for the proposed method.
In this paper, we study the proximal gradient algorithm with extrapolation for minimizing the sum of a Lipschitz differentiable function and a proper closed convex function. Under the error bound condition used in [19] for analyzing the convergence of the proximal gradient algorithm, we show that there exists a thresho…
A new method combines extrapolation and line search for solving nonconvex, nonsmooth optimization problems.
problem Nonconvex, nonsmooth optimization problems in machine learning and image processing.
method Proximal gradient method with extrapolation and line search (PGels).
result The method reduces to existing algorithms under proper parameter choices and converges to stationary points.
Unified framework for training neural networks with non-smooth, non-convex regularizers.
problem Training neural networks with non-smooth, non-convex regularizers.
method ProxGen framework for stochastic proximal gradient descent.
result ProxGen framework achieves the same convergence rate as standard methods and outperforms subgradient-based approaches.
A new PGA algorithm ensures stable, robust, and noise-immune solutions for non-negative inverse problems.
problem Stable convergence and suboptimal solutions in inverse problems due to negative values and high sensitivity to hyperparameters.
method A novel multiplicative update proximal gradient algorithm (SSO-PGA) that enforces non-negativity and boundedness through a learnable sigmoid-based operator.
result Significantly surpasses traditional PGA and other state-of-the-art algorithms in performance and stability.
Develops an asynchronous distributed algorithm for convex optimization.
problem Distributed convex optimization with varying communication costs and delays.
method A flexible proximal gradient algorithm that adapts to different levels of communication and delays.
result The algorithm converges linearly in the strongly convex case and provides convergence guarantees for non-strongly convex functions.
Proposes a new metric selection for VM-PG with improved convergence.
problem Improves convergence of VM-PG methods for ill-conditioned problems.
method Diagonal Barzilai-Borwein stepsize for adaptive metric selection.
result Improved convergence results for ill-conditioned problems.
A new method relaxes Boolean Matrix Factorization to make it more efficient.
problem High computational cost of solving NP-hard combinatorial optimization problems in Boolean Matrix Factorization.
method Proposes a proximal gradient algorithm using an elastic-binary regularizer to relax BMF.
result Demonstrates improved runtime and better recall, loss, and interpretability on real-world data.
A new algorithm SLA reduces bias in sampling from measures using Langevin dynamics.
problem Reducing bias in sampling from complex target measures using Langevin dynamics.
method Proposed symmetrized Langevin algorithm (SLA) to correct bias in ULA.
result SLA is consistent for Gaussian target measures, while ULA is not.
New algorithm solves non-convex, non-differentiable min-max games.
problem Limited theoretical understanding of non-smooth min-max games.
method Proximal gradient descent-ascent algorithm for convex-strongly convex games.
result Algorithm converges to ε-Nash equilibrium with polynomial gradient evaluations.
A new method solves optimization problems over Stiefel manifold.
problem Optimization problems over the Stiefel manifold with smooth and nonsmooth components.
method Proximal gradient method based on retraction.
result The method globally converges to a stationary point and has iteration complexity.
We study the problem of estimating high-dimensional regression models regularized by a structured sparsity-inducing penalty that encodes prior structural information on either the input or output variables. We consider two widely adopted types of penalties of this kind as motivating examples: (1) the general overlappin…
New RNN reconstructs video frames from sparse measurements.
problem Sequential signal reconstruction from compressive measurements.
method Unfolding proximal gradient method for l1-l1 minimization.
result Outperforms state-of-the-art RNN models in video frame reconstruction.
Paper introduces a new reinforcement learning method with improved performance.
problem Designing and analyzing efficient reinforcement learning algorithms.
method Proximal gradient temporal difference learning (GTD) with accelerated algorithm GTD2-MP.
result GTD algorithms have linear complexity and improved convergence rate.
A new algorithm speeds up convex clustering.
problem Optimizing clustering with convex optimization and avoiding local minima.
method Smoothing proximal gradient algorithm (Sproga) for convex clustering.
result Sproga is faster and uses less memory than existing methods.
A new screening rule speeds up OWL regression solving.
problem High-dimensional sparse learning with OWL regression's computational cost and memory usage.
method Safe screening rule for OWL regression using iterative strategy.
result Significant computational gain without accuracy loss.
Physics insights into optimization algorithms using differential equations.
problem Understanding dynamics of optimization algorithms in machine learning.
method Unified framework based on physical systems analysis of popular optimization algorithms.
result Unified analysis applicable to non-convex and non-strongly convex problems.
Extends geometric descent method for convex composite problems.
problem Nonsmooth and strongly convex composite problems.
method Geometric Proximal Gradient Method (GeoPG)
result Achieves optimal linear convergence rate of (1-1/\sqrtκ).
Adaptive sampling method solves constrained and composite optimization problems.
problem Solving constrained optimization problems with stochastic objectives and deterministic constraints.
method Proximal gradient method with adaptive sampling to improve gradient approximation quality.
result Convergence results established for both strongly convex and general convex objectives.
New method tackles nonconvex-nonconcave problems with local KL condition.
problem Nonconvex-nonconcave minimax problems under varying KL conditions.
method Inexact proximal gradient method for KL-structured subproblems.
result Complexity guarantees for approximate stationary points.
The paper proposes an efficient algorithm for solving Schatten-p quasi-norm problems.
problem Finding low-rank solutions of linear inverse problems with Schatten-p quasi-norm regularization. method Dynamic proximal gradient algorithm using Cayley transformation and adaptive step size selection.
result The algorithm converges to a stationary point of the objective function under mild assumptions.
New method improves structure learning on sparse graphs.
problem Structure learning on sparse directed acyclic graphs (DAGs).
method Bregman proximal gradient method to address non-convex, high-curvature problem.
result Significantly improved convergence and efficiency.
NASP uses proximal gradient descent to speed up neural architecture search.
problem Efficiently search for high-performance neural architectures.
method Differentiable Neural Architecture Search using Proximal gradient descent.
result NASP achieves 10 times speedup over DARTS while maintaining high performance.
A new decentralized optimization method with independent step-sizes and separated convergence rates.
problem Decentralized optimization with composite objective terms.
method Proximal-gradient algorithm with uncoordinated step-sizes and separated convergence rates.
result Linear convergence for special case without non-smooth terms under strong convexity.
Polyak-Łojasiewicz inequality simplifies linear convergence proofs.
problem Proving linear convergence without strong convexity.
method Using Polyak-Łojasiewicz inequality to analyze various optimization methods.
result New analyses and proofs of linear convergence for multiple machine learning problems.
Two new methods solve nonsmooth optimization on Riemannian Stiefel manifold.
problem Optimization over nonsmooth, non-differentiable functions on Riemannian manifolds.
method R-ProxSGD and R-ProxSPB, generalizing proximal SGD and SpiderBoost.
result R-ProxSPB finds ε-stationary points with IFO complexity of Ø(ε^(-3)) in online and Ø(n + √nε^(-2)) in finite-sum cases.
A new method reduces bias in adaptive Lasso estimates.
problem Bias in adaptive Lasso estimates.
method Proximal gradient approach to learn penalty coefficients as decision variables.
result Reduces bias in estimates and encourages arbitrary sparsity structure.
Accelerates coordinate descent methods for machine learning problems.
problem Slowness of coordinate descent methods in machine learning.
method Extrapolation-based accelerated coordinate descent.
result Significant speed-up in practice compared to existing methods.
Paper tackles NAS problem by modeling it as a sparse supernet.
problem Neural Architecture Search (NAS) problem, particularly Mixed-Path Search.
method Model NAS as a sparse supernet with sparsity constraints. Use hierarchical accelerated proximal gradient algorithm for optimization.
result Proposed method finds compact, general, and powerful neural architectures.
New method improves robust low-rank matrix completion for computer vision.
problem Robust low-rank matrix completion for partially observed data.
method Formulated as a nonsmooth Riemannian optimization problem over Grassmann manifold, solved with an alternating manifold proximal gradient continuation method.
result Demonstrated advantages over existing approaches in background extraction from surveillance videos.
PGD algorithm converges to local minima in nonconvex matrix completion.
problem Matrix completion with low-rank promotion using nonconvex penalties.
method Proximal gradient descent algorithm for nonconvex penalties.
result PGD algorithm converges to restricted strictly local minimizers with eventually linear rate.