Inertial proximal gradient algorithm shows monotonically decreasing values.
problem Optimizing functions with inertia.
method Proximal gradient algorithm with alternated inertia.
result Algorithm with alternated inertia achieves monotonically decreasing functional values.
We consider a proximal operator given by a quadratic function subject to bound constraints and give an optimization algorithm using the alternating direction method of multipliers (ADMM). The algorithm is particularly efficient to solve a collection of proximal operators that share the same quadratic form, or if the qu…
New method solves sparse PCA and CCA with guaranteed convergence.
problem Sparse PCA and CCA for large-scale data analysis.
method Alternating manifold proximal gradient method.
result Unified convergence analysis for the proposed method.
New method uses zeroth-order queries to approximate proximal sampling efficiently.
problem Approximating proximal sampling with zeroth-order information.
method Direct simulation of heat flow dynamics, treating intermediate distribution as Gaussian mixture.
result Inherits exponential convergence under isoperimetric conditions, avoids rejection sampling.
The paper analyzes convergence properties of NGA and PAMe for L1-norm PCA.
problem Finite-step convergence of L1-norm PCA algorithms. method Conditional subgradient and alternating maximization interpretations of NGA, and PAMe with extrapolation.
result Iterative points of modified NGA and PAMe remain constant after finitely many steps under certain conditions.
New algorithm improves convergence rates for convex optimization problems.
problem Convex optimization problems with noisy stochastic data.
method Stochastic proximal point algorithm with weak linear regularity condition.
result Achieves $\mathcal{O}\left(\frac{1}{k}
ight)$ convergence rate for SPP.
New sampling algorithm for non-smooth potentials.
problem Sampling from non-smooth potentials.
method Proximal algorithm based on rejection sampling.
result Achieves better complexity than existing methods.
Revisits PPO design choices, exposing failure modes and proposing alternatives.
problem Failure modes of standard PPO in new environments.
method Revisits standard PPO design choices, exposes failure modes, and proposes alternative approaches.
result Alternative design choices prevent failure modes in new environments.
The Alternating Direction Method of Multipliers (ADMM) has been studied for years. The traditional ADMM algorithm needs to compute, at each iteration, an (empirical) expected loss function on all training examples, resulting in a computational complexity proportional to the number of training examples. To reduce the ti…
Efficient algorithms solve joint graphical lasso problems.
problem Learning graphical models from sparse data.
method Proximal gradient procedures with ADMM backtracking option.
result Proposed algorithms achieve high accuracy and precision.
A new algorithm speeds up convex clustering.
problem Optimizing clustering with convex optimization and avoiding local minima.
method Smoothing proximal gradient algorithm (Sproga) for convex clustering.
result Sproga is faster and uses less memory than existing methods.
Paper proposes a new method for supervised manifold learning using random forest proximities.
problem Existing supervised manifold learning methods fail to uncover meaningful embeddings due to using class-conditional distances.
method Proposes a data-geometry-preserving variant of random forest proximities as an initialization for manifold learning methods.
result Local and global structure preservation is near universal across manifold learning approaches using diffusion-based algorithms.
This paper converts ADMM to proximal gradient for efficient sparse estimation.
problem Sparse estimation problems like fused lasso and convex clustering.
method General method converting ADMM to proximal gradient, assuming Lipschitz continuity of derivative.
result Significant improvement in efficiency for sparse estimation problems.
New model approximates sparse mean-CVaR portfolio optimization efficiently.
problem NP-hard ℓ0-constrained mean-CVaR optimization. method Proximal alternating linearized minimization algorithm with nested fixed-point proximity.
result The model offers a guaranteed approximation of the ℓ0-constrained mean-CVaR model. New algorithm solves minimax games with linear constraints.
problem Nonconvex minimax games with coupled linear constraints.
method Primal-dual alternating proximal gradient (PDAPG) algorithm.
result Achieves ε-stationary solution within O(ε^(-2)) iterations for strongly concave settings.
We propose a new proximal, path-following framework for a class of constrained convex problems. We consider settings where the nonlinear---and possibly non-smooth---objective part is endowed with a proximity operator, and the constraint set is equipped with a self-concordant barrier. Our approach relies on the followin…
New algorithms solve complex minimax problems without needing derivatives.
problem Solving nonconvex-concave minimax problems efficiently.
method Zeroth-order alternating and proximal gradient algorithms.
result Iteration complexity and function value estimation bounds established.
New method improves matrix factorization speed and accuracy.
problem Matrix factorization optimization problems suffer from biased solutions and lack of convergence guarantees.
method Proposes a novel Bregman distance for matrix factorization, enabling non-alternating schemes with convergence proof.
result Convergence to a stationary point proved for matrix factorization problems.
Complex embeddings handle non-metric proximity data better than traditional methods.
problem Proximities not always metric or inner product-based, causing convergence issues.
method Proposes complex-valued embeddings for non-vectorial data.
result Complex embeddings outperform traditional techniques on benchmarks.
Paper proposes a new method to separate low rank and sparse matrices without bias.
problem Recovering low rank and sparse matrices from measurements.
method Uses nonconvex regularizers and alternating proximal gradient descent.
result Error bounds for the algorithm applied to sparse optimization, matrix completion, and robust PCA.
New methods solve non-Lipschitz smooth problems with guaranteed convergence.
problem Non-Lipschitz smooth problems in machine learning and signal processing.
method Bregman-divergence based algorithms for relatively smooth problems.
result Guaranteed convergence to second-order stationary points for any relatively smooth problem.
Chandrasekaran, Parrilo and Willsky (2010) proposed a convex optimization problem to characterize graphical model selection in the presence of unobserved variables. This convex optimization problem aims to estimate an inverse covariance matrix that can be decomposed into a sparse matrix minus a low-rank matrix from sam…
New algorithms for faster Schatten quasi-norm minimization.
problem Efficiently approximate matrix rank for large-scale problems.
method Define and solve tractable Schatten quasi-norms, design efficient algorithms.
result Proven algorithms are orders of magnitude faster and more accurate.
New algorithm solves GDS with faster convergence and FDR control.
problem Generalized Dantzig Selector estimation problem.
method Primal-dual proximal extragradient algorithm with saddle-point reformulation.
result Achieves optimal O(1/k) convergence rate. POP3D is a new reinforcement learning algorithm that improves upon PPO.
problem The shortcomings of existing reinforcement learning algorithms.
method Policy Optimization with Penalized Point Probability Distance (POP3D) as a lower bound to the square of total variance divergence.
result POP3D is highly competitive compared to PPO in various benchmarks.
New algorithms accelerate model-based optimization for stochastic problems.
problem Optimizing model-based stochastic optimization problems efficiently.
method Proposed new model-based algorithms with acceleration and minibatch techniques.
result Non-asymptotic convergence guarantees with linear speedup in minibatch size.
In this paper, we address the problem of embedded feature selection for ranking on top of the list problems. We pose this problem as a regularized empirical risk minimization with p-norm push loss function (p=∞) and sparsity inducing regularizers. We leverage the issues related to this challenging optimization…
New method improves robust low-rank matrix completion for computer vision.
problem Robust low-rank matrix completion for partially observed data.
method Formulated as a nonsmooth Riemannian optimization problem over Grassmann manifold, solved with an alternating manifold proximal gradient continuation method.
result Demonstrated advantages over existing approaches in background extraction from surveillance videos.
We consider the problem of minimizing the sum of a smooth function h with a bounded Hessian, and a nonsmooth function. We assume that the latter function is a composition of a proper closed function P and a surjective linear map M, with the proximal mappings of τP, τ>0, simple to compute. This problem i…
In this paper, we consider the problem of minimizing the sum of two convex functions subject to linear linking constraints. The classical alternating direction type methods usually assume that the two convex functions have relatively easy proximal mappings. However, many problems arising from statistics, image processi…
Unified view of accelerated and stochastic optimization methods.
problem Optimization challenges in machine learning and physics.
method Unified gradient flow approach to proximal algorithms and their accelerated variants.
result Unified framework for accelerated and stochastic optimization methods.
Introduces PPMM algorithm for nonconvex robust regression problems.
problem Nonconvex tuning-free robust regression problems.
method PPMM algorithm with inner subproblems solved by SSN-PPA.
result Converges to d-stationary point with KL property.
New inexact proximal gradient methods solve non-convex optimization problems.
problem Solving non-convex optimization problems with non-smooth regularization.
method Proposed three inexact proximal gradient algorithms, including basic and Nesterov's accelerated versions.
result Theoretical analysis shows convergence rates similar to exact methods.
Paper analyzes convergence of proximal algorithm in metric spaces without geodesic convexity.
problem Analyzing convergence of proximal algorithm in general metric spaces.
method Analysis of the Wasserstein proximal algorithm without geodesic convexity assumption.
result Establishes unbiased and linear convergence rate for proximal algorithm under natural Wasserstein inequality.
Two fast algorithms improve SNMF for clustering.
problem Improving clustering quality with SNMF.
method Variable splitting, APG, ADMM.
result New algorithms outperform state of the art.
Improves convex biclustering for high-dimensional data.
problem Discovering meaningful biclusters in high-dimensional data.
method Biconvex modification with adaptive feature weighting.
result Consistently recovers biclusters and selects features appropriately.
Paper tackles multivariate shape-constrained convex regression problems.
problem Fitting a convex function to data with component-wise monotonicity and uniform Lipschitz continuity.
method Least squares estimator via solving a constrained convex quadratic programming problem. Efficient algorithms designed: sGS-ADMM and pALM.
result Both proposed algorithms outperform state-of-the-art methods in numerical experiments.
Paper proposes a new method for sparse spectral clustering on Stiefel manifold.
problem Sparse spectral clustering on Stiefel manifold with nonsmooth and nonconvex objective.
method Proposes a manifold proximal linear method (ManPL) to solve the original SSC formulation.
result Demonstrates the advantage of ManPL over existing methods on single-cell RNA sequencing data.
ProxQuant improves quantized neural networks using proximal operators.
problem Making neural networks work on devices with limited resources.
method Formulates quantized network training as a regularized learning problem and optimizes it via the prox-gradient method.
result ProxQuant outperforms state-of-the-art results on binary quantization and is on par with state-of-the-art on multi-bit quantization.
New unsupervised learning technique learns independent kernels for better machine learning tasks.
problem Improving unsupervised representation learning for machine learning tasks.
method Stacking convolutional transforms using alternating proximal minimization scheme.
result DCTL outperforms shallow version CTL on benchmark datasets.
Proximal Diffusion Models improve generative model efficiency.
problem Improving generative model efficiency and accuracy.
method Developed Proximal Diffusion Models using proximal maps instead of scores.
result Proximal Diffusion Models achieve faster convergence and higher accuracy.
A new method for RLHF using proximal point Nash learning.
problem Capturing real human preferences in RLHF.
method Proximal point Nash learning, embedding self-play updates into a proximal point framework.
result High-probability last-iterate convergence for the combined method.
Proximal algorithms applied to current deformation into cycles.
problem Deformation of de Rham currents into cycles.
method Proximal algorithms, total variation denoising for differential forms.
result Calibrated cycles constructed in calibrated manifolds.
Deep neural networks improve proximal inference for causal effects.
problem Estimating causal effects in the presence of unmeasured confounders.
method Flexible deep neural network to estimate the bridge function.
result Achieves state-of-the-art performance on benchmarks.
Proximal algorithms work well for SQRT-Lasso despite its nonsmooth loss.
problem Tackles the optimization of SQRT-Lasso regression.
method Applies proximal algorithms without concern for nonsmooth loss.
result Proximal algorithms converge fast with high probability.
New PnP algorithm converges with relaxed proximal gradient descent.
problem Convergence issues in PnP methods with deep denoisers.
method Relaxed proximal gradient descent for PnP with weakly convex regularization.
result Proposed PnP-αPGD converges for a wider range of regularization parameters. Improved bounds for proximal gradient algorithms with computational errors.
problem Analyzing convergence of proximal gradient algorithms with inaccuracies.
method Deriving new tighter deterministic and probabilistic bounds for convex composite problems.
result Probabilistic bounds are more robust and accurate for algorithm verification and performance guarantees.
Paper proposes a single task optimization for endmembers' number estimation and unmixing.
problem Endmembers' number estimation and unmixing in hyperspectral images.
method Low-rank and sparse nonnegative matrix factorization with alternating proximal algorithm.
result Effectiveness of the proposed approach verified by experiments.