In this paper, we investigate the attractive properties of the proximal gradient algorithm with inertia. Notably, we show that using alternated inertia yields monotonically decreasing functional values, which contrasts with usual accelerated proximal gradient methods. We also provide convergence rates for the algorithm…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “proximal alternating algorithm”
We consider a proximal operator given by a quadratic function subject to bound constraints and give an optimization algorithm using the alternating direction method of multipliers (ADMM). The algorithm is particularly efficient to solve a collection of proximal operators that share the same quadratic form, or if the qu…
New method uses zeroth-order queries to approximate proximal sampling efficiently.
problem Approximating proximal sampling with zeroth-order information.
method Direct simulation of heat flow dynamics, treating intermediate distribution as Gaussian mixture.
result Inherits exponential convergence under isoperimetric conditions, avoids rejection sampling.
The paper analyzes convergence properties of NGA and PAMe for -norm PCA.
problem Finite-step convergence of -norm PCA algorithms.
method Conditional subgradient and alternating maximization interpretations of NGA, and PAMe with extrapolation.
result Iterative points of modified NGA and PAMe remain constant after finitely many steps under certain conditions.
New sampling algorithm for non-smooth potentials.
problem Sampling from non-smooth potentials.
method Proximal algorithm based on rejection sampling.
result Achieves better complexity than existing methods.
Revisits PPO design choices, exposing failure modes and proposing alternatives.
problem Failure modes of standard PPO in new environments.
method Revisits standard PPO design choices, exposes failure modes, and proposes alternative approaches.
result Alternative design choices prevent failure modes in new environments.
Sparse principal component analysis (PCA) and sparse canonical correlation analysis (CCA) are two essential techniques from high-dimensional statistics and machine learning for analyzing large-scale data. Both problems can be formulated as an optimization problem with nonsmooth objective and nonconvex constraints. Sinc…
The Alternating Direction Method of Multipliers (ADMM) has been studied for years. The traditional ADMM algorithm needs to compute, at each iteration, an (empirical) expected loss function on all training examples, resulting in a computational complexity proportional to the number of training examples. To reduce the ti…
Large sectors of the recent optimization literature focused in the last decade on the development of optimal stochastic first order schemes for constrained convex models under progressively relaxed assumptions. Stochastic proximal point is an iterative scheme born from the adaptation of proximal point algorithm to nois…
Efficient algorithms solve joint graphical lasso problems.
problem Learning graphical models from sparse data.
method Proximal gradient procedures with ADMM backtracking option.
result Proposed algorithms achieve high accuracy and precision.
A new algorithm speeds up convex clustering.
problem Optimizing clustering with convex optimization and avoiding local minima.
method Smoothing proximal gradient algorithm (Sproga) for convex clustering.
result Sproga is faster and uses less memory than existing methods.
Paper proposes a new method for supervised manifold learning using random forest proximities.
problem Existing supervised manifold learning methods fail to uncover meaningful embeddings due to using class-conditional distances.
method Proposes a data-geometry-preserving variant of random forest proximities as an initialization for manifold learning methods.
result Local and global structure preservation is near universal across manifold learning approaches using diffusion-based algorithms.
This paper converts ADMM to proximal gradient for efficient sparse estimation.
problem Sparse estimation problems like fused lasso and convex clustering.
method General method converting ADMM to proximal gradient, assuming Lipschitz continuity of derivative.
result Significant improvement in efficiency for sparse estimation problems.
New model approximates sparse mean-CVaR portfolio optimization efficiently.
problem NP-hard -constrained mean-CVaR optimization.
method Proximal alternating linearized minimization algorithm with nested fixed-point proximity.
result The model offers a guaranteed approximation of the -constrained mean-CVaR model.
Matrix Factorization is a popular non-convex optimization problem, for which alternating minimization schemes are mostly used. They usually suffer from the major drawback that the solution is biased towards one of the optimization variables. A remedy is non-alternating schemes. However, due to a lack of Lipschitz conti…
The (global) Lipschitz smoothness condition is crucial in establishing the convergence theory for most optimization methods. Unfortunately, most machine learning and signal processing problems are not Lipschitz smooth. This motivates us to generalize the concept of Lipschitz smoothness condition to the relative smoothn…
New algorithm solves minimax games with linear constraints.
problem Nonconvex minimax games with coupled linear constraints.
method Primal-dual alternating proximal gradient (PDAPG) algorithm.
result Achieves ε-stationary solution within O(ε^(-2)) iterations for strongly concave settings.
We propose a new proximal, path-following framework for a class of constrained convex problems. We consider settings where the nonlinear---and possibly non-smooth---objective part is endowed with a proximity operator, and the constraint set is equipped with a self-concordant barrier. Our approach relies on the followin…
Derivative-free Alternating Projection Algorithms for General Nonconvex-Concave Minimax Problemsmath.OC
New algorithms solve complex minimax problems without needing derivatives.
problem Solving nonconvex-concave minimax problems efficiently.
method Zeroth-order alternating and proximal gradient algorithms.
result Iteration complexity and function value estimation bounds established.
Complex embeddings handle non-metric proximity data better than traditional methods.
problem Proximities not always metric or inner product-based, causing convergence issues.
method Proposes complex-valued embeddings for non-vectorial data.
result Complex embeddings outperform traditional techniques on benchmarks.
Paper proposes a new method to separate low rank and sparse matrices without bias.
problem Recovering low rank and sparse matrices from measurements.
method Uses nonconvex regularizers and alternating proximal gradient descent.
result Error bounds for the algorithm applied to sparse optimization, matrix completion, and robust PCA.
Chandrasekaran, Parrilo and Willsky (2010) proposed a convex optimization problem to characterize graphical model selection in the presence of unobserved variables. This convex optimization problem aims to estimate an inverse covariance matrix that can be decomposed into a sparse matrix minus a low-rank matrix from sam…
New algorithms accelerate model-based optimization for stochastic problems.
problem Optimizing model-based stochastic optimization problems efficiently.
method Proposed new model-based algorithms with acceleration and minibatch techniques.
result Non-asymptotic convergence guarantees with linear speedup in minibatch size.
In this paper, we address the problem of embedded feature selection for ranking on top of the list problems. We pose this problem as a regularized empirical risk minimization with -norm push loss function () and sparsity inducing regularizers. We leverage the issues related to this challenging optimization…
Robust Low-rank Matrix Completion via an Alternating Manifold Proximal Gradient Continuation Methodcs.LG
New method improves robust low-rank matrix completion for computer vision.
problem Robust low-rank matrix completion for partially observed data.
method Formulated as a nonsmooth Riemannian optimization problem over Grassmann manifold, solved with an alternating manifold proximal gradient continuation method.
result Demonstrated advantages over existing approaches in background extraction from surveillance videos.
We consider the problem of minimizing the sum of a smooth function with a bounded Hessian, and a nonsmooth function. We assume that the latter function is a composition of a proper closed function and a surjective linear map , with the proximal mappings of , , simple to compute. This problem i…
In this paper, we consider the problem of minimizing the sum of two convex functions subject to linear linking constraints. The classical alternating direction type methods usually assume that the two convex functions have relatively easy proximal mappings. However, many problems arising from statistics, image processi…
Introduces PPMM algorithm for nonconvex robust regression problems.
problem Nonconvex tuning-free robust regression problems.
method PPMM algorithm with inner subproblems solved by SSN-PPA.
result Converges to d-stationary point with KL property.
In machine learning research, the proximal gradient methods are popular for solving various optimization problems with non-smooth regularization. Inexact proximal gradient methods are extremely important when exactly solving the proximal operator is time-consuming, or the proximal operator does not have an analytic sol…
Paper analyzes convergence of proximal algorithm in metric spaces without geodesic convexity.
problem Analyzing convergence of proximal algorithm in general metric spaces.
method Analysis of the Wasserstein proximal algorithm without geodesic convexity assumption.
result Establishes unbiased and linear convergence rate for proximal algorithm under natural Wasserstein inequality.
In this paper we propose a primal-dual proximal extragradient algorithm to solve the generalized Dantzig selector (GDS) estimation problem, based on a new convex-concave saddle-point (SP) reformulation. Our new formulation makes it possible to adopt recent developments in saddle-point optimization, to achieve the optim…
Biconvex Biclusteringstat.ML
Improves convex biclustering for high-dimensional data.
problem Discovering meaningful biclusters in high-dimensional data.
method Biconvex modification with adaptive feature weighting.
result Consistently recovers biclusters and selects features appropriately.
As the most successful variant and improvement for Trust Region Policy Optimization (TRPO), proximal policy optimization (PPO) has been widely applied across various domains with several advantages: efficient data utilization, easy implementation, and good parallelism. In this paper, a first-order gradient reinforcemen…
Paper tackles multivariate shape-constrained convex regression problems.
problem Fitting a convex function to data with component-wise monotonicity and uniform Lipschitz continuity.
method Least squares estimator via solving a constrained convex quadratic programming problem. Efficient algorithms designed: sGS-ADMM and pALM.
result Both proposed algorithms outperform state-of-the-art methods in numerical experiments.
Paper proposes a new method for sparse spectral clustering on Stiefel manifold.
problem Sparse spectral clustering on Stiefel manifold with nonsmooth and nonconvex objective.
method Proposes a manifold proximal linear method (ManPL) to solve the original SSC formulation.
result Demonstrates the advantage of ManPL over existing methods on single-cell RNA sequencing data.
Proximal algorithms applied to current deformation into cycles.
problem Deformation of de Rham currents into cycles.
method Proximal algorithms, total variation denoising for differential forms.
result Calibrated cycles constructed in calibrated manifolds.
A relaxed proximal gradient descent algorithm for convergent plug-and-play with proximal denoiserstat.ML
New PnP algorithm converges with relaxed proximal gradient descent.
problem Convergence issues in PnP methods with deep denoisers.
method Relaxed proximal gradient descent for PnP with weakly convex regularization.
result Proposed PnP-PGD converges for a wider range of regularization parameters.
Improved bounds for proximal gradient algorithms with computational errors.
problem Analyzing convergence of proximal gradient algorithms with inaccuracies.
method Deriving new tighter deterministic and probabilistic bounds for convex composite problems.
result Probabilistic bounds are more robust and accurate for algorithm verification and performance guarantees.
Proximal Diffusion Models improve generative model efficiency.
problem Improving generative model efficiency and accuracy.
method Developed Proximal Diffusion Models using proximal maps instead of scores.
result Proximal Diffusion Models achieve faster convergence and higher accuracy.
New unsupervised learning technique learns independent kernels for better machine learning tasks.
problem Improving unsupervised representation learning for machine learning tasks.
method Stacking convolutional transforms using alternating proximal minimization scheme.
result DCTL outperforms shallow version CTL on benchmark datasets.
Deep neural networks improve proximal inference for causal effects.
problem Estimating causal effects in the presence of unmeasured confounders.
method Flexible deep neural network to estimate the bridge function.
result Achieves state-of-the-art performance on benchmarks.
A new method for RLHF using proximal point Nash learning.
problem Capturing real human preferences in RLHF.
method Proximal point Nash learning, embedding self-play updates into a proximal point framework.
result High-probability last-iterate convergence for the combined method.
Optimization is at the heart of machine learning, statistics and many applied scientific disciplines. It also has a long history in physics, ranging from the minimal action principle to finding ground states of disordered systems such as spin glasses. Proximal algorithms form a class of methods that are broadly applica…
The use of convex regularizers allows for easy optimization, though they often produce biased estimation and inferior prediction performance. Recently, nonconvex regularizers have attracted a lot of attention and outperformed convex ones. However, the resultant optimization problem is much harder. In this paper, for a …
Stochastic version of proximal distance algorithm analyzed and validated.
problem Optimization of constrained estimation problems.
method Stochastic proximal distance algorithm, with convergence guarantees and finite error bounds.
result Convergence guarantees and finite error bounds for the first time.
Online algorithm identifies PDEs from noisy data snapshots.
problem Identifying PDEs from sequential solution snapshots.
method Combines weak-form discretization with online proximal gradient descent.
result Efficiently identifies and tracks systems with time-varying coefficients.
New transport method simplifies cutoff phenomenon for Markov processes.
problem Understanding the cutoff phenomenon for Markov processes.
method A new W-TV transport inequality combined with a parabolic regularization estimate.
result Recovery and extension of previous results on cutoff phenomena.
In this paper we develop proximal methods for statistical learning. Proximal point algorithms are useful in statistics and machine learning for obtaining optimization solutions for composite functions. Our approach exploits closed-form solutions of proximal operators and envelope representations based on the Moreau, Fo…