Accelerates sampling from Gibbs distributions using ARWP method.
problem Sampling from Gibbs distributions efficiently.
method ARWP method, combining Nesterov acceleration and regularized Wasserstein proximal.
result ARWP exhibits higher contraction rate and faster tail exploration.
New inexact proximal gradient methods solve non-convex optimization problems.
problem Solving non-convex optimization problems with non-smooth regularization.
method Proposed three inexact proximal gradient algorithms, including basic and Nesterov's accelerated versions.
result Theoretical analysis shows convergence rates similar to exact methods.
Improved quasi-Newton method for convex optimization with linear and accelerated convergence.
problem Efficiently solving composite optimization problems with strong convexity.
method Proximal quasi-Newton algorithm with accelerated variant.
result Acceleration may not improve convergence in quasi-Newton setting.
Stochastic proximal point algorithm with momentum converges faster and is more stable than standard methods.
problem Improving convergence and stability of stochastic optimization methods.
method Developed and analyzed the convergence and stability of the stochastic proximal point algorithm with momentum (SPPAM).
result SPPAM converges faster and is more stable than standard stochastic proximal point algorithm (SPPA) and stochastic gradient descent with momentum (SGDM).
Accelerates stochastic optimization for convex and strongly convex problems.
problem Improving convergence rates in noisy stochastic optimization.
method Extends Catalyst approach to stochastic settings, handles inexact proximal operators.
result Achieves optimal worst-case complexity for noise-dominated regions.
Catalyst accelerates convex optimization methods with practical guidelines.
problem Improving convergence of gradient-based optimization methods.
method Catalyst scheme based on inexact accelerated proximal point algorithm.
result Faster convergence rates for various convex optimization methods.
The paper accelerates ISTA and FISTA algorithms for composite optimization problems.
problem Improving convergence rates of ISTA and FISTA for composite optimization.
method Improved proximal subgradient norm minimization using Lyapunov function.
result Convergence rates of ISTA and FISTA are accelerated.
Paper develops accelerated APCD for nonconvex nonsmooth problems with performance guarantees.
problem Efficient methods for nonconvex nonsmooth optimization problems with performance guarantees.
method Asynchronous Accelerated Proximal Coordinate Descent (AAPCD) for nonsmooth and nonconvex problems.
result AAPCD ensures that every limit point is a critical point and achieves linear and sublinear convergence rates.
New algorithms accelerate model-based optimization for stochastic problems.
problem Optimizing model-based stochastic optimization problems efficiently.
method Proposed new model-based algorithms with acceleration and minibatch techniques.
result Non-asymptotic convergence guarantees with linear speedup in minibatch size.
Proposes an accelerated optimization algorithm for composite objectives.
problem Gradient-based optimization with sparse solutions and high dimensions.
method Inexact variable-metric proximal point algorithm (QNing) with limited-memory BFGS.
result Significant improvements over competing methods in training machine learning models.
Inertial proximal gradient algorithm shows monotonically decreasing values.
problem Optimizing functions with inertia.
method Proximal gradient algorithm with alternated inertia.
result Algorithm with alternated inertia achieves monotonically decreasing functional values.
Unified view of accelerated and stochastic optimization methods.
problem Optimization challenges in machine learning and physics.
method Unified gradient flow approach to proximal algorithms and their accelerated variants.
result Unified framework for accelerated and stochastic optimization methods.
Paper introduces a new reinforcement learning method with improved performance.
problem Designing and analyzing efficient reinforcement learning algorithms.
method Proximal gradient temporal difference learning (GTD) with accelerated algorithm GTD2-MP.
result GTD algorithms have linear complexity and improved convergence rate.
This paper accelerates TV regularization algorithms by unrolling proximal gradient descent.
problem Solving Total Variation (TV) regularized problems with iterative algorithms.
method Unrolling proximal gradient descent solvers to learn their parameters.
result Two approaches to compute derivatives through proximal operators improve performance.
Improved bounds for proximal gradient algorithms with computational errors.
problem Analyzing convergence of proximal gradient algorithms with inaccuracies.
method Deriving new tighter deterministic and probabilistic bounds for convex composite problems.
result Probabilistic bounds are more robust and accurate for algorithm verification and performance guarantees.
We analyze Riemannian accelerated methods using a new framework.
problem Understanding Riemannian accelerated gradient methods.
method Riemannian A-HPE framework, focusing on Euclidean A-HPE insights and metric distortion control.
result Characterization of acceleration for various Riemannian methods.
New method accelerates Bayesian imaging using Langevin sampling.
problem Bayesian inference in imaging inverse problems with convex geometry.
method Stochastic relaxed proximal-point iteration targeting posterior distribution.
result Accelerated convergence for κ-strongly log-concave targets. In this paper we develop proximal methods for statistical learning. Proximal point algorithms are useful in statistics and machine learning for obtaining optimization solutions for composite functions. Our approach exploits closed-form solutions of proximal operators and envelope representations based on the Moreau, Fo…
In this paper, we propose a new algorithm to speed-up the convergence of accelerated proximal gradient (APG) methods. In order to minimize a convex function f(x), our algorithm introduces a simple line search step after each proximal gradient step in APG so that a biconvex function f(θx) is minimi…
A novel BMC model with nonconvex regularizers and accelerated proximal algorithm for binary matrix completion.
problem Recovering a binary matrix from partial observed positive elements.
method Proposes a novel BMC model with nonconvex regularizers and accelerates proximal algorithm for solving the nonconvex optimization problem.
result The proposed model and algorithm outperform other methods in both synthetic and real-world data sets.
We consider a regularized least squares problem, with regularization by structured sparsity-inducing norms, which extend the usual ℓ1 and the group lasso penalty, by allowing the subsets to overlap. Such regularizations lead to nonsmooth problems that are difficult to optimize, and we propose in this paper a suit…
New algorithm accelerates single-pass SGD for generalized linear prediction.
problem Improving single-pass non-quadratic stochastic optimization.
method Data-dependent proximal method incorporating dual-momentum acceleration.
result Momentum acceleration resolves open problem in streaming setting.
New method speeds up optimization for complex functions.
problem Optimizing complex functions with stochastic composition.
method Accelerated stochastic compositional proximal gradient (ASC-PG) method.
result ASC-PG achieves faster convergence and optimal sample-error complexity.
Physics insights into optimization algorithms using differential equations.
problem Understanding dynamics of optimization algorithms in machine learning.
method Unified framework based on physical systems analysis of popular optimization algorithms.
result Unified analysis applicable to non-convex and non-strongly convex problems.
We develop a family of accelerated stochastic algorithms that minimize sums of convex functions. Our algorithms improve upon the fastest running time for empirical risk minimization (ERM), and in particular linear least-squares regression, across a wide range of problem settings. To achieve this, we establish a framewo…
New method solves non-strongly convex optimization problems without quadratic regularization.
problem Minimizing the sum of an average of smooth convex components and a non-differentiable convex function.
method Accelerated randomized mirror descent algorithm without strongly convex assumption.
result Performance of algorithms improved without quadratic regularization.
RFX accelerates and compresses Random Forests for large datasets.
problem Memory bottleneck in proximity matrices limits Random Forest analysis.
method QLORA compression, CPU TriBlock storage, GPU batch sizing, 3D MDS visualization.
result Proximity-based Random Forest analysis on larger datasets is feasible.
A new optimization method, BPM, converges linearly in non-convex, non-smooth problems.
problem Non-smooth and non-convex optimization challenges.
method Ball-Proximal Point Method (BPM), inspired by Proximal Point Method (PPM).
result BPM converges linearly and in a finite number of steps in non-convex, non-smooth problems.
Paper analyzes why LISTA accelerates sparse coding.
problem Sparse coding optimization with generic methods.
method Matrix factorization of Gram kernel for acceleration.
result Improved convergence for LISTA compared to non-adaptive methods.
Accelerates coordinate descent methods for machine learning problems.
problem Slowness of coordinate descent methods in machine learning.
method Extrapolation-based accelerated coordinate descent.
result Significant speed-up in practice compared to existing methods.
New insights show NAG and FISTA converge linearly without knowing strong convexity modulus.
problem Understanding linear convergence of NAG and FISTA without strong convexity modulus knowledge.
method High-resolution ODE framework, dynamically adapting kinetic energy coefficient.
result NAG and FISTA demonstrate linear convergence without requiring strong convexity modulus knowledge.
The paper analyzes why LISTA accelerates sparse coding.
problem Sparse coding optimization using generic methods is slow.
method The paper studies LISTA, a neural network-accelerated sparse coding method.
result The acceleration is due to a specific matrix factorization of the Gram kernel.
New method accelerates optimization for nonconvex problems in statistics.
problem Nonconvex optimization problems in statistics and machine learning.
method Adaptive momentum and block coordinate proximal gradient method with specific update rules.
result The method achieves provable local linear convergence in sparse linear regression problems.
We introduce a proximal version of the stochastic dual coordinate ascent method and show how to accelerate the method using an inner-outer iteration procedure. We analyze the runtime of the framework and obtain rates that improve state-of-the-art results for various key machine learning optimization problems including …
New algorithm speeds up solving saddle-point problems with large condition numbers.
problem Solving saddle-point problems with large condition numbers.
method Proposes a stochastic proximal point algorithm that accelerates variance reduction methods.
result Reduces logarithmic term of condition number for iteration complexity.
This paper analyzes and improves monotonic accelerated algorithms like M-NAG and M-FISTA.
problem Establishing linear convergence of M-NAG and M-FISTA under strong convexity.
method Lyapunov analysis and modified Lyapunov functions.
result Linear convergence of M-NAG and M-FISTA is guaranteed without full NAG iterates.
In this paper we propose a primal-dual proximal extragradient algorithm to solve the generalized Dantzig selector (GDS) estimation problem, based on a new convex-concave saddle-point (SP) reformulation. Our new formulation makes it possible to adopt recent developments in saddle-point optimization, to achieve the optim…
ASVRG accelerates stochastic variance reduction methods with simplicity and efficiency.
problem Efficiently solving convex and non-convex optimization problems.
method Accelerated proximal stochastic variance reduced gradient (ASVRG) method with momentum acceleration.
result ASVRG achieves best known oracle complexities for strongly and non-strongly convex objectives.
New algorithm solves complex medical radiation therapy problems.
problem Optimizing radiation therapy treatment plans.
method Majorization-minimization principle applied to non-linear split feasibility problems.
result Euclidean norm in proximity function replaced by Bregman divergences.
Paper solves minimax optimization gap with near-optimal algorithms.
problem Designing efficient algorithms for smooth and strongly-convex-strongly-concave minimax problems.
method Accelerated proximal point method and accelerated solver for minimax proximal steps.
result First algorithm with gradient complexity matching the lower bound up to logarithmic factors.
We develop a projected Nesterov's proximal-gradient (PNPG) approach for sparse signal reconstruction that combines adaptive step size with Nesterov's momentum acceleration. The objective function that we wish to minimize is the sum of a convex differentiable data-fidelity (negative log-likelihood (NLL)) term and a conv…
Fault-tolerant federated learning for non-uniform data.
problem Faulty workers corrupting data in federated learning.
method Fault-resilient proximal gradient (FRPG) algorithm with Nesterov's acceleration and local FRPG for reduced communication.
result FRPG and LFRPG converge faster than robust stochastic aggregation.
Algorithm solves robust linear regression with block Lewis weights.
problem Group distributionally robust least squares problem.
method Algorithm based on geometric construction and block Lewis weights, using accelerated proximal methods.
result Improves over known methods for moderate accuracy regimes and matches state-of-the-art guarantees.
New method for efficient matrix completion with nonignorable missing data.
problem Nonignorable missing data in matrix completion.
method Nuclear norm regularized U-statistic loss function and accelerated proximal gradient algorithm.
result Near minimax optimal statistical convergence rate for nonignorable missing data.
New algorithms optimize convex functions with high-order derivatives.
problem Optimizing convex functions with high-order derivatives under various norms.
method Developed a non-Euclidean inexact accelerated proximal point method using an inexact uniformly convex regularizer.
result Showed nearly optimal algorithms for high dimensions in the black-box oracle model for ℓp-settings and all q≥1. New methods for convex optimization with locally Lipschitz gradient, achieving faster convergence.
problem Optimization problems with locally Lipschitz continuous gradient.
method Accelerated proximal gradient (APG) methods and proximal augmented Lagrangian method.
result Achieved faster convergence rates for convex optimization problems with locally Lipschitz gradient.
Paper tackles NAS problem by modeling it as a sparse supernet.
problem Neural Architecture Search (NAS) problem, particularly Mixed-Path Search.
method Model NAS as a sparse supernet with sparsity constraints. Use hierarchical accelerated proximal gradient algorithm for optimization.
result Proposed method finds compact, general, and powerful neural architectures.
Develops a new OT framework for class-based data with improved robustness.
problem Understand and recover class structure in optimal transport schemes.
method Proposes a convex OT program with sum-of-norms regularization and an accelerated proximal algorithm.
result The new regularizer preserves class structure better and is more robust to data geometry.