We introduce a generic scheme for accelerating gradient-based optimization methods in the sense of Nesterov. The approach, called Catalyst, builds upon the inexact accelerated proximal point algorithm for minimizing a convex objective function, and consists of approximately solving a sequence of well-chosen auxiliary p…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “Catalyst acceleration”
Accelerates stochastic optimization for convex and strongly convex problems.
problem Improving convergence rates in noisy stochastic optimization.
method Extends Catalyst approach to stochastic settings, handles inexact proximal operators.
result Achieves optimal worst-case complexity for noise-dominated regions.
Catalyst.RL accelerates RL research with efficient training.
problem Efficient reinforcement learning training in complex environments.
method Open-source PyTorch framework with distributed training and RL algorithms.
result Catalyst.RL achieved 2nd place in a computationally expensive RL challenge.
A Unifying Framework for Variance Reduction Algorithms for Finding Zeroes of Monotone Operatorsstat.ML
Unified framework for variance reduction to solve monotone operator problems.
problem Large-scale monotone inclusion problems with finite sum structure.
method Developed a general framework for variance-reduced forward-backward splitting algorithms.
result Linear convergence rate under mild assumptions, with Catalyst acceleration and asynchronous implementation.
A new method for distributed optimization reduces communication rounds without minibatches.
problem Efficient training in distributed machine learning with different data distributions.
method A primal-dual method (GA-MSGD) applied to the Lagrangian of distributed optimization.
result Achieves linear convergence in communication rounds for strongly convex objectives.
We introduce a generic scheme to solve nonconvex optimization problems using gradient-based algorithms originally designed for minimizing convex functions. Even though these methods may originally require convexity to operate, the proposed approach allows one to use them on weakly convex objectives, which covers a larg…
New method uses machine learning to analyze catalyst reactions.
problem Understanding complex reaction mechanisms in catalytic materials.
method Combining transient kinetics and machine learning.
result Correct estimates of micro-kinetic coefficients and mechanism.
We propose a generic framework based on a new stochastic variance-reduced gradient descent algorithm for accelerating nonconvex low-rank matrix recovery. Starting from an appropriate initial estimator, our proposed algorithm performs projected gradient descent based on a novel semi-stochastic gradient specifically desi…
New algorithm speeds up solving saddle-point problems with large condition numbers.
problem Solving saddle-point problems with large condition numbers.
method Proposes a stochastic proximal point algorithm that accelerates variance reduction methods.
result Reduces logarithmic term of condition number for iteration complexity.
Despite the recent progress in deep reinforcement learning field (RL), and, arguably because of it, a large body of work remains to be done in reproducing and carefully comparing different RL algorithms. We present catalyst.RL, an open source framework for RL research with a focus on reproducibility and flexibility. Ma…
Large GNNs trained with Graph Parallelism improve atomic simulation accuracy.
problem Training memory-intensive GNNs for high-order atomic interactions.
method Graph Parallelism to distribute graphs across GPUs.
result Graph-parallelized models achieve state-of-the-art results on catalyst datasets.
Catalyst design using actively learned machine with non-ab initio input features towards CO2 reduction reactionscond-mat.mtrl-sci
In conventional chemisorption model, the d-band center theory (augmented sometimes with the upper edge of d-band for imporved accuarcy) plays a central role in predicting adsorption energies and catalytic activity as a function of d-band center of the solid surfaces, but it requires density functional calculations that…
Noisy, sparse, nonlinear: Navigating the Bermuda Triangle of physical inference with deep filteringphysics.chem-ph
Deep filtering improves robustness of models from noisy, sparse data.
problem Challenges in understanding microscopic interactions from noisy, sparse, and biased data.
method Statistical approach based on deep filtering of nonlinear feature networks.
result Physicochemical models are more robust, transparent, and generalize better.
Two new algorithms improve federated optimization under second-order similarity.
problem Federated learning under communication constraints and second-order similarity.
method SVRP and Catalyzed SVRP algorithms combining proximal point evaluations, client sampling, and variance reduction.
result Achieves superior performance and uniformly improves upon existing algorithms for federated optimization under second-order similarity and strong convexity.
Gryffin optimizes categorical variables in materials design, leveraging expert knowledge.
problem Optimizing categorical variables in complex design choices like molecule selection.
method Bayesian optimization with smooth approximations to categorical distributions, incorporating expert knowledge.
result Gryffin accelerates discovery of promising molecules and materials, highlighting relevant correlations.
Polynomial chaos surrogates quantify epistemic uncertainty in AI-driven scientific models.
problem Uncertainty in reward estimates hinders interpretability in sequential generative models.
method Fit polynomial chaos expansions to trained models to propagate epistemic uncertainty and quantify sensitivity.
result Interpretable decomposition of reward components driving generative decisions.
The paper studies geometric properties of -harmonic maps and proves Liouville type results.
problem Exploring geometric properties of -harmonic maps.
method Unified geometric analytic methods, first and second variation formulas, stress-energy tensor, conservation law, monotonicity formula, asymptotic assumption, extrinsic average variational method.
result Proves Liouville type results for -harmonic maps.
Company mergers and acquisitions are often perceived to act as catalysts for corporate growth in free markets systems: it is conventional wisdom that those activities lead to better and more efficient markets. However, the broad adoption of this perception into corporate strategy is prone to result in a less diverse an…
Precision farming uses data analysis to optimize crop management.
problem Meeting increased food demand with sustainable practices.
method Statistical tools and machine learning for pattern recognition and prediction.
result Optimized farming processes in real-time reduce resource use.
Recently, locality sensitive hashing (LSH) was shown to be effective for MIPS and several algorithms including -ALSH, Sign-ALSH and Simple-LSH have been proposed. In this paper, we introduce the norm-range partition technique, which partitions the original dataset into sub-datasets containing items with similar 2-…
Accelerates Riemannian gradient methods with extrapolation.
problem Optimizing functions on manifolds efficiently.
method Extrapolating iterates in Riemannian gradient descent.
result Achieves optimal convergence rate and computational advantage.
Maximal acceleration metrics limit spacetime curvature.
problem Bounding spacetime curvature under maximal acceleration.
method Developed a geometric framework for maximal acceleration metrics and associated connections, proving curvature bounds.
result Uniform bounds on curvature components follow from uniform bounds on maximal acceleration.
Study shows activist board representation improves Japanese companies' performance.
problem Lack of innovation and improvement in Japanese companies.
method Examined two Japanese companies with activist board representation, analyzing performance metrics.
result Companies with activist board representation experienced significant improvements in stock returns and operational metrics.
Accelerates optimization in asynchronous systems with sparse updates.
problem Optimizing finite-sum objectives in asynchronous lock-free environments.
method New accelerated SVRG variant with sparse updates.
result Achieves optimal incremental gradient complexity.
We analyze Riemannian accelerated methods using a new framework.
problem Understanding Riemannian accelerated gradient methods.
method Riemannian A-HPE framework, focusing on Euclidean A-HPE insights and metric distortion control.
result Characterization of acceleration for various Riemannian methods.
Super-acceleration of gradient descent with momentum improves loss function minimization.
problem Minimizing loss functions in machine learning.
method Extending Nesterov acceleration by using gradients at multiple steps ahead.
result Super-acceleration of the momentum algorithm is beneficial for various loss landscapes and tasks.
PF-LaCG removes the need for knowing smoothness and strong convexity parameters for locally accelerated CG.
problem Locally accelerated CG requires knowledge of smoothness and strong convexity parameters.
method Parameter-Free Locally Accelerated CG (PF-LaCG) algorithm.
result PF-LaCG achieves local acceleration without requiring knowledge of smoothness and strong convexity parameters.
Accelerates coordinate descent methods for machine learning problems.
problem Slowness of coordinate descent methods in machine learning.
method Extrapolation-based accelerated coordinate descent.
result Significant speed-up in practice compared to existing methods.
Acceleration in Hilbert spaces reduces computations but not accuracy.
problem Improving learning accuracy with fewer computations.
method Analysis of Nesterov acceleration and heavy-ball methods in Hilbert spaces.
result Acceleration can reduce computations but not improve accuracy with respect to gradient descent.
A Continuized View on Nesterov Acceleration for Stochastic Gradient Descent and Randomized Gossipmath.OC
Continuized Nesterov acceleration accelerates stochastic gradient descent and gossip algorithms.
problem Improving the convergence rate of stochastic gradient descent and gossip algorithms.
method Introducing a continuized variant of Nesterov acceleration, which mixes variables continuously and takes gradient steps at random times.
result The continuized Nesterov acceleration achieves convergence rates similar to Nesterov's original acceleration but with random parameters.
Accelerated gradient methods play a central role in optimization, achieving optimal rates in many settings. While many generalizations and extensions of Nesterov's original acceleration method have been proposed, it is not yet clear what is the natural scope of the acceleration concept. In this paper, we study accelera…
Develops accelerated methods for optimization using low-dimensional projected-gradient information.
problem Optimization with low-dimensional projected-gradient information and Nesterov acceleration.
method Randomized-subspace Nesterov accelerated gradient methods for smooth convex and strongly convex optimization.
result Established accelerated oracle-complexity guarantees and unified basis for comparing sketch families.
Accelerates sampling from Gibbs distributions using ARWP method.
problem Sampling from Gibbs distributions efficiently.
method ARWP method, combining Nesterov acceleration and regularized Wasserstein proximal.
result ARWP exhibits higher contraction rate and faster tail exploration.
FedAc accelerates Federated Averaging for distributed optimization.
problem Efficiently optimizing distributed machine learning models.
method Federated Accelerated Stochastic Gradient Descent (FedAc) using a potential-based perturbed iterate analysis.
result FedAc achieves faster convergence and lower communication costs than previous methods.
This research accelerates sampling methods using Nesterov's Acceleration.
problem Improving sampling efficiency in MCMC methods.
method Developed a Hessian-Free High-Resolution ODE reformulation of NAG-SC, injected noise, and discretized the diffusion process.
result Quantified acceleration beyond underdamped Langevin in distance for log-strongly-concave targets.
Locally Accelerated Conditional Gradients improve convergence rates for smooth convex optimization problems.
problem Achieving optimal convergence rates for smooth convex optimization problems over polytopes.
method Locally Accelerated Conditional Gradients, coupling accelerated steps with conditional gradient steps.
result Achieves optimal accelerated local convergence for smooth strongly convex problems.
This paper studies accelerations in Q-learning algorithms. We propose an accelerated target update scheme by incorporating the historical iterates of Q functions. The idea is conceptually inspired by the momentum-based accelerated methods in the optimization theory. Conditions under which the proposed accelerated algor…
Study detects endogenous bubbles in meme stocks using CI.
problem Detecting endogenous bubbles in meme stocks.
method Used Log-Periodic Power Law (LPPL) Confidence Indicator (CI).
result CI detected numerous bubbles in meme stocks but struggled with predicting exogenous rallies.
In this study, the concept of dual Lorentzian homotetic exponential motions in is discussed and their velocities, accelerations obtained. Also, some geometric results between velocity and acceleration vectors of a point in a spatial motion are obtained. Finally, the theorems related to acceleration and acceleration cen…
Improved Analysis of the Accelerated Noisy Power Method with Applications to Decentralized PCAstat.ML
Improved analysis of accelerated noisy power method for PCA.
problem Inexact matrix-vector products in PCA settings.
method Improved analysis of Accelerated Noisy Power Method under milder perturbation conditions.
result Worst-case optimal convergence rate with relaxed noise conditions.
Variance reduction is a simple and effective technique that accelerates convex (or non-convex) stochastic optimization. Among existing variance reduction methods, SVRG and SAGA adopt unbiased gradient estimators and are the most popular variance reduction methods in recent years. Although various accelerated variants o…
Hamiltonian dynamics-based algorithms achieve deterministic and accelerated convergence for convex optimization.
problem Accelerating convex optimization
method Hamiltonian dynamics
result Hamiltonian dynamics-based algorithms achieve deterministic and accelerated convergence for convex optimization.
Accelerates MMLE using SVGD with Nesterov acceleration.
problem Maximum Marginal Likelihood Estimation optimization.
method Stein variational gradient descent with Nesterov acceleration.
result Consistently accelerates convergence across various tasks.
Many applications require that we learn the parameters of a model from data. EM is a method used to learn the parameters of probabilistic models for which the data for some of the variables in the models is either missing or hidden. There are instances in which this method is slow to converge. Therefore, several accele…
We propose a novel method to accelerate Lloyd's algorithm for K-Means clustering. Unlike previous acceleration approaches that reduce computational cost per iterations or improve initialization, our approach is focused on reducing the number of iterations required for convergence. This is achieved by treating the assig…
We study optimal buying and selling strategies in target zone models. In these models the price is modeled by a diffusion process which is reflected at one or more barriers. Such models arise for example when a currency exchange rate is kept above a certain threshold due to central bank intervention. We consider the op…
Hamiltonian Descent Algorithms for Optimization: Accelerated Rates via Randomized Integration Timemath.OC
HF-opt uses Hamiltonian dynamics to optimize functions, achieving accelerated rates with randomized integration time.
problem Optimizing functions efficiently and accelerating convergence rates.
method Randomized Hamiltonian flow (RHF) with accelerated convergence rates.
result RHGD achieves accelerated convergence rates similar to Nesterov's AGD.
AGNES accelerates gradient descent with noisy gradients.
problem Minimizing smooth convex and strongly convex functions with noisy gradients.
method Generalization of Nesterov's accelerated gradient descent algorithm for noisy conditions.
result AGNES achieves acceleration for noisy gradients with a constant of proportionality up to 1.