Hamiltonian dynamics-based algorithms achieve deterministic and accelerated convergence for convex optimization.
problem Accelerating convex optimization
method Hamiltonian dynamics
result Hamiltonian dynamics-based algorithms achieve deterministic and accelerated convergence for convex optimization.
Paper discovers structural dynamics equations from only acceleration data.
problem Discovering equations from only acceleration measurements in structural dynamics.
method Library-based approach with Approximate Bayesian Computation (ABC) prioritizing parsimonious models.
result Efficacy demonstrated in four structural dynamics examples, including linear and nonlinear systems.
New dynamical system framework explains Nesterov acceleration.
problem Understanding Nesterov's accelerated gradient method.
method Dynamical system derivation without vanishing step size.
result Acceleration arises from discretizing an ODE with semi-implicit Euler.
SympFormer accelerates attention blocks using inertial dynamics on density spaces.
problem Improving the efficiency of self-attention blocks in Transformers.
method Introduced accelerated attention blocks derived from inertial Nesterov dynamics on density spaces.
result Accelerated attention blocks converge faster than classical blocks while preserving oracle calls.
Study of accelerated dynamics for convex function minimization with noisy gradients.
problem Minimizing smooth convex functions with noisy gradients.
method Formulate and study continuous-time stochastic dynamics, prove convergence rates.
result Derive estimates of convergence rates for function values, both persistent and asymptotic.
New algorithm accelerates optimization on Riemannian manifolds, including Wasserstein space.
problem Accelerating optimization methods in Riemannian geometry.
method Dynamic stepsize algorithms on Riemannian manifolds with specific vector transport.
result First provable accelerated gradient method in Wasserstein space.
HF-opt uses Hamiltonian dynamics to optimize functions, achieving accelerated rates with randomized integration time.
problem Optimizing functions efficiently and accelerating convergence rates.
method Randomized Hamiltonian flow (RHF) with accelerated convergence rates.
result RHGD achieves accelerated convergence rates similar to Nesterov's AGD.
Enhances stochastic gradient methods with accelerated dynamics.
problem Reducing computational cost in large dataset learning.
method Proposes a novel technique using accelerated stochastic dynamics to mitigate stochasticity in the stochastic Langevin gradient method.
result Demonstrates improved convergence and reduced correlation time through violating the detailed balance condition.
Improved Anderson acceleration speeds up nonlinear optimization.
problem Optimizing nonlinear functions efficiently.
method Combining Anderson acceleration with Chebyshev polynomials.
result Achieves optimal convergence rate for nonlinear problems.
Study accelerates optimization methods in non-convex problems, but doesn't improve the algorithm's performance.
problem Understanding the behavior of momentum-based acceleration methods in non-convex, high-dimensional landscapes.
method Used dynamical mean field theory to describe the average dynamics of heavy-ball momentum and Nesterov acceleration in a non-convex model.
result Accelerated dynamics but did not improve the algorithm's performance with respect to gradient descent.
Novel geometry-informed irreversible perturbation accelerates Langevin dynamics convergence.
problem Accelerating convergence of Langevin dynamics for Bayesian computation.
method Geometry-informed irreversible perturbation of Riemannian manifold Langevin dynamics.
result Improves estimation performance over irreversible perturbations that ignore geometry.
The paper connects dynamical systems to ADMM for solving nonsmooth constrained problems.
problem Solving nonsmooth and constrained optimization problems.
method Developed differential inclusions for accelerated ADMM variants and analyzed their convergence rates.
result Derived rates of convergence for the dynamical systems under different settings, showing tradeoffs between damping strategies.
Timewarp accelerates molecular dynamics by learning to simulate long timescales.
problem Efficiently simulating long timescales in molecular dynamics.
method Uses a normalizing flow to learn large time steps in Markov chain Monte Carlo.
result Generalizes to unseen small peptides, accelerating sampling.
RL accelerates portfolio optimization and option pricing by dynamically adjusting preconditioner sizes.
problem Large linear systems in portfolio optimization and option pricing lead to slow convergence.
method Reinforcement Learning (RL) dynamically adjusts block-preconditioner sizes to accelerate convergence.
result RL-driven solver significantly reduces computational cost and accelerates convergence.
A new method accelerates K-Means clustering by reducing the number of iterations.
problem Reducing the number of iterations required for K-Means clustering convergence.
method Applying Anderson acceleration to the assignment and update steps of Lloyd's algorithm, dynamically adjusting the number of previous iterates used.
result Achieves robust and consistent speedups across different problem instances, outperforming other algorithms in 106 out of 120 test cases.
Surrogate models speed up RL training in dynamic systems.
problem High computational cost of high-fidelity simulations.
method Developed and tested surrogate models for RL training.
result Surrogate models can significantly accelerate RL training.
ES improves training efficiency by dynamically selecting data samples.
problem Efficiently selecting informative data samples for faster learning.
method Evolved Sampling (ES) dynamically selects data samples based on loss dynamics and differences.
result ES achieves significant training acceleration without compromising model performance.
Improved reSGLD accelerates convergence in non-convex learning problems.
problem Inefficient swaps due to noisy energy estimators in reSGLD.
method Variance reduction for noisy energy estimators, theoretical analysis, and numerical experiments.
result Exponential acceleration in convergence for non-convex learning problems.
Paper analyzes and accelerates Langevin Monte Carlo methods using large deviations theory.
problem High-dimensional sampling problems in machine learning.
method Unified approach using large deviations theory to study and accelerate Langevin dynamics variants.
result Efficiency of Langevin dynamics variants demonstrated through numerical experiments.
The extragradient method accelerates convergence in complex game dynamics.
problem Complex interactions in game dynamics cause simple methods to diverge, necessitating more sophisticated approaches.
method A polynomial-based analysis to identify three scenarios for accelerated convergence of the momentum extragradient method.
result The momentum extragradient method achieves faster convergence under specific eigenvalue conditions.
Paper applies Anderson acceleration to speed up reinforcement learning.
problem Speeding up convergence in reinforcement learning.
method Applies Anderson acceleration, a method for accelerating fixed point computation.
result Shows significant speed up of convergence in preliminary experiments.
Acceleration in Hilbert spaces reduces computations but not accuracy.
problem Improving learning accuracy with fewer computations.
method Analysis of Nesterov acceleration and heavy-ball methods in Hilbert spaces.
result Acceleration can reduce computations but not improve accuracy with respect to gradient descent.
New birth-death dynamics accelerates convergence in neural networks.
problem Accelerating convergence in neural networks with large parameters.
method Proposed a non-local mass transport dynamics as a stochastic neuronal birth-death process.
result Proved that the birth-death dynamics accelerates the rate of convergence in the mean-field limit.
RHMC accelerates sampling from log-concave distributions.
problem Sampling from log-concave probability distributions efficiently.
method RHMC uses simulated Hamiltonian dynamics with random integration times.
result RHMC converges exponentially fast in KL divergence for log-concave distributions.
The ECME algorithm has proven to be an effective way of accelerating the EM algorithm for many problems. Recognising the limitation of using prefixed acceleration subspace in ECME, we propose the new Dynamic ECME (DECME) algorithm which allows the acceleration subspace to be chosen dynamically. Our investigation of an …
Superposition accelerates training to a universal power-law exponent.
problem Training dynamics in neural networks.
method Teacher-student framework and analytic theory.
result Superposition leads to a universal power-law exponent of ~1, independent of data and channel statistics.
BoostMD accelerates molecular dynamics simulations by 8x with ML force fields.
problem Long inference times of ML force fields limit practical use in molecular dynamics.
method BoostMD uses previous time-step features to predict energies and forces, reducing complexity and computational cost.
result BoostMD achieves an 8-fold speedup and accurately samples the Boltzmann distribution.
This work accelerates constrained sampling using large deviation principles.
problem Sampling constrained probability distributions efficiently.
method Large deviation principles applied to skew-reflected non-reversible Langevin dynamics.
result The skew-symmetric matrix accelerates convergence and reduces asymptotic variance.
This research accelerates sampling methods using Nesterov's Acceleration.
problem Improving sampling efficiency in MCMC methods.
method Developed a Hessian-Free High-Resolution ODE reformulation of NAG-SC, injected noise, and discretized the diffusion process.
result Quantified acceleration beyond underdamped Langevin in W2 distance for log-strongly-concave targets. Accelerates convergence in global non-convex optimization with reversible diffusion.
problem Global non-convex optimization challenges.
method Utilizes reversible diffusion processes with adaptive diffusion coefficients.
result Accelerated convergence with reduced discretization error.
Autoencoders discover and accelerate molecular dynamics simulations.
problem Efficient sampling of macromolecular folding landscapes with high free energy barriers.
method Employing auto-associative artificial neural networks to learn nonlinear collective variables (CVs) that are explicit and differentiable functions of atomic coordinates.
result Substantial speedups in exploration of configurational space and discovery of data-driven CVs.
New method accelerates energetic variational inference using particle dynamics.
problem Efficiently solving variational inference problems with reduced computational cost.
method Particle-based variational inference with implicit scheme, inspired by energy quadratization and operator splitting.
result Significantly reduces computational cost compared to existing methods.
Study adapts liquidity model to equity auctions, revealing accelerated event rates and reduced price impact.
problem Understanding and predicting price dynamics in equity auctions.
method Adapted latent/revealed order book framework to equity auctions, measuring order submissions, cancellations, and diffusion rates.
result Equity auctions exhibit accelerated event rates leading to reduced price impact and decreased volatility.
Paper analyzes ResNets from dynamical systems perspective and proposes a faster training method.
problem Understanding and optimizing the design and training of ResNets.
method Dynamical systems view and lesioning properties analysis, followed by a novel training acceleration method.
result Training ResNets and Wide ResNets is accelerated by more than 40% with superior or equivalent accuracy.
Momentum speeds up evolutionary processes in machine learning.
problem Accelerating convergence in evolutionary dynamics.
method Combining momentum from machine learning with evolutionary dynamics using information divergences as Lyapunov functions.
result Momentum accelerates convergence of evolutionary dynamics, including the replicator equation and Euclidean gradient descent.
Two new differentially private optimization algorithms derived from accelerated methods.
problem Improving privacy in optimization algorithms while maintaining convergence rates.
method Polyak's heavy ball method and Nesterov's accelerated gradient method with differential privacy.
result The proposed algorithms outperform existing differentially private optimization methods.
New study on No-U-Turn Sampler for accelerated mixing in Hamiltonian Monte Carlo.
problem Achieving accelerated convergence in Hamiltonian Monte Carlo.
method Combining concentration of measure and coupling analysis for mixing.
result Rigorous mixing guarantees for the No-U-Turn Sampler in certain Gaussian distributions.
New method accelerates smooth games using spectral shape analysis.
problem Accelerating optimization in smooth games with complex numerical challenges.
method Matrix iteration theory and spectral shape analysis to characterize and manipulate acceleration.
result Identified a continuum of optimization strategies from convex minimization to gradient descent.
AB dynamically scales gradients to mitigate asynchronous training delays.
problem Gradient delay in asynchronous training reduces model performance.
method Adaptive Braking (AB) dynamically scales gradients based on alignment.
result AB enables training with up to 32 update steps of delay without accuracy loss.
Generative models accelerate molecular dynamics by four orders of magnitude.
problem Femtosecond time steps limit access to slow molecular processes.
method Deep generative modeling framework that accelerates sampling.
result Quantitative characterization of equilibrium ensembles and dynamical relaxation processes.
New insights into GNN optimization reveal skip connections and depth accelerate training.
problem Understanding and optimizing the training of Graph Neural Networks (GNNs).
method Analysis of gradient dynamics in linearized GNNs and empirical validation.
result GNNs are implicitly accelerated by skip connections, more depth, and good label distribution during training.
GFM models neural network training as a dynamical system to forecast final weights.
problem Computational intensity and inefficiency in training deep neural networks.
method Gradient Flow Matching (GFM) treats training as a dynamical system with learned vector fields.
result GFM achieves forecasting accuracy competitive with Transformer-based models and significantly outperforms classical baselines.
Proposes a Gaussian process model for constrained dynamics learning.
problem Challenges in identifying constrained dynamics of mechanical systems.
method Combines analytical mechanics with Gaussian process regression.
result Improves data efficiency and constraint integrity in predictions.
We accelerate PMD algorithms for reinforcement learning using functional methods.
problem Improving reinforcement learning algorithms for large-scale optimization.
method Proposed a momentum-based update for PMD algorithms leveraging duality.
result The proposed method accelerates reinforcement learning algorithms.
New method improves sampling from non-convex distributions using HFHR dynamics.
problem Sampling from non-log-concave densities with non-convex potential functions.
method Hessian-free high-resolution dynamics (HFHR) with reflection/synchronous coupling.
result HFHR dynamics converges faster than kinetic Langevin dynamics (KLD) for non-convex potentials.
PruneTrain speeds up neural network training by dynamically pruning weights.
problem Efficiently training large neural networks with high compute and memory costs.
method Structured group-lasso regularization and reconfiguration techniques to reduce weights and model size.
result Achieved 39% reduction in end-to-end training time for ResNet50 on ImageNet.
Efficient methods accelerate diffusion model sampling.
problem Slow sample generation in diffusion models.
method Conjugate Integrators and Splitting Integrators.
result Hybrid method achieves best FID scores.
This paper analyzes two Lie group momentum optimization algorithms and their convergence rates.
problem Optimizing functions on Lie groups using momentum-based dynamics.
method Investigates Lie Heavy-Ball and Lie NAG-SC algorithms, quantifying their convergence rates under smoothness and convexity assumptions.
result Lie NAG-SC accelerates optimization over the momentumless case, while Lie Heavy-Ball does not.