Improved loss scaling for stochastic momentum algorithms in high dimensions.
problem Improving loss scaling for stochastic momentum algorithms in high dimensions.
method Dimension-adapted Nesterov acceleration (DANA) scales momentum hyperparameters based on model size and data complexity.
result DANA improves loss scaling exponents across various data and target complexities.
Efficient momentum-based methods for reinforcement learning with improved sample complexity.
problem Efficient reinforcement learning algorithms for non-concave performance functions.
method Adaptive momentum-based policy gradient methods using variance reduction and importance sampling techniques.
result Both IS-MBPG and HA-MBPG reach the best known sample complexity of O(ε−3) for finding an ε-stationary point. New methods solve optimization problems with heavy-tailed noise, improving upon existing complexity bounds.
problem Optimization problems with heavy-tailed noise and weakly average smoothness.
method Normalized stochastic first-order methods with Polyak, multi-extrapolated, and recursive momentum.
result First-order oracle complexity results for finding approximate stochastic stationary points under heavy-tailed noise.
Analysis of momentum methods on quadratic models, showing SGD's superiority.
problem Analysis of stochastic gradient algorithms with momentum on quadratic models.
method Inspired by random matrix theory, exact characterization of loss values.
result Stochastic heavy-ball momentum does not improve over SGD in the strongly convex setting.
Study norm-squared of momentum map in infinite dimensions with applications to symplectic geometry.
problem Understanding the norm-squared of the momentum map in infinite-dimensional settings.
method Calculation of Hessian, decomposition of stabilizer, application to symplectic and complex structures.
result Positive semi-definiteness of Hessian along complexified orbit and new central extensions of symplectomorphism group.
Paper proposes an algorithm to solve complex minimax problems efficiently.
problem Stochastic nonconvex-concave minimax problems in various fields.
method Accelerated first-order regularized momentum descent ascent algorithm (FORMDA).
result Achieves best-known complexity bound of ildeO(ε−6.5) for single-loop algorithms. New algorithms reduce bilevel optimization complexity to ε^(-1.5).
problem Efficiently solving bilevel optimization problems in machine learning.
method Proposed two new algorithms: one using momentum-based recursive iterations, the other using recursive gradient estimations.
result Achieved computational complexity of ε^(-1.5), significantly faster than previous methods.
Unified model learns from both time-series and cross-sectional momentum features.
problem Separate time-series and cross-sectional momentum strategies do not consider concurrent relationships.
method Spatio-Temporal Momentum strategies using neural networks to combine both types of momentum.
result Simple neural network with single fully connected layer generates trading signals for all assets.
Negative momentum accelerates convergence in minimax games but at a suboptimal rate.
problem The convergence rate of negative momentum in minimax games is suboptimal.
method Extending variational inequality formulation, connecting momentum method with Chebyshev polynomials.
result Negative momentum accelerates convergence locally but at a suboptimal rate.
INO learns physical models with momentum conservation laws.
problem Learning physical models without preserving fundamental laws.
method Designing an invariant neural operator that automatically satisfies momentum conservation laws.
result The model learns complex material behaviors and achieves state-of-the-art accuracy and efficiency.
This paper simplifies complex nonholonomic systems using momentum map reduction.
problem Reducing complex nonholonomic systems with symmetries.
method Using nonholonomic momentum bundle map and gauge transformation.
result Reduced manifolds are Chaplygin-type leaves with an almost symplectic form.
Paper develops momentum schemes with variance reduction for non-convex composition optimization.
problem Lack of convergence guarantee and efficient momentum design in existing algorithms.
method Develops various momentum schemes with SPIDER-based variance reduction.
result Achieves near-optimal sample complexity and linear convergence rate.
DeepUnifiedMom uses deep learning to create better momentum portfolios.
problem Lack of unified momentum portfolios across different time frames.
method Multi-task learning with multi-gate mixture of experts.
result DeepUnifiedMom outperforms benchmark models in diverse asset classes.
In this paper we study several classes of stochastic optimization algorithms enriched with heavy ball momentum. Among the methods studied are: stochastic gradient descent, stochastic Newton, stochastic proximal point and stochastic dual subspace ascent. This is the first time momentum variants of several of these metho…
Proposes EDM algorithm to accelerate model training in distributed networks.
problem Hindered effectiveness of distributed stochastic optimization algorithms due to data heterogeneity and network sparsity.
method Introduces Exact-Diffusion with Momentum (EDM) algorithm, incorporating momentum techniques to mitigate bias and enhance convergence rate.
result EDM algorithm converges sub-linearly to the optimal solution, radius independent of data heterogeneity, for non-convex objective functions.
Improved sample complexity for actor-critic algorithms in MDPs.
problem Achieving optimal policies with limited data in reinforcement learning.
method Single-timescale actor-critic with STORM (STOchastic Recursive Momentum) and a sample buffer.
result Optimal sample complexity of O(ε−2) for ε-optimal policies. AuON is a linear-time optimizer that improves upon Muon's performance without approximate orthogonal matrices.
problem High memory and computational costs of orthogonal momentum updates.
method AuON uses normalized nonlinear scaling and a 'emergency brake' to handle exploding attention logits.
result AuON achieves strong performance without approximate orthogonal matrices, preserving structural alignment and reconditioning.
STORM-PG uses momentum for faster policy gradient updates.
problem Improving policy gradient methods for reinforcement learning.
method Introduces STORM-PG, a SARAH-based algorithm with exponential moving average.
result Achieves O(1/ε3) sample complexity, matching best-known rate. Modeling financial bubbles and crashes with a cubic momentum function.
problem Capturing the micro-level dynamics of investor behavior and panic selling.
method Introducing a cubic function of market momentum to model trend-following and sudden crashes.
result The model successfully replicates complex, nonlinear bubble dynamics.
The space of smooth sections of a symplectic fiber bundle carries a natural symplectic structure. We provide a general framework to determine the momentum map for the action of the group of bundle automorphism on this space. Since, in general, this action does not admit a classical momentum map, we introduce the more g…
New method finds near-optimal solutions for non-convex optimization problems.
problem Finding near-optimal solutions for non-convex optimization problems.
method Riemannian stochastic recursive momentum method
result Achieves a near-optimal complexity of ildeO(ε−3). Improved time complexity for parallel stochastic optimization in heterogeneous systems.
problem Time complexity in parallel stochastic optimization for large-scale machine learning models.
method Proposes Rennala MVR, a variance-reduced extension of Rennala SGD based on momentum-based variance reduction.
result Variance reduction improves time complexity in relevant parameter regimes for parallel stochastic optimization in heterogeneous systems.
Momentum is a popular technique to accelerate the convergence in practical training, and its impact on convergence guarantee has been well-studied for first-order algorithms. However, such a successful acceleration technique has not yet been proposed for second-order algorithms in nonconvex optimization.In this paper, …
A necessary and sufficient condition for energy-momentum conservation is proved within a topological, pre-metric approach to classical electrodynamics including magnetic as well as electric charges. The extended Lorentz force, consisting of mutual actions by F=(E, B) on the electric current and G=(H, D) on the magnetic…
The study finds that factor momentum is significant only at short lags compared to stock momentum.
problem Investigating the relationship between factor momentum and stock momentum.
method Replicated earlier findings and conducted a spanning test controlling for stock momentum and factor exposure.
result Factor momentum is significant only at short lags after controlling for stock momentum and factor exposure.
Games generalize the single-objective optimization paradigm by introducing different objective functions for different players. Differentiable games often proceed by simultaneous or alternating gradient updates. In machine learning, games are gaining new importance through formulations like generative adversarial netwo…
The paper simplifies complex mechanical systems with external forces.
problem Analyzing symmetric discrete mechanical systems with external forces.
method Lagrangian reduction and reconstruction for principal bundles.
result Evolution of momentum maps and Poisson structures under different conditions.
New algorithm reduces FL sample and communication costs.
problem Optimizing FL for minimal samples and rounds.
method Stochastic two-sided momentum algorithm.
result Achieves near-optimal sample and communication complexities.
We review how a reduction procedure along a principal fibration and an unfolding procedure associated to a suitable momentum map allow to describe the Kähler geometry of a finite dimensional complex projective spaces.
Study character varieties for 3-punctured sphere group representations in PU(2,1).
problem Characterize representations with special elliptic elements.
method Analyze relative character varieties and momentum maps.
result Representations form fibers of the momentum map.
We test the price momentum effect in the Korean stock markets under the momentum universe shrinkage to subuniverses of the KOSPI 200. Performance of the momentum strategy is not homogeneous with respect to change of the momentum universe. It is found that some submarkets generate the higher momentum returns than other …
Introduces homotopy momentum sections on multisymplectic manifolds.
problem No specific problem stated; focuses on introducing a new concept.
method Introduces a new concept of homotopy momentum sections on multisymplectic manifolds.
result Shows that a gauged nonlinear sigma model with Wess-Zumino term has homotopy momentum section structure.
The extragradient method accelerates convergence in complex game dynamics.
problem Complex interactions in game dynamics cause simple methods to diverge, necessitating more sophisticated approaches.
method A polynomial-based analysis to identify three scenarios for accelerated convergence of the momentum extragradient method.
result The momentum extragradient method achieves faster convergence under specific eigenvalue conditions.
Customer momentum is a positive relationship between a firm's returns and past returns of its customers.
problem Understanding the relationship between a firm's returns and its customers' past returns.
method Examined customer momentum using a long-short equally-weighted decile portfolio and Fama-French factor models.
result Customer momentum generates significant monthly returns and is statistically significant.
In smooth strongly convex optimization, knowledge of the strong convexity parameter is critical for obtaining simple methods with accelerated rates. In this work, we study a class of methods, based on Polyak steps, where this knowledge is substituted by that of the optimal value, f∗. We first show slightly improved …
The abstract explores a new wave equation linking quantum mechanics and complex adaptive systems.
problem Understanding the underlying mechanism of distribution formation in complex quantum entanglement.
method Exploring the logical relationship between Schrödinger's wave equation and Shi's trading volume-price wave equation in finance.
result A non-localized wave equation in quantum mechanics reveals the invariance of interaction as a universal law.
This paper examines momentum spillover across multiple asset classes using only pricing data.
problem Challenges in studying momentum spillover across diverse asset classes due to lack of common characteristics.
method Utilised a linear and interpretable graph learning model to reveal momentum spillover network.
result Network momentum strategy yields a Sharpe ratio of 1.5 and an annual return of 22%.
Introduces VSMD to improve generative diffusion processes without high costs.
problem High training costs and scalability issues in generative diffusion processes.
method Introduces variational Schrödinger momentum diffusion (VSMD) with adaptively transport-optimized variational scores and critical-damping transform.
result Efficiently generates anisotropic shapes while maintaining transport efficacy, outperforming alternatives.
The paper analyzes how hyperparameters affect SGD with momentum's convergence rate.
problem The role of hyperparameters in SGD with momentum's convergence rate.
method Theoretical analysis using a hyperparameters-dependent stochastic differential equation (hp-dependent SDE).
result The optimal linear rate of convergence depends on both the learning rate and the momentum coefficient.
SUSTAIN algorithm tackles stochastic bilevel optimization with near-optimal complexity.
problem Stochastic bilevel optimization problems with specific convexity and smoothness properties.
method SUSTAIN algorithm using single-timescale double-momentum stochastic approximation.
result SUSTAIN achieves near-optimal complexity for finding ε-stationary solutions.
This paper proposes an accelerated proximal stochastic variance reduced gradient (ASVRG) method, in which we design a simple and effective momentum acceleration trick. Unlike most existing accelerated stochastic variance reduction methods such as Katyusha, ASVRG has only one additional variable and one momentum paramet…
This paper presents generalized momentum mappings for covariant Hamiltonian field theories. The new momentum mappings arise from a generalization of symplectic geometry to LVY, the bundle of vertically adapted linear frames over the bundle of field configurations Y. Specifically, the generalized field momentum obs…
We give a detailed discussion about existence and uniqueness of Lu's momentum map. More precisely, we introduce the infinitesimal momentum map, and we study its properties. This allows us to describe the theory of reconstruction of the momentum map from the infinitesimal one. We provide the conditions for the uniquenes…
Momentum ResNets improve ResNets' memory efficiency.
problem Memory inefficiency in deep residual neural networks (ResNets).
method Adding a momentum term to the forward rule of ResNets to make them invertible.
result Momentum ResNets can learn any linear mapping up to a multiplicative factor, improving memory efficiency.
Two single-timescale algorithms improve TD learning with nonlinear approximations.
problem Optimizing TD learning with nonlinear smooth function approximation.
method Proposes two single-timescale single-loop algorithms with momentum and variance reduction.
result Achieves O(ε−4) sample complexity for the first algorithm and O(ε−3) for the second. We introduce various quantitative and mathematical definitions for price momentum of financial instruments. The price momentum is quantified with velocity and mass concepts originated from the momentum in physics. By using the physical momentum of price as a selection criterion, the weekly contrarian strategies are imp…
Novel method for solving ODEs on k-polysymplectic manifolds.
problem Solving ordinary differential equations on k-polysymplectic manifolds.
method k-polysymplectic energy-momentum method.
result Novel stability analysis techniques applied to Hamiltonian systems.
New algorithm Momentum-QNG improves optimization of quantum circuits.
problem Optimizing variational quantum circuits to avoid local minima.
method Applied Langevin dynamics to QNG, introducing momentum term.
result Momentum-QNG outperforms basic QNG and other optimizers.