We introduce a simple algorithm, True Asymptotic Natural Gradient Optimization (TANGO), that converges to a true natural gradient descent in the limit of small learning rates, without explicit Fisher matrix estimation. For quadratic models the algorithm is also an instance of averaged stochastic gradient, where the par…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,786 papers · 148 categories
Trend · papers per month
48 results for “natural gradient optimization”
New methods using natural gradient for structured optimization.
problem Structured optimization problems.
method Structured second-order methods via natural gradient descent.
result Efficiency demonstrated on non-convex and deep learning problems.
Optimizes graph neural networks using natural gradient descent.
problem Improving efficiency and performance of graph neural networks.
method Employing natural gradient descent to optimize graph neural networks.
result Natural gradient optimization leads to superior performance compared to existing methods.
Natural gradient for Wasserstein metric approximated using kernel methods.
problem Optimization of cost functionals over probability distributions.
method Kernelized Wasserstein metric, natural gradient approximation.
result Effective gradient estimator for Wasserstein metric with theoretical guarantees.
The natural gradient of ELBO vanishes in unconstrained optimization, simplifying learning.
problem The gap between evidence and ELBO has a vanishing natural gradient.
method Analyzes the Fisher-Rao gradient of ELBO and its implications for learning.
result Maximizing ELBO is equivalent to minimizing KL divergence, simplifying learning.
This work proposes a new method for variational inference using Wasserstein gradient descent.
problem Optimizing variational parameters to match a true posterior distribution.
method Reinterpreting VI as an optimization problem over a variational parameter space, using Wasserstein gradient descent.
result The proposed Wasserstein gradient descent can be seen as a generalization of existing optimization techniques in VI.
Solves POMDPs with recurrent neural networks and natural policy gradient.
problem Non-stationarity in optimal policies of POMDPs.
method Integrates recurrent neural networks into natural policy gradient and temporal difference learning.
result Non-asymptotic theoretical guarantees for global optimality up to function approximation.
Bayesian inference plays an important role in advancing machine learning, but faces computational challenges when applied to complex models such as deep neural networks. Variational inference circumvents these challenges by formulating Bayesian inference as an optimization problem and solving it using gradient-based op…
Natural gradient descent avoids the magic of model parametrization, leading to different optimization outcomes.
problem Understanding the impact of model parametrization on optimization and generalization in deep learning.
method Characterization of natural gradient flow in deep linear networks and nonlinear neural networks.
result Natural gradient descent fails to generalize in some cases, while gradient descent with the right architecture performs well.
In optimization, the natural gradient method is well-known for likelihood maximization. The method uses the Kullback-Leibler divergence, corresponding infinitesimally to the Fisher-Rao metric, which is pulled back to the parameter space of a family of probability distributions. This way, gradients with respect to the p…
A new black-box optimizer using implicit natural gradient.
problem Efficient optimization for complex, computationally intensive problems.
method Stochastic update with implicit natural gradient of an exponential-family distribution.
result Theoretical convergence rate for convex functions and continuous non-differentiable functions.
A new method uses natural gradients for efficient distribution optimization.
problem Challenges in computing natural gradients for many distributions.
method Reframe optimization as a surrogate distribution with easy natural gradient computation.
result Expands set of distributions efficiently targetable with natural gradients.
We present a novel algorithm to train a deep Q-learning agent using natural-gradient techniques. We compare the original deep Q-network (DQN) algorithm to its natural-gradient counterpart, which we refer to as NGDQN, on a collection of classic control domains. Without employing target networks, NGDQN significantly outp…
Proposes a new stochastic optimization method for MLR models.
problem Slow convergence of SGD in big data scenarios.
method Dual Stochastic Natural Gradient Descent (DNSGD) based on manifold optimization.
result DNSGD converges and has linear computational complexity.
NES optimizes discrete structured VAEs effectively without gradient propagation.
problem Learning high-dimensional discrete latent spaces in generative models.
method Natural Evolution Strategies (NES) for gradient-free optimization of discrete structures.
result NES effectively optimizes discrete structured VAEs, comparable to gradient-based methods.
Square-root natural-gradient improves variational inference convergence.
problem Challenges in establishing theoretical convergence guarantees for natural-gradient descent.
method Square-root parameterization for Gaussian covariance.
result Establishes novel convergence guarantees for natural-gradient Gaussian inference.
NVA combines variational posteriors, annealing, and natural-gradient learning for multimodal optimization.
problem Finding multiple global and local modes in nonconvex objectives.
method NVA integrates variational posteriors, annealing, and natural-gradient learning.
result NVA outperforms gradient descent and evolution strategies on simulations and real-world problems.
Natural gradient descent is an optimization method traditionally motivated from the perspective of information geometry, and works well for many applications as an alternative to stochastic gradient descent. In this paper we critically analyze this method and its properties, and show how it can be viewed as a type of 2…
Off-policy stochastic actor-critic methods rely on approximating the stochastic policy gradient in order to derive an optimal policy. One may also derive the optimal policy by approximating the action-value gradient. The use of action-value gradients is desirable as policy improvement occurs along the direction of stee…
Variational inference transforms posterior inference into parametric optimization thereby enabling the use of latent variable models where otherwise impractical. However, variational inference can be finicky when different variational parameters control variables that are strongly correlated under the model. Traditiona…
Researchers propose a non-monotone quantum natural gradient for quantum systems.
problem Applying natural gradient methods to quantum systems without monotonicity.
method Introducing a non-monotone quantum natural gradient (QNG) and demonstrating its superiority over conventional QNG.
result Non-monotone QNG outperforms conventional QNG in terms of convergence speed.
Improves understanding of stochastic NGVI convergence rates.
problem Lack of knowledge about non-asymptotic convergence rates in stochastic NGVI.
method Proved non-asymptotic convergence rates for conjugate likelihoods and showed implicit optimization for non-conjugate likelihoods.
result First non-asymptotic convergence rate for stochastic NGVI in conjugate likelihoods.
Rank-1 Approximation of Inverse Fisher for Natural Policy Gradients in Deep Reinforcement Learningcs.LG
Paper presents a rank-1 approximation method for natural policy gradients in deep RL.
problem Computing natural gradients requires inverting the Fisher Information Matrix, which is computationally expensive.
method Develops a rank-1 approximation to the inverse Fisher Information Matrix for efficient natural policy optimization.
result The rank-1 approximation converges faster and has similar sample complexity to stochastic policy gradient methods.
This paper deals with estimating model parameters in graphical models. We reformulate it as an information geometric optimization problem and introduce a natural gradient descent strategy that incorporates additional meta parameters. We show that our approach is a strong alternative to the celebrated EM approach for le…
We study the Wasserstein natural gradient in parametric statistical models with continuous sample spaces. Our approach is to pull back the -Wasserstein metric tensor in the probability density space to a parameter space, equipping the latter with a positive definite metric tensor, under which it becomes a Riemanni…
RoNGBa: A Robustly Optimized Natural Gradient Boosting Training Approach with Leaf Number Clippingcs.LG
Improved natural gradient boosting with leaf number clipping for faster and better performance.
problem Slower training speed and poor performance on large datasets for natural gradient boosting.
method Leaf number clipping regularization to optimize hyperparameters and improve performance.
result Significant improvement in performance and up to 4.85x speed up on various datasets.
AOPU stabilizes NN training by approximating natural gradient, improving stability and convergence.
problem Stability and interpretability in online NN training for industrial soft sensors.
method AOPU truncates gradient backpropagation, optimizing trackable parameters, and approximating natural gradient.
result AOPU achieves stable convergence and superior performance on chemical process datasets.
Proves FR-NGD optimally approximates evolutionary dynamics and continuous Bayesian inference.
problem Optimizing continuous time replicator equations and continuous Bayesian inference.
method Fisher-Rao natural gradient descent (FR-NGD) and its correspondence with evolutionary dynamics.
result FR-NGD optimally approximates continuous time replicator equations and continuous Bayesian inference.
Develops a new method for optimizing policies in hierarchical models.
problem Optimizing complex policies in hierarchical models.
method Applies second-order methods in the space of state-action paths.
result The natural path gradient method can be computed exactly and reflects state-space hierarchy.
NGD improves multivariate Gaussian inference by optimizing Fisher information.
problem Efficiently optimizing multivariate Gaussian models.
method Natural Gradient Descent applied to multivariate Gaussian parameters.
result NGD updates are more efficient for symmetric covariance matrices.
QBVI uses natural gradients for efficient Bayesian learning.
problem Efficient Bayesian learning in complex models.
method Natural gradient updates in a black-box framework for exponential-family distributions.
result QBVI framework is effective for a wide range of Bayesian inference problems.
Optimal control methods achieve significantly smaller regret than previously thought.
problem Optimal control in linear dynamical systems with adversarial changes.
method Online gradient descent and online natural gradient methods.
result Achieves logarithmic regret scaling as O(poly(log T)) instead of O(sqrt(T)).
Paper improves online time series forecasting by combining natural gradient and robust t-distribution.
problem Online time series forecasting challenges in rapidly adapting to evolving data.
method Reframed neural network optimization as a parameter filtering problem, using natural gradient and Student's t likelihood.
result Natural Score-driven Replay (NatSR) achieves stronger forecasting performance than state-of-the-art methods.
DSPI connects natural policy gradient to policy iteration, proving global convergence.
problem Optimizing policies in reinforcement learning.
method DSPI framework, combining smoothed policy iteration and natural policy gradient.
result DSPI achieves geometric convergence and optimal complexity for policy optimization.
Trust-region methods have yielded state-of-the-art results in policy search. A common approach is to use KL-divergence to bound the region of trust resulting in a natural gradient policy update. We show that the natural gradient and trust region optimization are equivalent if we use the natural parameterization of a st…
Black box discrete optimization (BBDO) appears in wide range of engineering tasks. Evolutionary or other BBDO approaches have been applied, aiming at automating necessary tuning of system parameters, such as hyper parameter tuning of machine learning based systems when being installed for a specific task. However, auto…
New methods optimize training VQAs without barren plateaus, improving efficiency and applicability.
problem Barren plateaus in training variational quantum algorithms.
method Derive adaptive learning rates and use Gaussian kernels to optimize movement in parameter space.
result Optimized training methods outperform other routines and can train VQAs free of barren plateaus.
Stochastic variational inference (SVI) lets us scale up Bayesian computation to massive data. It uses stochastic optimization to fit a variational distribution, following easy-to-compute noisy natural gradients. As with most traditional stochastic optimization methods, SVI takes precautions to use unbiased stochastic g…
The paper studies efficient Hessian fitting methods for stochastic optimization.
problem Efficient Hessian fitting for stochastic optimization.
method Preconditioned Stochastic Gradient Descent (PSGD) method and Lie groups.
result Hessian fitting problem is strongly convex in certain Lie groups.
BONG optimizes Bayesian inference online with natural gradient descent.
problem Sequential Bayesian inference in online settings.
method Bayesian online natural gradient (BONG) approach based on variational Bayes.
result BONG outperforms other online VB methods in non-conjugate settings.
We use differential equations based approaches to provide some {\it \textbf{physics}} insights into analyzing the dynamics of popular optimization algorithms in machine learning. In particular, we study gradient descent, proximal gradient descent, coordinate gradient descent, proximal coordinate gradient, and Newton's …
Quantum Natural Gradientquant-ph
A quantum generalization of Natural Gradient Descent is presented as part of a general-purpose optimization framework for variational quantum circuits. The optimization dynamics is interpreted as moving in the steepest descent direction with respect to the Quantum Information Geometry, corresponding to the real part of…
Gradient descent solves robust mean estimation in high dimensions.
problem High-dimensional robust mean estimation in the presence of adversarial outliers.
method Gradient descent with a structural lemma showing near-optimal solutions.
result Gradient descent can solve the robust mean estimation problem directly.
Improved VI method for deep mixed models in finance.
problem Inaccurate and slow variational inference in high dimensions.
method Natural gradient hybrid VI method targeting joint posterior.
result Natural gradient method is faster and more accurate than existing methods.
New findings on optimal transport gradient for generative models, addressing numerical instabilities.
problem Numerical instabilities in training Wasserstein Generative Adversarial Networks (WGAN).
method Valid differentiation theorem for entropic regularized transport, semi-discrete gradient formulation, and optimization algorithm.
result Existence of optimal transport gradient for generative models under specified conditions.
Natural gradient descent is a robust optimization method for machine learning.
problem Training poorly parameterized networks efficiently.
method Optimization algorithms with natural transformation properties.
result Optimization algorithms with natural transformation properties are more efficient for poorly parameterized networks.
Optimistic Natural Policy Gradient: a Simple Efficient Policy Optimization Framework for Online RLcs.LG
Optimistic NPG improves policy optimization in online RL with efficient sample complexity.
problem Limited theoretical understanding of policy optimization, especially in online RL.
method Combines natural policy gradient with optimistic policy evaluation.
result Achieves optimal dimension dependence sample complexity for learning near-optimal policies.
Proposes CoPO, a new policy optimization method for competitive games.
problem Designing efficient optimization methods for competitive Markov decision processes.
method Competitive policy optimization (CoPO) approach that exploits game-theoretic nature of competitive games.
result Stable optimization, convergence to sophisticated strategies, and higher scores compared to baseline methods.