A quantum generalization of Natural Gradient Descent is presented as part of a general-purpose optimization framework for variational quantum circuits. The optimization dynamics is interpreted as moving in the steepest descent direction with respect to the Quantum Information Geometry, corresponding to the real part of…
Develops an analytic theory for quantum imaginary time evolution.
problem Lack of a first-principle understanding of quantum imaginary time evolution.
method Interprets QITE as a form of VQA trained with QNGD and connects it to the geometric geodesic distance in the quantum Fisher information metric.
result QITE converges faster than vanilla gradient descent-based VQAs, though the advantage is suppressed by Hilbert space dimensionality.
Analyzes dynamics of quantum neural networks, predicting exponential decay of training error.
problem Understanding convergence rate of quantum neural networks training.
method Analytic theory for gradient descent dynamics of wide quantum neural networks.
result Simple analytic formula predicts exponential decay of training error.
New algorithm Momentum-QNG improves optimization of quantum circuits.
problem Optimizing variational quantum circuits to avoid local minima.
method Applied Langevin dynamics to QNG, introducing momentum term.
result Momentum-QNG outperforms basic QNG and other optimizers.
ECD algorithm speeds up non-convex optimization, offering quantum and stochastic enhancements.
problem Non-convex optimization challenges in machine learning.
method Energy Conserving Descent (ECD) algorithm, stochastic ECD dynamics (sECD), quantum ECD Hamiltonian (qECD).
result ECD and its quantum version achieve exponential speedup over gradient descent.
A new optimizer saves significant shots in quantum machine learning.
problem Training variational QML algorithms is challenging due to large datasets and shot-count overhead.
method Proposed Refoqus optimizer that samples over both dataset and measurement operators.
result Refoqus can save several orders of magnitude in shot cost compared to existing methods.
Quantum systems learn like machine learning models, influenced by dissipation.
problem Understanding how quantum systems learn and evolve.
method Hydrodynamical formulation of quantum mechanics, gradient descent model, empirical demonstration.
result Quantum systems follow a disrupted gradient descent model, influenced by dissipation.
A quantum reinforcement learning algorithm reduces sample complexity.
problem Quantum reinforcement learning under model-free settings with quantum oracle access.
method Quantum Natural Policy Gradient (QNPG) algorithm replacing random sampling with deterministic gradient estimation.
result QNPG achieves a sample complexity of i l d e O ( ε − 1.5 ) ilde{\mathcal{O}}(ε^{-1.5}) i l d e O ( ε − 1.5 ) for queries to the quantum oracle, significantly improving classical lower bound. Quantum networks offer exponential communication savings for large machine learning models.
problem Training and inference of large models require efficient communication.
method Quantum encoding and gradient descent for distributed computation.
result Exponential reduction in communication for gradient descent on quantum networks.
New algorithm improves efficiency of quantum system modeling.
problem Intractable complexities in quantum Hamiltonian learning and Gibbs sampling.
method Generalized quantum natural gradient descent and Quantum-Probabilistic Mirror Descent.
result Data sample efficiency proven using information geometry and quantum metrology.
Researchers propose a non-monotone quantum natural gradient for quantum systems.
problem Applying natural gradient methods to quantum systems without monotonicity.
method Introducing a non-monotone quantum natural gradient (QNG) and demonstrating its superiority over conventional QNG.
result Non-monotone QNG outperforms conventional QNG in terms of convergence speed.
SGLBO optimizes quantum circuits with fewer measurements, improving accuracy and noise resilience.
problem Efficiently optimizing parameterized quantum circuits with reduced measurement shots and noise.
method Developed SGLBO combining SGD and BO, with adaptive measurement-shot strategy and suffix averaging.
result Significantly reduces measurement-shot cost while improving accuracy and noise resilience.
Quantum machine learning faces 'laziness' and 'barren plateaus', but noise can mitigate the latter.
problem Quantum machine learning's loss function landscape issues.
method Theoretical analysis of quantum variational circuits, neural tangent kernels, and noise effects.
result Noise can mitigate barren plateaus in quantum machine learning.
We study the projected gradient descent method on low-rank matrix problems with a strongly convex objective. We use the Burer-Monteiro factorization approach to implicitly enforce low-rankness; such factorization introduces non-convexity in the objective. We focus on constraint sets that include both positive semi-defi…
Quantum method speeds up VB estimation in machine learning.
problem Prohibitively expensive natural gradient in high dimensions.
method Regression-based natural gradient estimation with quantum matrix inversion.
result Quantum method enables efficient VB estimation.
Natural gradient descent avoids the magic of model parametrization, leading to different optimization outcomes.
problem Understanding the impact of model parametrization on optimization and generalization in deep learning.
method Characterization of natural gradient flow in deep linear networks and nonlinear neural networks.
result Natural gradient descent fails to generalize in some cases, while gradient descent with the right architecture performs well.
We extend quantum probabilistic models and develop a learning algorithm.
problem Characterize expressiveness and learn hidden quantum Markov models.
method Show HQMMs are a subclass of OOMs without negative probabilities. Develop a feasible gradient descent algorithm on the Stiefel manifold.
result Our learning algorithm is faster and scales better than previous methods.
Derives Mirror Descent from gradient flow on a Riemannian manifold.
problem No specific problem stated; focuses on derivation.
method Derives Mirror Descent from gradient flow on a Riemannian manifold with a natural discretization.
result Generalizes Mirror Descent to non-Hessian metrics.
We introduce two quantum algorithms for solving structured prediction problems. We first show that a stochastic gradient descent that uses the quantum minimum finding algorithm and takes its probabilistic failure into account solves the structured prediction problem with a runtime that scales with the square root of th…
Study evaluates capacity and trainability of parametrized quantum circuits.
problem Finding the best type of circuits for hybrid quantum-classical algorithms.
method Geometric structure of parameter space, effective quantum dimension, and circuit expressiveness.
result Identifies a transition in quantum geometry leading to decay of quantum natural gradient for deep circuits.
New methods optimize training VQAs without barren plateaus, improving efficiency and applicability.
problem Barren plateaus in training variational quantum algorithms.
method Derive adaptive learning rates and use Gaussian kernels to optimize movement in parameter space.
result Optimized training methods outperform other routines and can train VQAs free of barren plateaus.
New methods using natural gradient for structured optimization.
problem Structured optimization problems.
method Structured second-order methods via natural gradient descent.
result Efficiency demonstrated on non-convex and deep learning problems.
Quantum kernel improves probabilistic time series forecasting.
problem Quantifying uncertainty in probabilistic time series predictions.
method Integrates quantum kernel with Gaussian process regression.
result Quantum kernel enhances forecasting performance.
Deeper quantum circuits can improve performance on unseen data, contrary to traditional views.
problem Understanding scaling behavior of parameterized quantum circuits and their generalization.
method Gradient-based PQCs, add-one-in perturbation techniques, spectral properties of random matrices.
result Gradient-based PQCs can exhibit improved performance on unseen data as model size increases, displaying double descent behavior.
We introduce a simple algorithm, True Asymptotic Natural Gradient Optimization (TANGO), that converges to a true natural gradient descent in the limit of small learning rates, without explicit Fisher matrix estimation. For quadratic models the algorithm is also an instance of averaged stochastic gradient, where the par…
This work proposes a new method for variational inference using Wasserstein gradient descent.
problem Optimizing variational parameters to match a true posterior distribution.
method Reinterpreting VI as an optimization problem over a variational parameter space, using Wasserstein gradient descent.
result The proposed Wasserstein gradient descent can be seen as a generalization of existing optimization techniques in VI.
Quantum computing speeds up neural network training and retraining.
problem Inefficient classical training and retraining of neural networks.
method Adiabatic quantum computing to optimize Kolmogorov-Arnold Networks using Bezier curves.
result Quantum optimization achieves 100x faster retraining compared to classical methods.
A new method speeds up quantum state estimation.
problem Exponential growth in sample size and dimension for quantum state tomography.
method Stochastic mirror descent with Burg entropy.
result Optimization error vanishes at a O ( ( 1 / t ) d log t ) O (\sqrt{ ( 1 / t ) d \log t }) O ( ( 1/ t ) d log t ) rate. NGD improves multivariate Gaussian inference by optimizing Fisher information.
problem Efficiently optimizing multivariate Gaussian models.
method Natural Gradient Descent applied to multivariate Gaussian parameters.
result NGD updates are more efficient for symmetric covariance matrices.
Study quantum aspects of 1-form symmetries using BV-BRST cohomology.
problem Quantum aspects of gauging continuous 1-form global symmetries.
method BV-BRST quantization of a U(1) 2-form gauge field, Lie 2-algebroid construction, Čech-de Rham bicomplex.
result Anomaly descent for U(1) 1-form symmetries is naturally set up in the Čech-de Rham bicomplex.
Study quantum aspects of 1-form symmetries using BV-BRST cohomology and gerbes.
problem Quantum aspects of gauging continuous 1-form global symmetries.
method BV-BRST quantization of a U(1) 2-form gauge field, Lie 2-algebroid construction, Čech-de Rham bicomplex.
result Anomaly descent for U(1) 1-form symmetries is naturally set up in the Čech-de Rham bicomplex.
Warm starts improve variational quantum algorithms by avoiding barren plateaus.
problem Barren plateaus in variational quantum algorithms limit scaling.
method Exploring warm starts in iterative variational methods for quantum circuits.
result Warm starts can lead to substantial gradients in small regions, suggesting trainability.
The Conant-Ashby theorem is verified for hypergraph observers, leading to unique learning rules.
problem Verifying conditions for hypergraph observers to maintain internal models.
method Formalizing persistent observers, applying the Conant-Ashby theorem, and using natural gradient descent.
result Natural gradient descent is the unique admissible learning rule for hypergraph observers.
Optimizes graph neural networks using natural gradient descent.
problem Improving efficiency and performance of graph neural networks.
method Employing natural gradient descent to optimize graph neural networks.
result Natural gradient optimization leads to superior performance compared to existing methods.
This paper investigates the relationship between algebraic quantum field theories and factorization algebras on globally hyperbolic Lorentzian manifolds. Functorial constructions that map between these two types of theories in both directions are developed under certain natural hypotheses, including suitable variants o…
Paper proposes ZO-NGD for more efficient black-box attacks.
problem Vulnerability of state-of-the-art DNNs to adversarial attacks.
method Zeroth-order natural gradient descent (ZO-NGD) for black-box attacks.
result ZO-NGD achieves significantly lower model query complexities.
Natural gradient learning improves synaptic plasticity in spiking neurons.
problem Parametrization dependence leads to inconsistencies in classical synaptic plasticity theories.
method Proposes natural gradient descent in Riemannian geometry for spiking neurons.
result Derives a synaptic learning rule that explains biological phenomena.
Proposes a new stochastic optimization method for MLR models.
problem Slow convergence of SGD in big data scenarios.
method Dual Stochastic Natural Gradient Descent (DNSGD) based on manifold optimization.
result DNSGD converges and has linear computational complexity.
We present theoretical results on the convergence of \emph{non-convex} accelerated gradient descent in matrix factorization models with ℓ 2 \ell_2 ℓ 2 -norm loss. The purpose of this work is to study the effects of acceleration in non-convex settings, where provable convergence with acceleration should not be considered a \em…
Natural gradient descent has proven effective at mitigating the effects of pathological curvature in neural network optimization, but little is known theoretically about its convergence properties, especially for \emph{nonlinear} networks. In this work, we analyze for the first time the speed of convergence of natural …
Quantum models show improved performance in overparameterized regimes.
problem Overfitting in quantum machine learning models.
method Analytical demonstration and numerical experiments on quantum kernel methods.
result Quantum models can operate in the modern, overparameterized regime without overfitting.
Quantum field theory connects deep neural networks to criticality.
problem Understanding the criticality and training dynamics of deep neural networks.
method Constructing quantum field theory for deep neural networks, computing corrections to correlation functions.
result Found precise analogy with O ( N ) O(N) O ( N ) vector model, providing corrections to correlation length. QBC uses quantum computers to speed up Bayesian computation.
problem Exponential speed-up in Bayesian computation.
method Quantum von Neumann measurement for simulating ML algorithms.
result Quantum versions of regression, Gaussian processes, and SGD.
In this paper, we provide an overview of first-order and second-order variants of the gradient descent method that are commonly used in machine learning. We propose a general framework in which 6 of these variants can be interpreted as different instances of the same approach. They are the vanilla gradient descent, the…
Natural gradient descent is an optimization method traditionally motivated from the perspective of information geometry, and works well for many applications as an alternative to stochastic gradient descent. In this paper we critically analyze this method and its properties, and show how it can be viewed as a type of 2…
We develop a more efficient NGD method for structured parameters.
problem Computational challenges in NGD for structured parameter spaces.
method Local-parameter coordinates to simplify Fisher-matrix computations.
result New structured second-order algorithms and learning methods.
Gradient descent solves robust mean estimation in high dimensions.
problem High-dimensional robust mean estimation in the presence of adversarial outliers.
method Gradient descent with a structural lemma showing near-optimal solutions.
result Gradient descent can solve the robust mean estimation problem directly.
Natural gradient descent, which preconditions a gradient descent update with the Fisher information matrix of the underlying statistical model, is a way to capture partial second-order information. Several highly visible works have advocated an approximation known as the empirical Fisher, drawing connections between ap…