arXiv research

A quantum generalization of Natural Gradient Descent is presented as part of a general-purpose optimization framework for variational quantum circuits. The optimization dynamics is interpreted as moving in the steepest descent direction with respect to the Quantum Information Geometry, corresponding to the real part of…

Develops an analytic theory for quantum imaginary time evolution.

Analyzes dynamics of quantum neural networks, predicting exponential decay of training error.

New algorithm Momentum-QNG improves optimization of quantum circuits.

ECD algorithm speeds up non-convex optimization, offering quantum and stochastic enhancements.

A new optimizer saves significant shots in quantum machine learning.

Quantum systems learn like machine learning models, influenced by dissipation.

A quantum reinforcement learning algorithm reduces sample complexity.

Quantum networks offer exponential communication savings for large machine learning models.

New algorithm improves efficiency of quantum system modeling.

Researchers propose a non-monotone quantum natural gradient for quantum systems.

SGLBO optimizes quantum circuits with fewer measurements, improving accuracy and noise resilience.

Quantum machine learning faces 'laziness' and 'barren plateaus', but noise can mitigate the latter.

We study the projected gradient descent method on low-rank matrix problems with a strongly convex objective. We use the Burer-Monteiro factorization approach to implicitly enforce low-rankness; such factorization introduces non-convexity in the objective. We focus on constraint sets that include both positive semi-defi…

Natural gradient descent avoids the magic of model parametrization, leading to different optimization outcomes.

Quantum method speeds up VB estimation in machine learning.

Derives Mirror Descent from gradient flow on a Riemannian manifold.

We introduce two quantum algorithms for solving structured prediction problems. We first show that a stochastic gradient descent that uses the quantum minimum finding algorithm and takes its probabilistic failure into account solves the structured prediction problem with a runtime that scales with the square root of th…

Study evaluates capacity and trainability of parametrized quantum circuits.

New methods optimize training VQAs without barren plateaus, improving efficiency and applicability.

New methods using natural gradient for structured optimization.

Quantum kernel improves probabilistic time series forecasting.

We introduce a simple algorithm, True Asymptotic Natural Gradient Optimization (TANGO), that converges to a true natural gradient descent in the limit of small learning rates, without explicit Fisher matrix estimation. For quadratic models the algorithm is also an instance of averaged stochastic gradient, where the par…

This work proposes a new method for variational inference using Wasserstein gradient descent.

Quantum computing speeds up neural network training and retraining.

A new method speeds up quantum state estimation.

Study quantum aspects of 1-form symmetries using BV-BRST cohomology.

Study quantum aspects of 1-form symmetries using BV-BRST cohomology and gerbes.

Warm starts improve variational quantum algorithms by avoiding barren plateaus.

The Conant-Ashby theorem is verified for hypergraph observers, leading to unique learning rules.

Optimizes graph neural networks using natural gradient descent.

This paper investigates the relationship between algebraic quantum field theories and factorization algebras on globally hyperbolic Lorentzian manifolds. Functorial constructions that map between these two types of theories in both directions are developed under certain natural hypotheses, including suitable variants o…

Paper proposes ZO-NGD for more efficient black-box attacks.

Natural gradient learning improves synaptic plasticity in spiking neurons.

We present theoretical results on the convergence of \emph{non-convex} accelerated gradient descent in matrix factorization models with $\ell_2$-norm loss. The purpose of this work is to study the effects of acceleration in non-convex settings, where provable convergence with acceleration should not be considered a \em…

The multinomial logistic regression (MLR) model is widely used in statistics and machine learning. Stochastic gradient descent (SGD) is the most common approach for determining the parameters of a MLR model in big data scenarios. However, SGD has slow sub-linear rates of convergence. A way to improve these rates of con…

Natural gradient descent has proven effective at mitigating the effects of pathological curvature in neural network optimization, but little is known theoretically about its convergence properties, especially for \emph{nonlinear} networks. In this work, we analyze for the first time the speed of convergence of natural …

Quantum models show improved performance in overparameterized regimes.

This short note reviews so-called Natural Gradient Descent (NGD) for multivariate Gaussians. The Fisher Information Matrix (FIM) is derived for several different parameterizations of Gaussians. Careful attention is paid to the symmetric nature of the covariance matrix when calculating derivatives. We show that there ar…

Quantum field theory connects deep neural networks to criticality.

QBC uses quantum computers to speed up Bayesian computation.

In this paper, we provide an overview of first-order and second-order variants of the gradient descent method that are commonly used in machine learning. We propose a general framework in which 6 of these variants can be interpreted as different instances of the same approach. They are the vanilla gradient descent, the…

Natural gradient descent is an optimization method traditionally motivated from the perspective of information geometry, and works well for many applications as an alternative to stochastic gradient descent. In this paper we critically analyze this method and its properties, and show how it can be viewed as a type of 2…

We develop a more efficient NGD method for structured parameters.

Gradient descent solves robust mean estimation in high dimensions.

Natural gradient descent, which preconditions a gradient descent update with the Fisher information matrix of the underlying statistical model, is a way to capture partial second-order information. Several highly visible works have advocated an approximation known as the empirical Fisher, drawing connections between ap…

Unified analysis of online optimization with self-concordant barriers, improving regret bounds.

Gradient descent struggles with high-dimensional data fitting.