Develops an analytic theory for quantum imaginary time evolution.
problem Lack of a first-principle understanding of quantum imaginary time evolution.
method Interprets QITE as a form of VQA trained with QNGD and connects it to the geometric geodesic distance in the quantum Fisher information metric.
result QITE converges faster than vanilla gradient descent-based VQAs, though the advantage is suppressed by Hilbert space dimensionality.
The probability distribution function (PDF) for prices on financial markets is derived by extremization of Fisher information. It is shown how on that basis the quantum-like description for financial markets arises and different financial market models are mapped by quantum mechanical ones.
Combining insights from machine learning and quantum Monte Carlo, the stochastic reconfiguration method with neural network Ansatz states is a promising new direction for high-precision ground state estimation of quantum many-body problems. Even though this method works well in practice, little is known about the learn…
Market strategies minimize Fisher information to minimize risk.
problem Applying minimum Fisher information principle to market dynamics.
method Analytical extension to quantum harmonic oscillator eigenstates and Gibbs distribution.
result Minimizing Fisher information reduces information and risk.
A deep neural network is a hierarchical nonlinear model transforming input signals to output signals. Its input-output relation is considered to be stochastic, being described for a given input by a parameterized conditional probability distribution of outputs. The space of parameters consisting of weights and biases i…
Machine learning approximates phase transitions using Fisher information.
problem Understanding phase transitions from data using machine learning.
method Information geometry and Fisher information.
result Machine learning indicators approximate the square root of Fisher information.
We introduce DQFIM to quantify and improve generalization of quantum machine learning models.
problem Understanding and improving generalization of quantum machine learning models.
method Data quantum Fisher information metric (DQFIM) to quantify circuit parameters and training data.
result Improves generalization by breaking symmetries of training data and using a low number of training states.
NG+ method improves deep learning efficiency and accuracy.
problem Efficiency and accuracy in deep learning models.
method Proposes NG+ method using matrix-product natural gradient approach.
result Established global convergence and provided regret bound.
Noise can affect the overparametrization of QNNs, enabling new directions but also suppressing sensitivity.
problem The overparametrization of QNNs in the presence of noise.
method Analyzing the Quantum Fisher Information Matrix (QFIM) to understand how noise affects the rank of QFIM.
result Noise can turn previously-zero eigenvalues of the QFIM to non-zero, enabling exploration of new directions.
A new measure of model complexity based on Fisher Information.
problem Model complexity measurement in statistical models.
method Effective dimension defined by the number of cubes needed to cover the model space.
result The effective dimension is scale-dependent and measures model complexity.
Overparametrization improves QNN trainability by reducing spurious local minima.
problem Understanding how overparametrization affects the loss landscape of QNNs.
method Rigorous analysis of overparametrization in QNNs with periodic structure.
result Overparametrization corresponds to a computational phase transition improving QNN trainability.
Two Fisher information matrix estimators are analyzed for neural networks, focusing on their variances and trade-offs.
problem Estimating the Fisher information matrix in neural networks due to its high computational cost.
method Examined two popular diagonal Fisher information matrix estimators and their variances in neural networks for regression and classification.
result The variances of the estimators depend on the non-linearity with respect to different parameter groups and should not be neglected.
NGD improves multivariate Gaussian inference by optimizing Fisher information.
problem Efficiently optimizing multivariate Gaussian models.
method Natural Gradient Descent applied to multivariate Gaussian parameters.
result NGD updates are more efficient for symmetric covariance matrices.
We propose a modified χβ-divergence, give some of its properties, and show that this leads to the definition of a generalized Fisher information. We give generalized Cramér-Rao inequalities, involving this Fisher information, an extension of the Fisher information matrix, and arbitrary norms and power of the estimat…
Noise-resilient optimization on noisy quantum computers.
problem Noise's impact on hybrid quantum-classical optimization.
method Iterative quantum circuit with noise consideration, using Quantum Fisher Information bound.
result Algorithm robustness against different noise strengths.
In a graph convolutional network, we assume that the graph G is generated wrt some observation noise. During learning, we make small random perturbations ΔG of the graph and try to improve generalization. Based on quantum information geometry, ΔG can be characterized by the eigendecomposition of the graph Laplaci…
Natural gradient descent, which preconditions a gradient descent update with the Fisher information matrix of the underlying statistical model, is a way to capture partial second-order information. Several highly visible works have advocated an approximation known as the empirical Fisher, drawing connections between ap…
Quantum machine learning tackles large datasets with randomized measurements.
problem Efficiently process large, high-dimensional datasets on quantum computers.
method Randomized measurements to scale linearly with dataset size and quadratic for post-processing.
result Substantial speed-up for noisy quantum computers, enabling image classification.
The Fisher-Rao geometry is applied to elliptical distributions for optimization and classification.
problem Optimizing and classifying covariance matrices using geometric tools.
method Riemannian optimization and intrinsic Cramér-Rao bounds.
result Geometric tools enhance covariance matrix estimation and classification.
Paper identifies key function spaces for ReLU networks based on Fisher information.
problem Understanding the structure of Fisher information matrices in ReLU networks.
method Spectral decomposition of Fisher information matrices, focusing on the first three eigenspaces.
result The first three eigenspaces account for 97.7% of the trace of the Fisher information matrix, corresponding to spherical harmonic functions of order ≤2.
We introduce RSE to measure robustness in estimation problems.
problem Estimating statistical models from observed data.
method Developed theory for spectral functions of measures to compute RSE.
result RSE reveals a reciprocal relationship with problem complexity.
A family of probability distributions parametrized by an open domain Λ in Rn defines the Fisher information matrix on this domain which is positive semi-definite. In information geometry the standard assumption has been that the Fisher information matrix tensor is positive definite defining in this way a Riemannia…
This paper solves the intractability barrier in non-parametric information geometry by introducing a novel framework.
problem The intractability barrier in non-parametric information geometry due to the Fisher-Rao metric being a functional.
method Introducing an Orthogonal Decomposition of the Tangent Space and deriving the Covariate Fisher Information Matrix (cFIM).
result Established a rigorous foundation for the G-entropy and provided fundamental limits of variance for semi-parametric estimators.
Let (M,g) be a compact, connected and oriented Riemannian manifold. We denote D the space of smooth probability density functions on M. In this paper, we show that the Frechet manifold D is equipped with a Riemannian metric g^{D} and an affine connection \nabla^{D} which are infinite dimensional analogues of the Fisher…
A new method improves uncertainty estimation in deep learning, especially for hard-to-label samples.
problem Improving uncertainty estimation for hard-to-label samples in deep learning.
method Introduces Fisher Information Matrix (FIM) to dynamically reweight objective loss terms.
result Consistently outperforms traditional evidential neural networks in uncertainty estimation tasks.
Develops a new geometric framework for quantum metrics.
problem Quantum metric generalization for pure two-qubit states.
method Support-projected Petz monotone geometry for pure two-qubit families.
result Strictly generalizes SLD/Bures case and includes other metrics.
Early training phase affects deep neural network optimization and generalization.
problem The choice of learning rate influences generalization in deep learning models.
method Showed that SGD implicitly penalizes the trace of the Fisher Information Matrix (FIM) from the start of training, and explicitly penalizing the trace of FIM improves generalization.
result Catastrophic Fisher explosion (large trace of FIM early in training) is linked to poor generalization.
We analyze the variance of Fisher information estimators in deep learning models.
problem Understanding the variance of Fisher information in deep learning models.
method Investigated two unbiased and consistent estimators of Fisher information matrix.
result The variance of estimators is influenced by the model's parametric structure.
We study two types of preconditioners and preconditioned stochastic gradient descent (SGD) methods in a unified framework. We call the first one the Newton type due to its close relationship to the Newton method, and the second one the Fisher type as its preconditioner is closely related to the inverse of Fisher inform…
Estimates metric tensor on neuromanifolds using Fisher information and random methods.
problem Computing the metric tensor on high-dimensional neuromanifolds efficiently and accurately.
method Deterministic bounds and unbiased random estimators based on Hutchinson's trace method.
result An efficient random estimator with bounded standard deviation.
Kernel networks' stability edge linked to Fisher Information singularity.
problem Understanding the stability edge in high-capacity kernel Hopfield networks.
method Statistical manifold analysis and Riemannian geometry.
result The Ridge of Optimization corresponds to the Edge of Stability, revealing a dual equilibrium.
Optimal preconditioning improves Langevin sampling efficiency.
problem Improving sampling efficiency in high-dimensional target distributions.
method Optimal preconditioning using Fisher information, applied to MALA.
result Adaptive MCMC scheme significantly outperforms other methods.
Develops information geometry for Lévy processes in finance.
problem Understanding the statistical properties of Lévy processes for financial modeling.
method Deriving α-divergences from Lévy triplets, identifying Fisher information matrix and α-connection. result Identifies statistical implications and differential-geometric structures of Lévy processes.
Natural gradient descent is an optimization method traditionally motivated from the perspective of information geometry, and works well for many applications as an alternative to stochastic gradient descent. In this paper we critically analyze this method and its properties, and show how it can be viewed as a type of 2…
One way to avoid overfitting in machine learning is to use model parameters distributed according to a Bayesian posterior given the data, rather than the maximum likelihood estimator. Stochastic gradient Langevin dynamics (SGLD) is one algorithm to approximate such Bayesian posteriors for large models and datasets. SGL…
This paper analyzes Barlow Twins' representation efficiency using information-geometric methods.
problem Understanding and comparing the efficiency of self-supervised learning methods.
method Introduces an information-geometric framework to quantify representation efficiency and applies it to Barlow Twins.
result Proves that Barlow Twins achieves optimal representation efficiency (η=1).
The Riemannian Bures metric on the space of (normalized) complex positive matrices is used for parameter estimation of mixed quantum states based on repeated measurements just as the Fisher information in classical statistics. It appears also in the concept of purifications of mixed states in quantum physics. Here we d…
Unified approach for learning quantum operations from measurements.
problem Accurate reconstruction of unknown quantum operations from noisy measurements.
method Matrix sensing techniques, randomized measurement design, blockwise measurement design, alternating least squares (ALS).
result The proposed method provides theoretical guarantees for the identifiability and recovery of low-rank superoperators in the presence of noise.
A new iterative K-FAC algorithm reduces training time and memory usage.
problem Training deep learning models efficiently.
method Uses conjugate gradient to approximate Fisher information matrix without generating the matrix or factors.
result Time and memory complexity of iterative CG-FAC is less than standard K-FAC.
New tensor framework connects Fisher information, hypergraphs, and multi-observable correlations.
problem Missing structure in pairwise Fisher graphs for multi-observable radiation patterns.
method Higher-order Fisher tensors and natural exponential-family coordinates.
result Exact triality of Fisher tensors, cumulants, and hypergraphs.
Paper presents a rank-1 approximation method for natural policy gradients in deep RL.
problem Computing natural gradients requires inverting the Fisher Information Matrix, which is computationally expensive.
method Develops a rank-1 approximation to the inverse Fisher Information Matrix for efficient natural policy optimization.
result The rank-1 approximation converges faster and has similar sample complexity to stochastic policy gradient methods.
New methods improve Fisher Matrix approximations for neural networks at low cost.
problem High cost of solving Fisher Information Matrix (FIM) in neural networks.
method Direct minimization via Kronecker product singular value decomposition.
result Improved approximations to FIM provide more accurate and faster optimization.
We address the information content of European option prices about volatility in terms of the Fisher information matrix. We assume that observed option prices are centred on the theoretical price provided by Heston's model disturbed by additive Gaussian noise. We fit the likelihood function on the components of the VIX…
Many deep learning models are vulnerable to the adversarial attack, i.e., imperceptible but intentionally-designed perturbations to the input can cause incorrect output of the networks. In this paper, using information geometry, we provide a reasonable explanation for the vulnerability of deep learning models. By consi…
The paper analyzes how quantization affects the Fisher Information Matrix's dominant eigenvalue.
problem The impact of quantization on the Fisher Information Matrix's dominant eigenvalue.
method The study examines spectral perturbation of the empirical Fisher Information Matrix under in-distribution input and quantized parameter perturbations.
result A bound on the eigenvalue under quantization noise, showing it strictly exceeds the unperturbed value at leading order.
Deep neural networks reveal a low-dimensional manifold structure in data.
problem Understanding the structure of data for better model performance.
method Model-centric analysis of the data manifold using the local data matrix and Fisher information matrix.
result The dataset lies on a data leaf with a dimension bounded by the number of labels.
The space of probability densities is an infinite-dimensional Riemannian manifold, with Riemannian metrics in two flavors: Wasserstein and Fisher--Rao. The former is pivotal in optimal mass transport (OMT), whereas the latter occurs in information geometry---the differential geometric approach to statistics. The Rieman…
New algorithm improves efficiency of quantum system modeling.
problem Intractable complexities in quantum Hamiltonian learning and Gibbs sampling.
method Generalized quantum natural gradient descent and Quantum-Probabilistic Mirror Descent.
result Data sample efficiency proven using information geometry and quantum metrology.