arXiv research

Develops an analytic theory for quantum imaginary time evolution.

The probability distribution function (PDF) for prices on financial markets is derived by extremization of Fisher information. It is shown how on that basis the quantum-like description for financial markets arises and different financial market models are mapped by quantum mechanical ones.

Combining insights from machine learning and quantum Monte Carlo, the stochastic reconfiguration method with neural network Ansatz states is a promising new direction for high-precision ground state estimation of quantum many-body problems. Even though this method works well in practice, little is known about the learn…

Market strategies minimize Fisher information to minimize risk.

A deep neural network is a hierarchical nonlinear model transforming input signals to output signals. Its input-output relation is considered to be stochastic, being described for a given input by a parameterized conditional probability distribution of outputs. The space of parameters consisting of weights and biases i…

Machine learning approximates phase transitions using Fisher information.

We introduce DQFIM to quantify and improve generalization of quantum machine learning models.

NG+ method improves deep learning efficiency and accuracy.

Noise can affect the overparametrization of QNNs, enabling new directions but also suppressing sensitivity.

Overparametrization improves QNN trainability by reducing spurious local minima.

Two Fisher information matrix estimators are analyzed for neural networks, focusing on their variances and trade-offs.

We propose a modified $χ^β$-divergence, give some of its properties, and show that this leads to the definition of a generalized Fisher information. We give generalized Cramér-Rao inequalities, involving this Fisher information, an extension of the Fisher information matrix, and arbitrary norms and power of the estimat…

In a graph convolutional network, we assume that the graph $G$ is generated wrt some observation noise. During learning, we make small random perturbations $ΔG$ of the graph and try to improve generalization. Based on quantum information geometry, $ΔG$ can be characterized by the eigendecomposition of the graph Laplaci…

Natural gradient descent, which preconditions a gradient descent update with the Fisher information matrix of the underlying statistical model, is a way to capture partial second-order information. Several highly visible works have advocated an approximation known as the empirical Fisher, drawing connections between ap…

Quantum machine learning tackles large datasets with randomized measurements.

The Fisher-Rao geometry is applied to elliptical distributions for optimization and classification.

We introduce a notion of "effective dimension" of a statistical model based on the number of cubes of size $1/\sqrt{n}$ needed to cover the model space when endowed with the Fisher Information Matrix as metric, $n$ being the number of observations. The number of observations fixes a natural scale or resolution. The eff…

Paper identifies key function spaces for ReLU networks based on Fisher information.

We introduce RSE to measure robustness in estimation problems.

A family of probability distributions parametrized by an open domain $Λ$ in $R^n$ defines the Fisher information matrix on this domain which is positive semi-definite. In information geometry the standard assumption has been that the Fisher information matrix tensor is positive definite defining in this way a Riemannia…

This paper solves the intractability barrier in non-parametric information geometry by introducing a novel framework.

Let (M,g) be a compact, connected and oriented Riemannian manifold. We denote D the space of smooth probability density functions on M. In this paper, we show that the Frechet manifold D is equipped with a Riemannian metric g^{D} and an affine connection \nabla^{D} which are infinite dimensional analogues of the Fisher…

A new method improves uncertainty estimation in deep learning, especially for hard-to-label samples.

Develops a new geometric framework for quantum metrics.

Early training phase affects deep neural network optimization and generalization.

We analyze the variance of Fisher information estimators in deep learning models.

We study two types of preconditioners and preconditioned stochastic gradient descent (SGD) methods in a unified framework. We call the first one the Newton type due to its close relationship to the Newton method, and the second one the Fisher type as its preconditioner is closely related to the inverse of Fisher inform…

Estimates metric tensor on neuromanifolds using Fisher information and random methods.

Kernel networks' stability edge linked to Fisher Information singularity.

Optimal preconditioning improves Langevin sampling efficiency.

Develops information geometry for Lévy processes in finance.

Natural gradient descent is an optimization method traditionally motivated from the perspective of information geometry, and works well for many applications as an alternative to stochastic gradient descent. In this paper we critically analyze this method and its properties, and show how it can be viewed as a type of 2…

One way to avoid overfitting in machine learning is to use model parameters distributed according to a Bayesian posterior given the data, rather than the maximum likelihood estimator. Stochastic gradient Langevin dynamics (SGLD) is one algorithm to approximate such Bayesian posteriors for large models and datasets. SGL…

This paper analyzes Barlow Twins' representation efficiency using information-geometric methods.

The Riemannian Bures metric on the space of (normalized) complex positive matrices is used for parameter estimation of mixed quantum states based on repeated measurements just as the Fisher information in classical statistics. It appears also in the concept of purifications of mixed states in quantum physics. Here we d…

Unified approach for learning quantum operations from measurements.

A new iterative K-FAC algorithm reduces training time and memory usage.

New tensor framework connects Fisher information, hypergraphs, and multi-observable correlations.

Paper presents a rank-1 approximation method for natural policy gradients in deep RL.

New methods improve Fisher Matrix approximations for neural networks at low cost.

We address the information content of European option prices about volatility in terms of the Fisher information matrix. We assume that observed option prices are centred on the theoretical price provided by Heston's model disturbed by additive Gaussian noise. We fit the likelihood function on the components of the VIX…

Many deep learning models are vulnerable to the adversarial attack, i.e., imperceptible but intentionally-designed perturbations to the input can cause incorrect output of the networks. In this paper, using information geometry, we provide a reasonable explanation for the vulnerability of deep learning models. By consi…

The paper analyzes how quantization affects the Fisher Information Matrix's dominant eigenvalue.

Deep neural networks reveal a low-dimensional manifold structure in data.

The space of probability densities is an infinite-dimensional Riemannian manifold, with Riemannian metrics in two flavors: Wasserstein and Fisher--Rao. The former is pivotal in optimal mass transport (OMT), whereas the latter occurs in information geometry---the differential geometric approach to statistics. The Rieman…

New algorithm improves efficiency of quantum system modeling.

We formulate and solve a tensor model using a latent-variable approach.

This short note reviews so-called Natural Gradient Descent (NGD) for multivariate Gaussians. The Fisher Information Matrix (FIM) is derived for several different parameterizations of Gaussians. Careful attention is paid to the symmetric nature of the covariance matrix when calculating derivatives. We show that there ar…