We propose a modified -divergence, give some of its properties, and show that this leads to the definition of a generalized Fisher information. We give generalized Cramér-Rao inequalities, involving this Fisher information, an extension of the Fisher information matrix, and arbitrary norms and power of the estimat…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Fisher error”
New method for natural policy gradients converges linearly.
problem Improving natural policy gradient methods for better convergence.
method Fisher-Rao gradient flow applied to state-action distributions.
result Linear convergence rate with geometry-dependent factor.
Training-free source selection for LLM families with shared vocabularies
problem Source selection for LLM families with shared vocabularies
method Fisher alignment at vocabulary scale
result Fisher alignment is a cosine between kernel mean embeddings in the joint activation-error space
We identify spectral conditions for reliable neural probe interpretation.
problem Unreliable performance of linear probes in interpreting neural representations.
method Formalized Spectral Identifiability Principle (SIP) based on eigengap and Fisher error.
result Reliability of neural probes depends on the eigengap relative to Fisher estimation error.
Data processing inequalities link Fisher information to local differential privacy constraints.
problem Understanding how Fisher information scales with local differential privacy constraints.
method Developed data processing inequalities for Fisher information under local differential privacy.
result Implications for private estimation with optimal bounds and error rates.
The paper solves optimal bounds for separating data points in high dimensions.
problem Correcting AI errors and analyzing vulnerabilities in high-dimensional data.
method General stochastic separation theorems with optimal probability estimates.
result Explicit and optimal estimates of separation probabilities for important classes of distributions.
Paper improves Fisher information estimation methods.
problem Estimating Fisher information for location parameters.
method Revisits and improves Bhattacharya estimator, introduces clipped estimator.
result Clipped estimator shows superior convergence rates in Gaussian noise.
Bayesian inference typically requires the computation of an approximation to the posterior distribution. An important requirement for an approximate Bayesian inference algorithm is to output high-accuracy posterior mean and uncertainty estimates. Classical Monte Carlo methods, particularly Markov Chain Monte Carlo, rem…
FedFisher improves one-shot FL by using Fisher information.
problem Reducing communication rounds in federated learning.
method Bayesian perspective, Fisher information matrices, diagonal Fisher, K-FAC approximation.
result FedFisher achieves vanishingly small error in two-layer neural networks.
Modified training direction reduces generalization error in neural networks.
problem Reducing generalization error in neural networks.
method Theoretical analysis of modified natural gradient descent in function space.
result Modifying training direction in function space reduces total generalization error.
New tensor framework connects Fisher information, hypergraphs, and multi-observable correlations.
problem Missing structure in pairwise Fisher graphs for multi-observable radiation patterns.
method Higher-order Fisher tensors and natural exponential-family coordinates.
result Exact triality of Fisher tensors, cumulants, and hypergraphs.
Paper proposes efficient method to calculate Fisher-Bingham distribution normalizing constant.
problem Efficiently calculating the normalizing constant of Fisher-Bingham distributions.
method Numerical integration with continuous Euler transform to Fourier-type integral representation.
result The method is fast and accurate, applicable to high-dimensional distributions.
Improved location estimation for high-dimensional data with finite sample size.
problem Estimating the shift in high-dimensional data with limited samples.
method Smoothed estimators and bounds on subgamma vectors.
result Convergence to Cramér-Rao bound for finite sample sizes.
Optimal ability estimation in adaptive testing with binary responses.
problem Estimating a continuous ability parameter from sequential binary responses.
method Adaptive selection of questions to maximize Fisher information, updating estimate using method-of-moments, and deciding accuracy with a test statistic.
result Fisher-tracking strategy achieves optimal performance in fixed-confidence and fixed-budget regimes.
The Fisher information approximation (FIA) is an implementation of the minimum description length principle for model selection. Unlike information criteria such as AIC or BIC, it has the advantage of taking the functional form of a model into account. Unfortunately, FIA can be misleading in finite samples, resulting i…
Automates debiasing for large language model evaluations through Fisher random walk.
problem Rigorous and scalable evaluation of large language models.
method Semiparametric efficient estimator using Fisher random walk for weighted residual balancing.
result Efficient estimation of contextual preference scores for large language models.
We study the relationship between geometry and capacity measures for deep neural networks from an invariance viewpoint. We introduce a new notion of capacity --- the Fisher-Rao norm --- that possesses desirable invariance properties and is motivated by Information Geometry. We discover an analytical characterization of…
Training on some out-of-distribution data improves generalization error before it deteriorates.
problem Generalization error improves with some out-of-distribution data but deteriorates with more.
method Synthetic datasets and deep networks on computer vision benchmarks.
result Non-monotonic trend in generalization error with OOD samples.
Fisher Auto-Encodersstat.ML
Fisher auto-encoders use Fisher divergence for more robust generative modeling.
problem Model uncertainty in generative models.
method Minimizing Fisher divergence between true and modeled joint distributions.
result Fisher auto-encoders can more accurately quantify model uncertainty.
Natural gradient descent, which preconditions a gradient descent update with the Fisher information matrix of the underlying statistical model, is a way to capture partial second-order information. Several highly visible works have advocated an approximation known as the empirical Fisher, drawing connections between ap…
Paper defines Fisher co-metric on cotangent bundle and clarifies its relation to variance.
problem Defines Fisher co-metric on cotangent bundle and clarifies its relation to variance.
method Defines Fisher co-metric directly from Fisher metric without going through tangent bundle, using a natural correspondence between cotangent vectors and random variables.
result Clarifies the relation between Fisher co-metric and variance/covariance, trivializing the Cramér-Rao inequality.
Paper establishes limits for accurately estimating low-rank matrices from noisy, non-linear data.
problem Estimating low-rank matrices from noisy, non-linear observations.
method Proves strong universality result with equivalent Gaussian model and effective prior parameters.
result Signal-to-noise ratio requirement grows as $N^{ rac 12 (1-1/k_F)}$ for accurate reconstruction.
In this communication, we describe some interrelations between generalized -entropies and a generalized version of Fisher information. In information theory, the de Bruijn identity links the Fisher information and the derivative of the entropy. We show that this identity can be extended to generalized versions of en…
Market strategies minimize Fisher information to minimize risk.
problem Applying minimum Fisher information principle to market dynamics.
method Analytical extension to quantum harmonic oscillator eigenstates and Gibbs distribution.
result Minimizing Fisher information reduces information and risk.
Algebraic topology methods have recently played an important role for statistical analysis with complicated geometric structured data such as shapes, linked twist maps, and material data. Among them, \textit{persistent homology} is a well-known tool to extract robust topological features, and outputs as \textit{persist…
Fisher GANcs.LG
Generative Adversarial Networks (GANs) are powerful models for learning complex distributions. Stable training of GANs has been addressed in many recent works which explore different metrics between distributions. In this paper we introduce Fisher GAN which fits within the Integral Probability Metrics (IPM) framework f…
The study examines Fisher-Riemann geodesics for nonparametric probability densities.
problem Understanding nonparametric probability densities using Fisher-Riemann geometry.
method Obtaining Fisher-Riemann geodesics as a limit of parametric cases with increasing parameters.
result The weak limit approach for nonparametric probability densities.
Blog post discusses various implementations of Fisher Information for EWC in continual learning.
problem Improving Elastic Weight Consolidation (EWC) results by optimizing Fisher Information computation.
method Empirically compares different implementations of Fisher Information for EWC.
result Many reported EWC results can be improved by changing Fisher Information computation methods.
Fisher width is a geometric measure of complexity on statistical manifolds.
problem Complexity measures on statistical manifolds
method Introducing Fisher width as a Fisher-geometric analogue of Gaussian width
result Fisher width retains key structural features of Gaussian width while capturing anisotropic geometric effects
The paper studies metrics on Lie groups SO(2) and SO(3) and their integrability.
problem Integrability of gradient systems on Lie groups via Fisher metrics.
method Analysis of Souriau-Fisher metrics and 2-cocycles on Lie groups SO(2) and SO(3).
result Cocycles can locally modify Fisher metrics on Lie group orbits.
This paper shows any Kähler metric can be a Fisher information metric.
problem Establishing a new characterization of Kähler and coKähler manifolds.
method Statistical approach using Fisher information and exponential families.
result Any Kähler metric is a Fisher information metric.
Proofs Fisher-Rao distance on Gaussian covariance manifold.
problem Proving Fisher-Rao distance on Gaussian covariance manifold.
method Basic Riemannian geometry.
result Proof of Fisher-Rao distance on covariance cone.
Survey on closed-form Fisher-Rao distance expressions.
problem Finding closed-form expressions for Fisher-Rao distance.
method Collect and present examples of closed-form expressions for Fisher-Rao distance of discrete and continuous distributions.
result Presentation of closed-form expressions for Fisher-Rao distance of various distributions.
New statistics are introduced that maintain the Fisher metric structure closely, akin to sufficient statistics.
problem Maintaining the Fisher metric structure in statistical models.
method Characterizing statistics that maintain the Fisher metric structure bi-Lipschitz equivalently.
result Characterized statistics that preserve the Fisher metric structure closely.
This work proposes external correctors for quick AI error corrections without system modification.
problem Quick corrections of AI errors without modifying legacy systems.
method Special `external` devices with classifiers for high-dimensional data.
result Simple classifiers can correct small errors in high-dimensional data.
We consider three different approaches to define natural Riemannian metrics on polytopes of stochastic matrices. First, we define a natural class of stochastic maps between these polytopes and give a metric characterization of Chentsov type in terms of invariance with respect to these maps. Second, we consider the Fish…
Fisher score is one of the most widely used supervised feature selection methods. However, it selects each feature independently according to their scores under the Fisher criterion, which leads to a suboptimal subset of features. In this paper, we present a generalized Fisher score to jointly select features. It aims …
The study finds that memorization is necessary or harmful depending on the prior distribution and noise level.
problem The impact of memorization on generalization in overparameterized models.
method An overparameterized linear model with general priors in a Bayesian setup.
result Explicit conditions for optimal generalization based on the prior distribution and noise level.
We introduce Fisher consistency in the sense of unbiasedness as a desirable property for estimators of class prior probabilities. Lack of Fisher consistency could be used as a criterion to dismiss estimators that are unlikely to deliver precise estimates in test datasets under prior probability and more general dataset…
Bringing errors into focusmath.PR
This lecture presents recent advances in the theory of errors propagation. We first explain in which cases the propagation of errors may be performed with a first order differential calculus or needs a second order differential calculus. Then we point out the link between error propagation and the concept of second ord…
Optimal convex loss function improves regression coefficient estimation.
problem Asymptotic variance improvement in linear regression estimation.
method Score matching extension for log-concave projection.
result Semiparametric estimator attains minimal asymptotic covariance.
In information theory, Fisher information and Shannon information (entropy) are respectively used to quantify the uncertainty associated with the distribution modeling and the uncertainty in specifying the outcome of given variables. These two quantities are complementary and are jointly applied to information behavior…
Study on geometry of Dirichlet distributions using Fisher-Rao metric.
problem Understanding the geometry of Dirichlet distributions.
method Analysis of Fisher-Rao metric on Dirichlet distribution parameter space.
result Geodesic completeness and negative sectional curvature of the space.
A deep neural network is a hierarchical nonlinear model transforming input signals to output signals. Its input-output relation is considered to be stochastic, being described for a given input by a parameterized conditional probability distribution of outputs. The space of parameters consisting of weights and biases i…
The Delta method is a classical procedure for quantifying epistemic uncertainty in statistical models, but its direct application to deep neural networks is prevented by the large number of parameters . We propose a low cost variant of the Delta method applicable to -regularized deep neural networks based on th…
Paper formulates particle flow using variational inference and Fisher-Rao gradient flow.
problem Estimating posterior densities in probabilistic models.
method Variational formulation of particle flow, Fisher-Rao gradient flow, Gaussian and Gaussian mixture approximations.
result Gaussian and Gaussian mixture approximations of Fisher-Rao particle flow reduce to Exact Daum and Huang particle flow under linear Gaussian assumptions.
Paper discusses the Fisher metric and differentiability in statistical models.
problem Understanding the relationship between Fisher metric and differentiability in statistical models.
method Comparison of different concepts and models in Information Geometry, mathematical statistics, and measure theory.
result Discussion of various models and their differentiability properties.
TopoFisher learns topological summaries by maximizing Fisher information, improving parameter efficiency and inference quality.
problem Simulation-based inference misses key information in low-order statistics, especially for non-Gaussian fields.
method TopoFisher uses a differentiable persistent-homology pipeline that learns topological summaries by maximizing local Gaussian Fisher information.
result TopoFisher recovers much of the available information and outperforms fixed topological vectorizations in weak gravitational lensing.