Discusses MultiFIT for multivariate dependence, comparing it to HSIC tests.
problem Comparing Multiscale Fisher's Independence Test (MultiFIT) to HSIC tests for multivariate dependence.
method Compares MultiFIT to HSIC tests, highlighting exact level control and performance limitations.
result Observes performance limitations of MultiFIT in terms of test power.
We introduce Fisher consistency in the sense of unbiasedness as a desirable property for estimators of class prior probabilities. Lack of Fisher consistency could be used as a criterion to dismiss estimators that are unlikely to deliver precise estimates in test datasets under prior probability and more general dataset…
Fisher score is one of the most widely used supervised feature selection methods. However, it selects each feature independently according to their scores under the Fisher criterion, which leads to a suboptimal subset of features. In this paper, we present a generalized Fisher score to jointly select features. It aims …
New discriminant analysis using GDS projection improves face recognition.
problem Improving face recognition accuracy with limited data.
method GDS projection onto generalized difference subspace, simplified Fisher criterion, normalization.
result GDS projection and gFDA are equivalent, inheriting FDA's discriminant ability.
Roy's `Safety First' criterion for selecting one risky asset from many is adapted to the case of non-normal returns, via Cornish Fisher expansion. The resulting investment objective is consistent with first order stochastic dominance, and is equal to the Sharpe ratio for the case of normal returns. An investor selectin…
New tensor framework connects Fisher information, hypergraphs, and multi-observable correlations.
problem Missing structure in pairwise Fisher graphs for multi-observable radiation patterns.
method Higher-order Fisher tensors and natural exponential-family coordinates.
result Exact triality of Fisher tensors, cumulants, and hypergraphs.
Information geometry provides a geometric approach to families of statistical models. The key geometric structures are the Fisher quadratic form and the Amari-Chentsov tensor. In statistics, the notion of sufficient statistic expresses the criterion for passing from one model to another without loss of information. Thi…
BC learning improves deep sound recognition performance.
problem Improving deep sound recognition using novel training data.
method BC learning: mixing sounds from different classes to generate between-class sounds and train models to recognize these.
result BC learning improves performance on various sound recognition networks, surpassing human level.
LS improves model selection for singular statistical models.
problem Challenges in model selection for singular statistical models.
method Integrates empirical loss from WAIC and sBIC penalty term.
result Enhanced utility for model selection without regularity constraints.
Mixture modelling involves explaining some observed evidence using a combination of probability distributions. The crux of the problem is the inference of an optimal number of mixture components and their corresponding parameters. This paper discusses unsupervised learning of mixture models using the Bayesian Minimum M…
Disputes the empirical Fisher approximation for natural gradient descent.
problem The empirical Fisher approximation fails to capture second-order information in general.
method Comparison of empirical Fisher and Fisher information matrices.
result The empirical Fisher does not generally approximate the Fisher or Hessian.
Modified training direction reduces generalization error in neural networks.
problem Reducing generalization error in neural networks.
method Theoretical analysis of modified natural gradient descent in function space.
result Modifying training direction in function space reduces total generalization error.
Study on Gaussian-width complexity on statistical manifolds and its applications in learning and recovery.
problem Understanding the geometry of statistical manifolds and its implications for learning and recovery.
method Analysis of Fisher width and inverse-Fisher width, proving their complementary roles and establishing a relation between them.
result Established a sharp relation between Fisher width and inverse-Fisher width, showing they cannot reduce relative to Euclidean scale.
Fisher auto-encoders use Fisher divergence for more robust generative modeling.
problem Model uncertainty in generative models.
method Minimizing Fisher divergence between true and modeled joint distributions.
result Fisher auto-encoders can more accurately quantify model uncertainty.
Paper defines Fisher co-metric on cotangent bundle and clarifies its relation to variance.
problem Defines Fisher co-metric on cotangent bundle and clarifies its relation to variance.
method Defines Fisher co-metric directly from Fisher metric without going through tangent bundle, using a natural correspondence between cotangent vectors and random variables.
result Clarifies the relation between Fisher co-metric and variance/covariance, trivializing the Cramér-Rao inequality.
Unified framework for Bayesian and Frequentist statistics.
problem Embedding Bayesian statistics within a broader decision-making framework.
method Game theory and statistical analysis.
result Statistical games unify Bayesian and Frequentist statistics.
Explains Fisher and Kernel Fisher Discriminant Analysis with examples and comparisons.
problem Classifying data with different features and dimensions.
method Projection and reconstruction, scatters analysis, PCA comparison, Fisher forest.
result Equivalence of Fisher and Linear Discriminant Analysis, effectiveness of Fisher forest.
In this communication, we describe some interrelations between generalized q-entropies and a generalized version of Fisher information. In information theory, the de Bruijn identity links the Fisher information and the derivative of the entropy. We show that this identity can be extended to generalized versions of en…
A fast feature selection method using OLS and SOCC for classification.
problem Feature selection for linear classification.
method Orthogonal Least Squares (OLS) with Squared Orthogonal Correlation Coefficient (SOCC).
result The proposed method outperforms other feature selection methods in speed and accuracy.
Paper introduces NICc for fast cluster-based validation of prediction models.
problem Validation of prediction models on clustered data.
method Derived NICc to approximate leave-one-cluster-out deviance for standard regression models.
result NICc provides more accurate model size and variable selection, especially with strong clustering.
This paper explores VAEs in Fisher-Shannon plane, revealing the relationship between Fisher information and Shannon entropy.
problem Understanding the relationship between Fisher information and Shannon entropy in VAEs.
method Investigation of VAEs in Fisher-Shannon plane, focusing on the trade-off between Fisher information and Shannon entropy.
result VAEs' representation learning and log-likelihood estimation are intrinsically related to Fisher information and Shannon entropy.
Market strategies minimize Fisher information to minimize risk.
problem Applying minimum Fisher information principle to market dynamics.
method Analytical extension to quantum harmonic oscillator eigenstates and Gibbs distribution.
result Minimizing Fisher information reduces information and risk.
Generative Adversarial Networks (GANs) are powerful models for learning complex distributions. Stable training of GANs has been addressed in many recent works which explore different metrics between distributions. In this paper we introduce Fisher GAN which fits within the Integral Probability Metrics (IPM) framework f…
We propose a modified χβ-divergence, give some of its properties, and show that this leads to the definition of a generalized Fisher information. We give generalized Cramér-Rao inequalities, involving this Fisher information, an extension of the Fisher information matrix, and arbitrary norms and power of the estimat…
The study examines Fisher-Riemann geodesics for nonparametric probability densities.
problem Understanding nonparametric probability densities using Fisher-Riemann geometry.
method Obtaining Fisher-Riemann geodesics as a limit of parametric cases with increasing parameters.
result The weak limit approach for nonparametric probability densities.
Blog post discusses various implementations of Fisher Information for EWC in continual learning.
problem Improving Elastic Weight Consolidation (EWC) results by optimizing Fisher Information computation.
method Empirically compares different implementations of Fisher Information for EWC.
result Many reported EWC results can be improved by changing Fisher Information computation methods.
Fisher width is a geometric measure of complexity on statistical manifolds.
problem Complexity measures on statistical manifolds
method Introducing Fisher width as a Fisher-geometric analogue of Gaussian width
result Fisher width retains key structural features of Gaussian width while capturing anisotropic geometric effects
The paper studies metrics on Lie groups SO(2) and SO(3) and their integrability.
problem Integrability of gradient systems on Lie groups via Fisher metrics.
method Analysis of Souriau-Fisher metrics and 2-cocycles on Lie groups SO(2) and SO(3).
result Cocycles can locally modify Fisher metrics on Lie group orbits.
This paper shows any Kähler metric can be a Fisher information metric.
problem Establishing a new characterization of Kähler and coKähler manifolds.
method Statistical approach using Fisher information and exponential families.
result Any Kähler metric is a Fisher information metric.
Proofs Fisher-Rao distance on Gaussian covariance manifold.
problem Proving Fisher-Rao distance on Gaussian covariance manifold.
method Basic Riemannian geometry.
result Proof of Fisher-Rao distance on covariance cone.
A statistical model or a learning machine is called regular if the map taking a parameter to a probability distribution is one-to-one and if its Fisher information matrix is always positive definite. If otherwise, it is called singular. In regular statistical models, the Bayes free energy, which is defined by the minus…
Survey on closed-form Fisher-Rao distance expressions.
problem Finding closed-form expressions for Fisher-Rao distance.
method Collect and present examples of closed-form expressions for Fisher-Rao distance of discrete and continuous distributions.
result Presentation of closed-form expressions for Fisher-Rao distance of various distributions.
The paper simplifies the Fisher information matrix for random deep networks, speeding up learning.
problem Learning deep neural networks efficiently with large parameter spaces.
method Statistical neurodynamical method to reveal Fisher information properties, proving unit-wise block diagonal structure and explicit inverse.
result Explicit natural gradient formula without matrix inversion, speeding up learning.
New statistics are introduced that maintain the Fisher metric structure closely, akin to sufficient statistics.
problem Maintaining the Fisher metric structure in statistical models.
method Characterizing statistics that maintain the Fisher metric structure bi-Lipschitz equivalently.
result Characterized statistics that preserve the Fisher metric structure closely.
We consider three different approaches to define natural Riemannian metrics on polytopes of stochastic matrices. First, we define a natural class of stochastic maps between these polytopes and give a metric characterization of Chentsov type in terms of invariance with respect to these maps. Second, we consider the Fish…
GANs can be used to extract Fisher vectors for unsupervised feature learning.
problem Unsupervised feature extraction for classification and similarity tasks.
method Derive Fisher Information and Fisher Vectors from GANs' density model.
result GAN-induced Fisher Vectors perform competitively in unsupervised feature extraction.
Study on geometry of Dirichlet distributions using Fisher-Rao metric.
problem Understanding the geometry of Dirichlet distributions.
method Analysis of Fisher-Rao metric on Dirichlet distribution parameter space.
result Geodesic completeness and negative sectional curvature of the space.
Paper formulates particle flow using variational inference and Fisher-Rao gradient flow.
problem Estimating posterior densities in probabilistic models.
method Variational formulation of particle flow, Fisher-Rao gradient flow, Gaussian and Gaussian mixture approximations.
result Gaussian and Gaussian mixture approximations of Fisher-Rao particle flow reduce to Exact Daum and Huang particle flow under linear Gaussian assumptions.
Paper discusses the Fisher metric and differentiability in statistical models.
problem Understanding the relationship between Fisher metric and differentiability in statistical models.
method Comparison of different concepts and models in Information Geometry, mathematical statistics, and measure theory.
result Discussion of various models and their differentiability properties.
TopoFisher learns topological summaries by maximizing Fisher information, improving parameter efficiency and inference quality.
problem Simulation-based inference misses key information in low-order statistics, especially for non-Gaussian fields.
method TopoFisher uses a differentiable persistent-homology pipeline that learns topological summaries by maximizing local Gaussian Fisher information.
result TopoFisher recovers much of the available information and outperforms fixed topological vectorizations in weak gravitational lensing.
Paper identifies key function spaces for ReLU networks based on Fisher information.
problem Understanding the structure of Fisher information matrices in ReLU networks.
method Spectral decomposition of Fisher information matrices, focusing on the first three eigenspaces.
result The first three eigenspaces account for 97.7% of the trace of the Fisher information matrix, corresponding to spherical harmonic functions of order ≤2.
A new measure of model complexity based on Fisher Information.
problem Model complexity measurement in statistical models.
method Effective dimension defined by the number of cubes needed to cover the model space.
result The effective dimension is scale-dependent and measures model complexity.
The Fisher-Rao geometry is applied to elliptical distributions for optimization and classification.
problem Optimizing and classifying covariance matrices using geometric tools.
method Riemannian optimization and intrinsic Cramér-Rao bounds.
result Geometric tools enhance covariance matrix estimation and classification.
Discriminative model identifies readers and assesses comprehension from eye movements.
problem Inferring readers' identities and estimating their text comprehension from eye movements.
method Generative model of gaze patterns, Fisher-score representation, Fisher-SVM with Fisher kernel.
result SVM with Fisher kernel excels at identifying readers, but not comprehending text.
The study examines Fisher information matrices and neural tangent kernels for simple ReLU networks with random weights.
problem Understanding the relationship between Fisher information matrices and neural tangent kernels for 2-layer ReLU networks.
method Analyzes Fisher information matrices and neural tangent kernels for 2-layer ReLU networks with random hidden weights, focusing on spectral decomposition and eigenfunctions.
result Obtained an approximation formula for functions represented by 2-layer neural networks.
New method for natural policy gradients converges linearly.
problem Improving natural policy gradient methods for better convergence.
method Fisher-Rao gradient flow applied to state-action distributions.
result Linear convergence rate with geometry-dependent factor.
Recently we proposed a general, ensemble-based feature engineering wrapper (FEW) that was paired with a number of machine learning methods to solve regression problems. Here, we adapt FEW for supervised classification and perform a thorough analysis of fitness and survival methods within this framework. Our tests demon…
Global gradient estimates for Fisher-KPP equation on Finsler metric measure spaces.
problem Establishing gradient estimates for the Finslerian Fisher-KPP equation.
method Global gradient estimates on compact and noncompact Finsler metric measure spaces using the traditional CD(K,N) condition and new comparison theorems. result Global gradient estimates for positive solutions of the Finslerian Fisher-KPP equation.