arXiv research

The paper proposes a novel MKL approach for OCC using $\ell_p$-norm constraints.

ZDP detects drift in large language models without labels, proving key theorems and metrics.

The kernel null-space technique and its regression-based formulation (called one-class kernel spectral regression, a.k.a. OC-KSR) is known to be an effective and computationally attractive one-class classification framework. Despite its outstanding performance, the applicability of kernel null-space method is limited d…

The one-class kernel spectral regression (OC-KSR), the regression-based formulation of the kernel null-space approach has been found to be an effective Fisher criterion-based methodology for one-class classification (OCC), achieving state-of-the-art performance in one-class classification while providing relatively hig…

Researchers use information geometry to analyze and improve DRWs for node classification.

Low-rank matrix recovery has found many applications in science and engineering such as machine learning, signal processing, collaborative filtering, system identification, and Euclidean embedding. But the low-rank matrix recovery problem is an NP hard problem and thus challenging. A commonly used heuristic approach is…

New findings on curvature and null spaces of Laplacians.

Bayesian deep learning avoids underfitting by projecting onto null space of generalized Gauss-Newton matrix.

In this paper, we carry out null space analysis for Class-Specific Discriminant Analysis (CSDA) and formulate a number of solutions based on the analysis. We analyze both theoretically and experimentally the significance of each algorithmic step. The innate subspace dimensionality resulting from the proposed solutions …

Study reveals hidden null components in overparametrized neural networks.

There is a class of Laplacian like conformally invariant differential operators on differential forms $L^\ell_k$ which may be considered the generalisation to differential forms of the conformally invariant powers of the Laplacian known as the Paneitz and GJMS operators. On conformally Einstein manifolds we give explic…

Integrates outlier detection into neural networks for improved performance.

The paper addresses fairness in machine learning models through structural econometrics, projecting indexes into null spaces to find fair solutions.

In the Compressed Sensing community, it is well known that given a matrix $X \in \mathbb R^{n\times p}$ with $\ell_2$ normalized columns, the Restricted Isometry Property (RIP) implies the Null Space Property (NSP). It is also well known that a small Coherence $μ$ implies a weak RIP, i.e. the singular values of $X_T$ l…

Nuclear norm minimization (NNM) has recently gained significant attention for its use in rank minimization problems. Similar to compressed sensing, using null space characterizations, recovery thresholds for NNM have been studied in \cite{arxiv,Recht_Xu_Hassibi}. However simulations show that the thresholds are far fro…

We propose a meta-learning algorithm utilizing a linear transformer that carries out null-space projection of neural network outputs. The main idea is to construct an alternative classification space such that the error signals during few-shot learning are quickly zero-forced on that space so that reliable classificati…

Machine learning identifies chimera states in complex dynamical systems.

Interior tomography for the region-of-interest (ROI) imaging has advantages of using a small detector and reducing X-ray radiation dose. However, standard analytic reconstruction suffers from severe cupping artifacts due to existence of null space in the truncated Radon transform. Existing penalized reconstruction meth…

Physics-informed neural networks improve by measuring effective dimensionality of constraints.

In this paper, we consider surfaces in 4--dimensional pseudo--Riemannian space--forms with index 2. First, we obtain some of geometrical properties of such surfaces considering their relative null space. Then, we get classifications of quasi--minimal surfaces with positive relative nullity.

Study on Gaussian-width complexity on statistical manifolds and its applications in learning and recovery.

Fisher auto-encoders use Fisher divergence for more robust generative modeling.

Natural gradient descent, which preconditions a gradient descent update with the Fisher information matrix of the underlying statistical model, is a way to capture partial second-order information. Several highly visible works have advocated an approximation known as the empirical Fisher, drawing connections between ap…

Paper defines Fisher co-metric on cotangent bundle and clarifies its relation to variance.

In this communication, we describe some interrelations between generalized $q$-entropies and a generalized version of Fisher information. In information theory, the de Bruijn identity links the Fisher information and the derivative of the entropy. We show that this identity can be extended to generalized versions of en…

Market strategies minimize Fisher information to minimize risk.

Generative Adversarial Networks (GANs) are powerful models for learning complex distributions. Stable training of GANs has been addressed in many recent works which explore different metrics between distributions. In this paper we introduce Fisher GAN which fits within the Integral Probability Metrics (IPM) framework f…

The study examines Fisher-Riemann geodesics for nonparametric probability densities.

We propose a modified $χ^β$-divergence, give some of its properties, and show that this leads to the definition of a generalized Fisher information. We give generalized Cramér-Rao inequalities, involving this Fisher information, an extension of the Fisher information matrix, and arbitrary norms and power of the estimat…

Blog post discusses various implementations of Fisher Information for EWC in continual learning.

Fisher width is a geometric measure of complexity on statistical manifolds.

The paper studies metrics on Lie groups SO(2) and SO(3) and their integrability.

For even dimensional conformal manifolds several new conformally invariant objects were found recently: invariant differential complexes related to, but distinct from, the de Rham complex (these are elliptic in the case of Riemannian signature); the cohomology spaces of these; conformally stable form spaces that we may…

This paper shows any Kähler metric can be a Fisher information metric.

Proofs Fisher-Rao distance on Gaussian covariance manifold.

Survey on closed-form Fisher-Rao distance expressions.

Study improves distributed linear estimation under adversarial conditions.

In this paper, we propose new efficient algorithms to verify the null space condition in compressed sensing (CS). Given an $(n-m) \times n$ ($m>0$) CS matrix $A$ and a positive $k$, we are interested in computing $\displaystyle α_k = \max_{\{z: Az=0,z\neq 0\}}\max_{\{K: |K|\leq k\}}$ ${\|z_K \|_{1}}{\|z\|_{1}}$, where …

New statistics are introduced that maintain the Fisher metric structure closely, akin to sufficient statistics.

We consider three different approaches to define natural Riemannian metrics on polytopes of stochastic matrices. First, we define a natural class of stochastic maps between these polytopes and give a metric characterization of Chentsov type in terms of invariance with respect to these maps. Second, we consider the Fish…

Fisher score is one of the most widely used supervised feature selection methods. However, it selects each feature independently according to their scores under the Fisher criterion, which leads to a suboptimal subset of features. In this paper, we present a generalized Fisher score to jointly select features. It aims …

We introduce Fisher consistency in the sense of unbiasedness as a desirable property for estimators of class prior probabilities. Lack of Fisher consistency could be used as a criterion to dismiss estimators that are unlikely to deliver precise estimates in test datasets under prior probability and more general dataset…

Learning non-linear systems from noisy, limited, and/or dependent data is an important task across various scientific fields including statistics, engineering, computer science, mathematics, and many more. In general, this learning task is ill-posed; however, additional information about the data's structure or on the …

In information theory, Fisher information and Shannon information (entropy) are respectively used to quantify the uncertainty associated with the distribution modeling and the uncertainty in specifying the outcome of given variables. These two quantities are complementary and are jointly applied to information behavior…

Study on geometry of Dirichlet distributions using Fisher-Rao metric.

A deep neural network is a hierarchical nonlinear model transforming input signals to output signals. Its input-output relation is considered to be stochastic, being described for a given input by a parameterized conditional probability distribution of outputs. The space of parameters consisting of weights and biases i…

Paper formulates particle flow using variational inference and Fisher-Rao gradient flow.

Paper discusses the Fisher metric and differentiability in statistical models.