Explains Fisher and Kernel Fisher Discriminant Analysis with examples and comparisons.
problem Classifying data with different features and dimensions.
method Projection and reconstruction, scatters analysis, PCA comparison, Fisher forest.
result Equivalence of Fisher and Linear Discriminant Analysis, effectiveness of Fisher forest.
Optimal domain adaptation model using Fisher's Linear Discriminant.
problem Improving classification accuracy across different domains.
method Convex combination of source and target hypotheses, derived under 0-1 loss.
result Effective classifier can be computed without direct source task information.
Paper introduces new loss functions for Siamese networks using FDA.
problem Training Siamese networks with improved loss functions.
method Proposes Fisher Discriminant Triplet (FDT) and Fisher Discriminant Contrastive (FDC) loss functions based on FDA.
result Shows effectiveness of FDT and FDC on MNIST and histopathology datasets.
A new iterative algorithm improves RFDA for high-dimensional data.
problem High-dimensional data challenges conventional FDA and RFDA.
method Iterative sketching-based algorithm with accuracy guarantees.
result Accurate approximations can be achieved with smaller sample sizes.
Unified analysis of multilabel Fisher discriminants with improved dimensionality and robustness.
problem Improving discriminant analysis for multilabel classification with enhanced dimensionality and robustness.
method Unified theoretical analysis of multilabel Fisher discriminants with algebraic and statistical guarantees.
result Unified characterization of multilabel Fisher objectives and their equivalence under orthogonality constraints.
Unified analysis of multilabel Fisher discriminants with improved dimensionality and robustness.
problem Improving discriminant analysis for multilabel classification with enhanced dimensionality and robustness.
method Unified algebraic and statistical analysis of multilabel Fisher discriminants with Stiefel orthogonality constraints.
result Equivalence of four Fisher objectives under the Stiefel constraint and improved discriminant dimensionality.
We propose a Bayesian framework of Gaussian process in order to extend Fisher's discriminant to classify functional data such as spectra and images. The probability structure for our extended Fisher's discriminant is explicitly formulated, and we utilize the smoothness assumptions of functional data as prior probabilit…
Fisher's linear discriminant analysis (FLDA) is an important dimension reduction method in statistical pattern recognition. It has been shown that FLDA is asymptotically Bayes optimal under the homoscedastic Gaussian assumption. However, this classical result has the following two major limitations: 1) it holds only fo…
QFDA combines machine learning and information theory for image classification.
problem Lack of literature on combining machine learning and information theory.
method Quantized Fisher Discriminant Analysis (QFDA) using a cost function for rate-distortion optimization.
result QFDA achieves at least as good classification accuracy as FDA on quantized images.
Extends Fisher's Discriminant Analysis for interval-valued data.
problem Classifying entities represented by intervals and histograms.
method Adapts Fisher's Discriminant Analysis using Moore's interval arithmetic and Mallows' distance.
result Discriminant directions for interval-valued data are numerically maximized.
Researchers use information geometry to analyze and improve DRWs for node classification.
problem Lack of theoretical foundations for Discriminative Random Walks (DRWs).
method Revisit DRWs through information geometry, treating hitting-time laws as a statistical manifold. Derived closed-form expressions and introduced sensitivity scores.
result Introduced a sensitivity score that bounds maximal first-order change in DRW betweenness under unit Fisher perturbations.
New discriminant analysis using GDS projection improves face recognition.
problem Improving face recognition accuracy with limited data.
method GDS projection onto generalized difference subspace, simplified Fisher criterion, normalization.
result GDS projection and gFDA are equivalent, inheriting FDA's discriminant ability.
Adaptive classifier optimizes high-dimensional data with spiked covariance structure.
problem Classification of high-dimensional data with spiked covariance structure.
method Adaptive classifier that whitens data, screens features, and applies Fisher linear discriminant.
result The classifier is Bayes optimal under certain conditions and performs well on real and synthetic data.
GANs can be used to extract Fisher vectors for unsupervised feature learning.
problem Unsupervised feature extraction for classification and similarity tasks.
method Derive Fisher Information and Fisher Vectors from GANs' density model.
result GAN-induced Fisher Vectors perform competitively in unsupervised feature extraction.
Explains LDA and QDA for binary and multiple classes.
problem Classification methods in statistical and probabilistic learning.
method Optimization of decision boundaries, estimation of parameters, relation to other methods.
result Equivalence of LDA and Fisher discriminant analysis.
Survey of spectral, probabilistic, and deep metric learning methods.
problem Developing effective distance metrics for various machine learning tasks.
method Divided into spectral, probabilistic, and deep approaches, covering various techniques and their applications.
result Comprehensive overview of metric learning methods, including new developments and applications.
A new QDA classifier for high-dimensional data with spiked covariance.
problem Classifying high-dimensional data with distinct covariance matrices.
method Proposes a novel quadratic classification technique with parameters chosen to maximize the fisher-discriminant ratio.
result The proposed classifier outperforms classical R-QDA and requires lower computational complexity.
Fisher loss improves deep domain adaptation by learning discriminative within-class compact and between-class separable representations.
problem Improving deep domain adaptation performance by learning discriminative representations.
method Proposes a Fisher loss to learn discriminative representations that are within-class compact and between-class separable.
result Noticeable improvements in deep domain adaptation performance, e.g., 6.67% absolute improvement in mean accuracy on the Office-Home dataset.
Logit regularization induces logit clustering, affecting classifier performance.
problem Understanding the mechanism of logit regularization in classification.
method Analysis of logit regularization in linear classification, proving logit clustering leads to Fisher's Linear Discriminant alignment.
result Logit regularization can halve critical sample complexity and induce robust generalization.
Kernel discriminant analysis uses nonlinear embeddings to improve classification.
problem Limited effectiveness of linear discriminant analysis in capturing nonlinear features.
method Study of nonlinear embeddings in kernel discriminant analysis using polynomial and Gaussian kernels, solving generalized eigenvalue problems.
result Polynomial and Gaussian discriminants capture class differences through population moments and randomized projections.
Discriminative model identifies readers and assesses comprehension from eye movements.
problem Inferring readers' identities and estimating their text comprehension from eye movements.
method Generative model of gaze patterns, Fisher-score representation, Fisher-SVM with Fisher kernel.
result SVM with Fisher kernel excels at identifying readers, but not comprehending text.
Generative models of eye gaze help identify viewers from images.
problem Identifying viewers from images based on their eye movements.
method Derived Fisher kernels from generative models of eye gaze to train a discriminative classifier.
result Performance of the classifier improves with better underlying generative models.
A new weighted FDA method improves face recognition accuracy.
problem Equal treatment of all class pairs in FDA leads to suboptimal performance.
method Cosine-weighted and automatically weighted FDA methods are proposed.
result Improved face recognition accuracy through weighted FDA.
Linear Discriminant Analysis (LDA) is a well-known method for dimensionality reduction and classification. Previous studies have also extended the binary-class case into multi-classes. However, many applications, such as object detection and keyframe extraction cannot provide consistent instance-label pairs, while LDA …
New method speeds up solving orthogonality constrained problems.
problem Solving orthogonality constrained problems efficiently.
method Riemannian optimization and Riemannian preconditioning.
result Preconditioning improves computational costs and convergence.
It is well known that in a supervised classification setting when the number of features is smaller than the number of observations, Fisher's linear discriminant rule is asymptotically Bayes. However, there are numerous modern applications where classification is needed in the high-dimensional setting. Naive implementa…
Paper improves matrix-valued data classification using nonparametric LDA.
problem Classification of matrix-valued data in neuroimaging and signal processing.
method Nonparametric LDA based on NPMLE for vectorized and scaled matrices.
result Improves classification performance across various data structures.
For high-dimensional classification, it is well known that naively performing the Fisher discriminant rule leads to poor results due to diverging spectra and noise accumulation. Therefore, researchers proposed independence rules to circumvent the diverse spectra, and sparse independence rules to mitigate the issue of n…
Steganalysis improves detection of adversarial examples.
problem Detecting adversarial examples from deep neural networks.
method Enhancing steganalysis features by estimating adversarial attack probabilities.
result The method accurately detects adversarial examples and resists secondary attacks.
A new classification rule for FDA improves classification performance by accounting for unequal covariance matrices.
problem Unequal covariance matrices in practical situations affect the performance of FDA and its variants.
method Proposes a novel classification rule for FDA that accounts for unequal covariance matrices, applicable to many FDA variants.
result The new classification rule improves classification performance compared to original FDA and variants.
A new method improves few-shot learning by combining ProtoNet with LFD.
problem Few-shot learning struggles with high variance support sets.
method Combines ProtoNet with Local Fisher Discriminant Analysis.
result Superior classification accuracy on miniImageNet and tieredImageNet.
Generative model gradients enhance MS/MS peptide identification.
problem Improving peptide identification from MS/MS spectra.
method Leverage log-likelihood gradients of generative models in a kernel-based classifier.
result Fisher kernel outperforms other methods on MS/MS datasets.
New framework assesses neural sensitivity to small perturbations.
problem Comparing neural representations' sensitivity to small changes.
method Local decodable information, Fisher information, and projected pullback/Fisher metric.
result Reveals differences in neural sensitivity not captured by activation alignment.
Explains eigenvalue and generalized eigenvalue problems with examples.
problem Eigenvalue and generalized eigenvalue problems.
method Introduction and examples from machine learning.
result Solutions to eigenvalue and generalized eigenvalue problems.
Clustering in high-dimensional spaces is nowadays a recurrent problem in many scientific domains but remains a difficult task from both the clustering accuracy and the result understanding points of view. This paper presents a discriminative latent mixture (DLM) model which fits the data in a latent orthonormal discrim…
Section 1.3 was incorrect, and 2.1 will be removed from further submissions. A rewritten version will be posted in the future.
We unify f-divergences, Bregman divergences, surrogate loss bounds (regret bounds), proper scoring rules, matching losses, cost curves, ROC-curves and information. We do this by systematically studying integral and variational representations of these objects and in so doing identify their primitives which all are rela…
SQFA learns features maximizing Fisher-Rao distance for better classification.
problem Improving classification accuracy through feature learning.
method SQFA learns linear features maximizing Fisher-Rao distance between class-conditional distributions.
result SQFA-H features achieve the best classification accuracy.
A new algorithm improves Wasserstein discriminant analysis for better data classification.
problem Improving data classification in machine learning.
method Bi-level nonlinear eigenvector algorithm (WDA-nepv) for optimal transport and trace ratio optimizations.
result WDA-nepv enhances classification accuracy and scalability.
AI systems can be corrected without rebuilding them, using simple linear methods.
problem Errors in AI systems can lead to serious consequences.
method Developed fast non-destructive methods using linear Fisher discriminant.
result Simple linear methods can separate error-prone situations from correct ones.
New method for natural policy gradients converges linearly.
problem Improving natural policy gradient methods for better convergence.
method Fisher-Rao gradient flow applied to state-action distributions.
result Linear convergence rate with geometry-dependent factor.
Training on some out-of-distribution data improves generalization error before it deteriorates.
problem Generalization error improves with some out-of-distribution data but deteriorates with more.
method Synthetic datasets and deep networks on computer vision benchmarks.
result Non-monotonic trend in generalization error with OOD samples.
In this paper, we propose a method for image-set classification based on convex cone models, focusing on the effectiveness of convolutional neural network (CNN) features as inputs. CNN features have non-negative values when using the rectified linear unit as an activation function. This naturally leads us to model a se…
A new method for few-shot learning using directional statistics.
problem Few-shot classification with limited training data.
method Generates class representatives using a mixture of von Mises-Fisher distributions to capture inter-class correlation.
result Outperforms other methods in miniImageNet and tieredImageNet datasets.
Classification is an important tool with many useful applications. Among the many classification methods, Fisher's Linear Discriminant Analysis (LDA) is a traditional model-based approach which makes use of the covariance information. However, in the high-dimensional, low-sample size setting, LDA cannot be directly dep…
A new method generalizing subspace learning for improved classification.
problem Improving classification accuracy using subspace learning methods.
method Roweis Discriminant Analysis (RDA) which generalizes PCA, SPCA, and FDA.
result RDA and kernel RDA improve classification accuracy on benchmark datasets.
Survey of Locally Linear Embedding and its variants.
problem Representing high-dimensional data in a lower-dimensional space while preserving local structure.
method Explains various LLE and variant methods, including kernel LLE, inverse LLE, feature fusion, out-of-sample embedding, incremental LLE, landmark LLE, supervised LLE, robust LLE, fusion with other methods, and weighted LLE.
result Comprehensive overview of LLE and its variants.
We propose to investigate test statistics for testing homogeneity in reproducing kernel Hilbert spaces. Asymptotic null distributions under null hypothesis are derived, and consistency against fixed and local alternatives is assessed. Finally, experimental evidence of the performance of the proposed approach on both ar…