Proposes a novel linear discriminant analysis for matrix-valued data.
problem Classification of high-dimensional matrix-valued data from imaging studies.
method Efficient nuclear norm penalized regression with low-rank structure.
result Superior performance compared to existing methods in simulations and EEG data.
Neural nets trained with linear discriminant initialization converge faster and more accurately.
problem Training feed-forward neural networks efficiently and accurately.
method Initialize first layer weights with linear discriminants.
result Asymptotic higher accuracy and faster convergence.
Paper proposes distributed sparse multicategory discriminant analysis for classification.
problem Sparse multicategory classification with distributed data.
method Convex formulation, distributed setting, invariant discriminant subspace recovery.
result Distributed sparse multicategory linear discriminant analysis performs as good as centralized version after a few rounds of communications.
PANDA improves linear discriminant analysis in high dimensions with minimal tuning.
problem Linear discriminant analysis in high-dimensional settings.
method PANDA: a tuning-insensitive method for linear discriminant analysis.
result PANDA achieves optimal convergence rates in estimation error and misclassification rate.
Generative classifiers' properties are linked to linear constraints.
problem Understanding the Markov property in generative classifiers.
method Characterization of discrimination functions using linear constraints and a second order finite difference operator.
result Discrimination functions of undirected Markov network classifiers are characterized by sets of linear constraints.
This paper analyzes implicit bias in Deep Linear Discriminant Analysis.
problem The implicit bias of Deep Linear Discriminant Analysis.
method Analyzing gradient flow on a L-layer diagonal linear network.
result Under balanced initialization, the network transforms additive updates into multiplicative updates, conserving the (2/L) quasi-norm.
Improved LDA method for better classification and dimensionality reduction.
problem Improving linear discriminant analysis for better classification performance.
method Integrates spectrally-corrected covariance matrix and regularized discriminant analysis.
result SRLDA has a linear classification global optimal solution under spiked model assumption.
Generative adversarial nets (GANs) are a promising technique for modeling a distribution from samples. It is however well known that GAN training suffers from instability due to the nature of its maximin formulation. In this paper, we explore ways to tackle the instability problem by dualizing the discriminator. We sta…
A new LDA variant improves multi-label classification performance.
problem Improving multi-label classification performance.
method Saliency-based weights redefine between-class and within-class scatter matrices for multi-label classification.
result The proposed method leads to performance improvements in various multi-label classification problems.
Proposes improved classification via transfer learning with regularized linear discriminant analysis.
problem High dimensionality and small sample sizes lead to poor classification performance.
method Regularized random-effects linear discriminant analysis, combining ridge estimates from target and source models.
result Explicit derivation of asymptotic weights and classification error rates in high-dimensional settings.
We develop a novel method for training of GANs for unsupervised and class conditional generation of images, called Linear Discriminant GAN (LD-GAN). The discriminator of an LD-GAN is trained to maximize the linear separability between distributions of hidden representations of generated and targeted samples, while the …
Kernel discriminant analysis uses nonlinear embeddings to improve classification.
problem Limited effectiveness of linear discriminant analysis in capturing nonlinear features.
method Study of nonlinear embeddings in kernel discriminant analysis using polynomial and Gaussian kernels, solving generalized eigenvalue problems.
result Polynomial and Gaussian discriminants capture class differences through population moments and randomized projections.
A new method for LDA using randomized Kaczmarz improves accuracy for large datasets.
problem Efficiently performing LDA on large datasets.
method Randomized Kaczmarz method applied to linear discriminant analysis.
result The method achieves comparable accuracy to full data LDA.
We present an alternative to the pseudo-inverse method for determining the hidden to output weight values for Extreme Learning Machines performing classification tasks. The method is based on linear discriminant analysis and provides Bayes optimal single point estimates for the weight values.
Semi-supervised learning is an important and active topic of research in pattern recognition. For classification using linear discriminant analysis specifically, several semi-supervised variants have been proposed. Using any one of these methods is not guaranteed to outperform the supervised classifier which does not t…
DNLL loss improves deep LDA accuracy and consistency.
problem Pathological solutions in unconstrained Deep LDA.
method Introducing Discriminative Negative Log-Likelihood (DNLL) loss.
result Deep LDA trained with DNLL produces clean latent spaces and better calibrated probabilities.
Generalized dual discriminator GANs improve upon traditional GANs by using two discriminators and a flexible loss function.
problem Mode collapse in GANs.
method Introducing dual discriminator α-GANs and extending the approach to arbitrary functions. result The approach reduces the optimization problem to a linear combination of an f-divergence and a reverse f-divergence. Improved LDA with capped l_{2,1}-norm reduces outlier sensitivity.
problem Outliers and noise sensitivity in classical LDA.
method Introducing capped l_{2,1}-norm and proposing CLDA.
result CLDA effectively removes outliers and suppresses noise.
Recent studies in the literature have paid much attention to the sparsity in linear classification tasks. One motivation of imposing sparsity assumption on the linear discriminant direction is to rule out the noninformative features, making hardly contribution to the classification problem. Most of those work were focu…
End-to-end CCA optimizes both discriminative and latent space projections for multi-view learning.
problem Lack of class label information in CCA for multi-view learning tasks.
method Simultaneously optimizes a CCA-based and a task objective in an end-to-end manner to learn a non-linear CCA projection.
result Significant improvement in cross-view classification, regularization with a second view, and semi-supervised learning.
NCC is inefficient in higher dimensions, NCDA improves performance.
problem Inefficiency of NCC in higher dimensions.
method Combining NCC with LDA to create NCDA.
result NCDA outperforms NCC and competes with LDA and QDA.
Optimal domain adaptation model using Fisher's Linear Discriminant.
problem Improving classification accuracy across different domains.
method Convex combination of source and target hypotheses, derived under 0-1 loss.
result Effective classifier can be computed without direct source task information.
New method for tensor classification with missing data.
problem Handling incomplete tensor data in high-dimensional classification.
method High-dimensional tensor linear discriminant analysis with TGMM and Tensor LDA-MD.
result Established convergence rates and minimax optimal bounds for misclassification rate.
We prove that the discriminant of a nonsingular space curve of genus g≥2 is stable with respect to the standard action of the special linear group.
The paper analyzes an ensemble of randomly projected linear discriminants for high-dimensional data.
problem Classification issues in small samples of high-dimensional data.
method Asymptotic analysis using random matrix theory.
result The ensemble offers a performance advantage under certain conditions.
We study the problem of supervised linear dimensionality reduction, taking an information-theoretic viewpoint. The linear projection matrix is designed by maximizing the mutual information between the projected signal and the class label (based on a Shannon entropy measure). By harnessing a recent theoretical result on…
Paper proposes faster incremental subclass discriminant analysis.
problem Efficiently classify subclasses in incremental data.
method Exact and approximate linear and kernelized solutions.
result Superior training time and accuracy compared to existing methods.
We present a novel approach to the formulation and the resolution of sparse Linear Discriminant Analysis (LDA). Our proposal, is based on penalized Optimal Scoring. It has an exact equivalence with penalized LDA, contrary to the multi-class approaches based on the regression of class indicator that have been proposed s…
MILDA uses unlabelled data to compute LDA projections.
problem Training LDA models with unlabelled data.
method Minimal prior information to compute LDA projection vector.
result MILDA closely matches supervised LDA performance and adapts to non-stationary data.
Explains LDA and QDA for binary and multiple classes.
problem Classification methods in statistical and probabilistic learning.
method Optimization of decision boundaries, estimation of parameters, relation to other methods.
result Equivalence of LDA and Fisher discriminant analysis.
We consider the problem of discriminative factor analysis for data that are in general non-Gaussian. A Bayesian model based on the ranks of the data is proposed. We first introduce a new {\em max-margin} version of the rank-likelihood. A discriminative factor model is then developed, integrating the max-margin rank-lik…
New framework improves classification accuracy using Pillai's trace and ULDA.
problem Traditional LDA's limitations in noise sensitivity and non-invertible matrices.
method Integrates Pillai's trace with ULDA for a unified classifier.
result Effective control of Type I error rates and improved classification accuracy.
SEDA improves RLDA for high-dimensional data.
problem Inconsistent performance of RLDA in high-dimensional scenarios.
method Developed a non-asymptotic approximation of misclassification rate, derived new theoretical results on eigenvectors, and proposed SEDA algorithm.
result SEDA achieves higher classification accuracy and dimensionality reduction compared to existing LDA methods.
New method reduces indirect discrimination in insurance risk models.
problem Indirect discrimination in insurance risk models using machine learning.
method Mathematical concepts of linear algebra to reduce indirect discrimination.
result Demonstrated promising performance in a concrete case of risk selection in life insurance.
In recent work on both generative and discriminative score to log-likelihood-ratio calibration, it was shown that linear transforms give good accuracy only for a limited range of operating points. Moreover, these methods required tailoring of the calibration training objective functions in order to target the desired r…
The Kalman filter (KF) is used in a variety of applications for computing the posterior distribution of latent states in a state space model. The model requires a linear relationship between states and observations. Extensions to the Kalman filter have been proposed that incorporate linear approximations to nonlinear m…
Metrics specifying distances between data points can be learned in a discriminative manner or from generative models. In this paper, we show how to unify generative and discriminative learning of metrics via a kernel learning framework. Specifically, we learn local metrics optimized from parametric generative models. T…
GNNs improve graph signal discrimination by adding nonlinearities.
problem Improving graph signal discrimination in physical networks.
method Analyzing the discriminability of GNNs and their relation to graph filter banks.
result GNNs are at least as discriminative as linear graph filter banks.
Discriminative latent-variable models are typically learned using EM or gradient-based optimization, which suffer from local optima. In this paper, we develop a new computationally efficient and provably consistent estimator for a mixture of linear regressions, a simple instance of a discriminative latent-variable mode…
Extends L2-norm LDA to 2D inputs using Bhattacharyya bound.
problem L2-norm LDA loses useful image information for 2D inputs.
method 2DBLDA maximizes matrix-based between-class distance and minimizes within-class distance, optimizing Bhattacharyya error bound.
result 2DBLDA improves image recognition and face reconstruction.
Study infinite Euclidean distance discriminants of algebraic varieties.
problem Understanding the structure of data points with infinitely many critical points in Euclidean distance correspondence.
method Developed computer code to compute discriminants and proved properties of fibers.
result Infinite Euclidean distance discriminants contain all data points with infinitely many critical points for the nearest-point problem.
Reinterprets DNF as a deep generative LDA model for complex data.
problem Limited applicability of LDA in complex data scenarios.
method Proposes a discriminative normalization flow (DNF) model and interprets it as a deep generative LDA.
result DNF and its subspace version outperform conventional LDA in modeling complex data.
Hidden Markov Chains and Linear-chain CRFs are equivalent.
problem Comparing Hidden Markov Chains and Conditional Random Fields.
method Constructing an HMC with the same posterior distribution as a CRF.
result HMCs and linear-chain CRFs are equivalent models.
WLDA enhances LDA for missing data, improving classification accuracy and interpretability.
problem Missing data in real-world datasets hinders classification accuracy and interpretability of LDA.
method WLDA incorporates a weight matrix to handle missing data directly, preserving interpretability.
result WLDA significantly outperforms traditional methods in datasets with missing values.
A new clustering method estimates non-linear boundaries and automatically selects the number of clusters.
problem Discriminative clustering with non-linear boundaries and data abnormalities.
method Regularized mutual information objective function with a mixture of Gaussian and uniform distributions.
result Automatic selection of the number of components and estimation of non-linear boundaries.
Paper detects proxies in linear regression models causing discrimination.
problem Discrimination in machine learning models using proxies for protected attributes.
method Formulated a definition of proxy use, identified proxies via second-order cone program, and extended to justified business necessity.
result Proxies in linear regression models can be efficiently identified and removed to reduce discrimination.
Fisher's linear discriminant analysis (FLDA) is an important dimension reduction method in statistical pattern recognition. It has been shown that FLDA is asymptotically Bayes optimal under the homoscedastic Gaussian assumption. However, this classical result has the following two major limitations: 1) it holds only fo…
New robust discriminant analysis for non-Gaussian data.
problem Classical discriminant analysis struggles with non-Gaussian distributions and contaminated datasets.
method Each data point follows its own ES distribution with arbitrary scale, leading to robust classification.
result Maximum-likelihood estimation and classification are simple, fast, and robust.