Bayesian SPLDA adapts model parameters for database transfer.
problem Adapting SPLDA for new databases with limited data.
method Variational Bayes estimation of SPLDA parameters.
result Adaptation of SPLDA model parameters for database transfer.
A new LDA variant improves multi-label classification performance.
problem Improving multi-label classification performance.
method Saliency-based weights redefine between-class and within-class scatter matrices for multi-label classification.
result The proposed method leads to performance improvements in various multi-label classification problems.
Paper introduces a new probabilistic model for class-specific discriminant analysis.
problem Lack of multi-modal structure consideration in existing class-specific methods.
method Formulates a probabilistic model that incorporates multi-modal negative class structure.
result Proposed model can be directly used for class-specific probabilistic classification.
Explains LDA and QDA for binary and multiple classes.
problem Classification methods in statistical and probabilistic learning.
method Optimization of decision boundaries, estimation of parameters, relation to other methods.
result Equivalence of LDA and Fisher discriminant analysis.
Survey of spectral, probabilistic, and deep metric learning methods.
problem Developing effective distance metrics for various machine learning tasks.
method Divided into spectral, probabilistic, and deep approaches, covering various techniques and their applications.
result Comprehensive overview of metric learning methods, including new developments and applications.
DNLL loss improves deep LDA accuracy and consistency.
problem Pathological solutions in unconstrained Deep LDA.
method Introducing Discriminative Negative Log-Likelihood (DNLL) loss.
result Deep LDA trained with DNLL produces clean latent spaces and better calibrated probabilities.
New discriminant analysis using GDS projection improves face recognition.
problem Improving face recognition accuracy with limited data.
method GDS projection onto generalized difference subspace, simplified Fisher criterion, normalization.
result GDS projection and gFDA are equivalent, inheriting FDA's discriminant ability.
Proposes a novel linear discriminant analysis for matrix-valued data.
problem Classification of high-dimensional matrix-valued data from imaging studies.
method Efficient nuclear norm penalized regression with low-rank structure.
result Superior performance compared to existing methods in simulations and EEG data.
Paper proposes distributed sparse multicategory discriminant analysis for classification.
problem Sparse multicategory classification with distributed data.
method Convex formulation, distributed setting, invariant discriminant subspace recovery.
result Distributed sparse multicategory linear discriminant analysis performs as good as centralized version after a few rounds of communications.
Paper simplifies concentration inequalities for easier probabilistic analysis.
problem Complexity in probabilistic analysis of random variables.
method Compact notations for concentration inequalities.
result Simplified expressions for typical sizes and tails of random variables.
PANDA improves linear discriminant analysis in high dimensions with minimal tuning.
problem Linear discriminant analysis in high-dimensional settings.
method PANDA: a tuning-insensitive method for linear discriminant analysis.
result PANDA achieves optimal convergence rates in estimation error and misclassification rate.
A new clustering method estimates non-linear boundaries and automatically selects the number of clusters.
problem Discriminative clustering with non-linear boundaries and data abnormalities.
method Regularized mutual information objective function with a mixture of Gaussian and uniform distributions.
result Automatic selection of the number of components and estimation of non-linear boundaries.
We present a unifying framework which reduces the construction of probabilistic component analysis techniques to a mere selection of the latent neighbourhood, thus providing an elegant and principled framework for creating novel component analysis models as well as constructing probabilistic equivalents of deterministi…
Improved LDA method for better classification and dimensionality reduction.
problem Improving linear discriminant analysis for better classification performance.
method Integrates spectrally-corrected covariance matrix and regularized discriminant analysis.
result SRLDA has a linear classification global optimal solution under spiked model assumption.
Paper tackles expected predictions computation for arbitrary generative models.
problem Hard to compute expected predictions for arbitrary generative models.
method Identifies tractable generative and discriminative models for expected predictions.
result Tractable computation of high-order moments and expectations for classification.
This paper analyzes implicit bias in Deep Linear Discriminant Analysis.
problem The implicit bias of Deep Linear Discriminant Analysis.
method Analyzing gradient flow on a L-layer diagonal linear network.
result Under balanced initialization, the network transforms additive updates into multiplicative updates, conserving the (2/L) quasi-norm.
A new neural network reduces high-dimensional time-series data for faster classification.
problem Classifying high-dimensional time-series patterns efficiently.
method Developed a time-series discriminant component network (TSDCN) using TSDCA for dimensionality reduction and classification.
result The TSDCN achieves high-accuracy classification and reduces training time.
A new method for high-dimensional data classification with improved feature selection.
problem High-dimensional data classification with limited interpretability and prediction accuracy.
method Integrates multiclass diagonal discriminant analysis with feature selection.
result Significantly improved prediction accuracy and feature interpretability.
Paper introduces CGPMs for probabilistic data analysis.
problem Difficulty in applying, combining, and comparing different probabilistic techniques.
method Composable generative population models (CGPMs) that extend graphical models and can describe and compose various probabilistic data analysis techniques.
result CGPMs enable efficient and accurate probabilistic data analysis tasks.
Proposes improved classification via transfer learning with regularized linear discriminant analysis.
problem High dimensionality and small sample sizes lead to poor classification performance.
method Regularized random-effects linear discriminant analysis, combining ridge estimates from target and source models.
result Explicit derivation of asymptotic weights and classification error rates in high-dimensional settings.
Kernel discriminant analysis uses nonlinear embeddings to improve classification.
problem Limited effectiveness of linear discriminant analysis in capturing nonlinear features.
method Study of nonlinear embeddings in kernel discriminant analysis using polynomial and Gaussian kernels, solving generalized eigenvalue problems.
result Polynomial and Gaussian discriminants capture class differences through population moments and randomized projections.
Probabilistic model for weakly supervised analysis dictionary learning.
problem Discriminative analysis dictionary learning under weak supervision.
method Probabilistic modeling with EM algorithm and graph reformulation.
result Improved classification performance compared to synthesis dictionary learning.
uGMM-NN integrates probabilistic reasoning into neural networks.
problem Capturing multimodality and uncertainty in neural network activations.
method Parameterizes activations as univariate Gaussian mixtures with learnable parameters.
result Competitive discriminative performance with probabilistic activations.
A new method for LDA using randomized Kaczmarz improves accuracy for large datasets.
problem Efficiently performing LDA on large datasets.
method Randomized Kaczmarz method applied to linear discriminant analysis.
result The method achieves comparable accuracy to full data LDA.
This work connects LLE, factor analysis, and probabilistic PCA through a stochastic perspective.
problem Exploring the theoretical connection between LLE, factor analysis, and probabilistic PCA.
method Solving the stochastic linear reconstruction of LLE using expectation maximization.
result LLE, factor analysis, and probabilistic PCA are shown to be connected through a stochastic perspective.
The paper proposes a method to focus on discriminative regions for better unsupervised domain adaptation.
problem Unsupervised domain adaptation with limited target domain labels.
method Probabilistic certainty estimate of regions to focus on during classification.
result State-of-the-art results on various datasets compared to recent methods.
Semi-supervised learning is an important and active topic of research in pattern recognition. For classification using linear discriminant analysis specifically, several semi-supervised variants have been proposed. Using any one of these methods is not guaranteed to outperform the supervised classifier which does not t…
Paper uses GMM and MAF for probabilistic classification, outperforming simpler models.
problem Classifying data with complex distributions.
method Density estimation using Gaussian Mixture Model and Masked Autoregressive Flow.
result Proposed classifiers outperform simpler models like linear discriminant analysis.
We present an alternative to the pseudo-inverse method for determining the hidden to output weight values for Extreme Learning Machines performing classification tasks. The method is based on linear discriminant analysis and provides Bayes optimal single point estimates for the weight values.
Paper proposes faster incremental subclass discriminant analysis.
problem Efficiently classify subclasses in incremental data.
method Exact and approximate linear and kernelized solutions.
result Superior training time and accuracy compared to existing methods.
New method for tensor classification with missing data.
problem Handling incomplete tensor data in high-dimensional classification.
method High-dimensional tensor linear discriminant analysis with TGMM and Tensor LDA-MD.
result Established convergence rates and minimax optimal bounds for misclassification rate.
Improved LDA with capped l_{2,1}-norm reduces outlier sensitivity.
problem Outliers and noise sensitivity in classical LDA.
method Introducing capped l_{2,1}-norm and proposing CLDA.
result CLDA effectively removes outliers and suppresses noise.
NCC is inefficient in higher dimensions, NCDA improves performance.
problem Inefficiency of NCC in higher dimensions.
method Combining NCC with LDA to create NCDA.
result NCDA outperforms NCC and competes with LDA and QDA.
Survey of Locally Linear Embedding and its variants.
problem Representing high-dimensional data in a lower-dimensional space while preserving local structure.
method Explains various LLE and variant methods, including kernel LLE, inverse LLE, feature fusion, out-of-sample embedding, incremental LLE, landmark LLE, supervised LLE, robust LLE, fusion with other methods, and weighted LLE.
result Comprehensive overview of LLE and its variants.
New methods for fair machine learning reduce bias in data analysis.
problem Bias in machine learning algorithms that treat sensitive features like gender or race.
method Novel fair regression and dimensionality reduction methods using Hilbert Schmidt independence criterion and kernel functions.
result The methods simplify the problem and allow dealing with multiple sensitive variables simultaneously.
AI systems can be corrected without rebuilding them, using simple linear methods.
problem Errors in AI systems can lead to serious consequences.
method Developed fast non-destructive methods using linear Fisher discriminant.
result Simple linear methods can separate error-prone situations from correct ones.
Simplified analysis of SGD for linear regression with weight averaging.
problem Understanding SGD optimization in linear regression models.
method Simplified analysis using linear algebra tools, bypassing complex operator manipulations.
result Recovery of bias and variance bounds for SGD in linear regression.
This paper presents a novel approach to speaker subspace modelling based on Gaussian-Binary Restricted Boltzmann Machines (GRBM). The proposed model is based on the idea of shared factors as in the Probabilistic Linear Discriminant Analysis (PLDA). GRBM hidden layer is divided into speaker and channel factors, herein t…
Unified analysis simplifies Johnson-Lindenstrauss lemma for data reduction.
problem Efficiently reducing high-dimensional data while preserving geometry.
method Unified analysis of various JL constructions using probabilistic tools.
result First rigorous proof and extension of spherical construction's effectiveness.
We study the problem of supervised linear dimensionality reduction, taking an information-theoretic viewpoint. The linear projection matrix is designed by maximizing the mutual information between the projected signal and the class label (based on a Shannon entropy measure). By harnessing a recent theoretical result on…
Simplified GAN model shows how discriminator improves generalization.
problem Understanding GAN's generalization ability and avoiding memorization.
method Analyzing a simplified GAN model with early stopping and Wasserstein metric.
result Generalization error escapes from curse of dimensionality with early stopping.
Layered graphical models improve discriminative learning efficiency.
problem Improving discriminative learning efficiency in graphical models.
method Designing layered graphical models (LGMs) in analogy to neural networks, using tensorized truncated variational inference and backpropagation.
result LGMs achieve competitive results in image classification, comparable to neural networks.
The paper analyzes an ensemble of randomly projected linear discriminants for high-dimensional data.
problem Classification issues in small samples of high-dimensional data.
method Asymptotic analysis using random matrix theory.
result The ensemble offers a performance advantage under certain conditions.
New framework improves classification accuracy using Pillai's trace and ULDA.
problem Traditional LDA's limitations in noise sensitivity and non-invertible matrices.
method Integrates Pillai's trace with ULDA for a unified classifier.
result Effective control of Type I error rates and improved classification accuracy.
Discriminative linear models are a popular tool in machine learning. These can be generally divided into two types: The first is linear classifiers, such as support vector machines, which are well studied and provide state-of-the-art results. One shortcoming of these models is that their output (known as the 'margin') …
Wasserstein Discriminant Analysis improves classification of high-dimensional data.
problem Improving classification of high-dimensional data.
method Computes a suitable linear map onto a lower dimensional subspace using regularized Wasserstein distances.
result WDA shows promising results in prediction and visualization on various datasets.
End-to-end CCA optimizes both discriminative and latent space projections for multi-view learning.
problem Lack of class label information in CCA for multi-view learning tasks.
method Simultaneously optimizes a CCA-based and a task objective in an end-to-end manner to learn a non-linear CCA projection.
result Significant improvement in cross-view classification, regularization with a second view, and semi-supervised learning.
We present a novel approach to the formulation and the resolution of sparse Linear Discriminant Analysis (LDA). Our proposal, is based on penalized Optimal Scoring. It has an exact equivalence with penalized LDA, contrary to the multi-class approaches based on the regression of class indicator that have been proposed s…