We present a new method which generalizes subspace learning based on eigenvalue and generalized eigenvalue problems. This method, Roweis Discriminant Analysis (RDA), is named after Sam Roweis to whom the field of subspace learning owes significantly. RDA is a family of infinite number of algorithms where Principal Comp…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Roweis discriminant analysis”
Roweisposes, Including Eigenposes, Supervised Eigenposes, and Fisherposes, for 3D Action Recognitioncs.CV
Paper proposes Roweisposes for 3D action recognition using generalized eigenvalue problem.
problem Need for basic methods in 3D action recognition.
method Roweisposes uses Roweis discriminant analysis for generalized subspace learning.
result Roweisposes is effective for 3D action recognition.
Local Procrustes for Manifold Embedding: A Measure of Embedding Quality and Embedding Algorithmsstat.ML
We present the Procrustes measure, a novel measure based on Procrustes rotation that enables quantitative comparison of the output of manifold-based embedding algorithms (such as LLE (Roweis and Saul, 2000) and Isomap (Tenenbaum et al, 2000)). The measure also serves as a natural tool when choosing dimension-reduction …
Kernel discriminant analysis uses nonlinear embeddings to improve classification.
problem Limited effectiveness of linear discriminant analysis in capturing nonlinear features.
method Study of nonlinear embeddings in kernel discriminant analysis using polynomial and Gaussian kernels, solving generalized eigenvalue problems.
result Polynomial and Gaussian discriminants capture class differences through population moments and randomized projections.
Paper proposes distributed sparse multicategory discriminant analysis for classification.
problem Sparse multicategory classification with distributed data.
method Convex formulation, distributed setting, invariant discriminant subspace recovery.
result Distributed sparse multicategory linear discriminant analysis performs as good as centralized version after a few rounds of communications.
Spectrally-Corrected and Regularized Linear Discriminant Analysis for Spiked Covariance Modelstat.ML
Improved LDA method for better classification and dimensionality reduction.
problem Improving linear discriminant analysis for better classification performance.
method Integrates spectrally-corrected covariance matrix and regularized discriminant analysis.
result SRLDA has a linear classification global optimal solution under spiked model assumption.
Sparse versions of principal component analysis (PCA) have imposed themselves as simple, yet powerful ways of selecting relevant features of high-dimensional data in an unsupervised manner. However, when several sparse principal components are computed, the interpretation of the selected variables is difficult since ea…
PANDA improves linear discriminant analysis in high dimensions with minimal tuning.
problem Linear discriminant analysis in high-dimensional settings.
method PANDA: a tuning-insensitive method for linear discriminant analysis.
result PANDA achieves optimal convergence rates in estimation error and misclassification rate.
New robust discriminant analysis for non-Gaussian data.
problem Classical discriminant analysis struggles with non-Gaussian distributions and contaminated datasets.
method Each data point follows its own ES distribution with arbitrary scale, leading to robust classification.
result Maximum-likelihood estimation and classification are simple, fast, and robust.
Proposes KMvDA for object recognition from multi-view data.
problem Recognizing objects from different views, even when views are heterogeneous.
method Introduces kernel multi-view discriminant analysis (KMvDA) and uses random Fourier features (RFF) for large-scale learning.
result KMvDA and RFF approximation improve object recognition from multi-view data.
This tutorial explains Linear Discriminant Analysis (LDA) and Quadratic Discriminant Analysis (QDA) as two fundamental classification methods in statistical and probabilistic learning. We start with the optimization of decision boundary on which the posteriors are equal. Then, LDA and QDA are derived for binary and mul…
This paper analyzes implicit bias in Deep Linear Discriminant Analysis.
problem The implicit bias of Deep Linear Discriminant Analysis.
method Analyzing gradient flow on a L-layer diagonal linear network.
result Under balanced initialization, the network transforms additive updates into multiplicative updates, conserving the (2/L) quasi-norm.
DFSOS improves sparse discriminant analysis for high-dimensional data.
problem Sparse discriminant analysis in high-dimensional settings with feature selection.
method Deflation-Free Sparse Optimal Scoring (DFSOS) using Bregman iteration and orthogonality-constrained optimization.
result DFSOS achieves comparable or better classification accuracy than deflation-based methods.
Discriminative Dictionary Learning (DL) methods have been widely advocated for image classification problems. To further sharpen their discriminative capabilities, most state-of-the-art DL methods have additional constraints included in the learning stages. These various constraints, however, lead to additional computa…
Proposes improved classification via transfer learning with regularized linear discriminant analysis.
problem High dimensionality and small sample sizes lead to poor classification performance.
method Regularized random-effects linear discriminant analysis, combining ridge estimates from target and source models.
result Explicit derivation of asymptotic weights and classification error rates in high-dimensional settings.
New method for tensor classification with missing data.
problem Handling incomplete tensor data in high-dimensional classification.
method High-dimensional tensor linear discriminant analysis with TGMM and Tensor LDA-MD.
result Established convergence rates and minimax optimal bounds for misclassification rate.
New framework improves classification accuracy using Pillai's trace and ULDA.
problem Traditional LDA's limitations in noise sensitivity and non-invertible matrices.
method Integrates Pillai's trace with ULDA for a unified classifier.
result Effective control of Type I error rates and improved classification accuracy.
New method compresses large sample data for faster discriminant analysis.
problem Large sample sizes in discriminant analysis increase computational burden.
method Proposes a new compression approach for reducing training samples.
result Significant computational gains and superior predictive ability compared to random sub-sampling.
This paper proposes a new subspace learning method, named Quantized Fisher Discriminant Analysis (QFDA), which makes use of both machine learning and information theory. There is a lack of literature for combination of machine learning and information theory and this paper tries to tackle this gap. QFDA finds a subspac…
We introduce a new discriminant analysis method (Empirical Discriminant Analysis or EDA) for binary classification in machine learning. Given a dataset of feature vectors, this method defines an empirical feature map transforming the training and test data into new data with components having Gaussian empirical distrib…
Kullback-Leibler Penalized Sparse Discriminant Analysis for Event-Related Potential Classificationcs.CV
A brain computer interface (BCI) is a system which provides direct communication between the mind of a person and the outside world by using only brain activity (EEG). The event-related potential (ERP)-based BCI problem consists of a binary pattern recognition. Linear discriminant analysis (LDA) is widely used to solve…
Avoiding C-hacking when evaluating survival distribution predictions with discrimination measuresstat.ML
The paper addresses evaluating survival predictions using discrimination measures, finding a robust method to convert distributions to risks.
problem Evaluating survival distribution predictions with discrimination measures is challenging and often leads to unfair comparisons.
method The paper surveys existing methods and recommends summing over the predicted cumulative hazard as the most robust method to convert distributions to risks.
result Summing over the predicted cumulative hazard is the most robust method to convert distribution predictions to risk predictions.
We propose a novel linear discriminant analysis approach for the classification of high-dimensional matrix-valued data that commonly arises from imaging studies. Motivated by the equivalence of the conventional linear discriminant analysis and the ordinary least squares, we consider an efficient nuclear norm penalized …
Extends Fisher's Discriminant Analysis for interval-valued data.
problem Classifying entities represented by intervals and histograms.
method Adapts Fisher's Discriminant Analysis using Moore's interval arithmetic and Mallows' distance.
result Discriminant directions for interval-valued data are numerically maximized.
A new method for LDA using randomized Kaczmarz improves accuracy for large datasets.
problem Efficiently performing LDA on large datasets.
method Randomized Kaczmarz method applied to linear discriminant analysis.
result The method achieves comparable accuracy to full data LDA.
Alternating direction method of multipliers for penalized zero-variance discriminant analysisstat.ML
We consider the task of classification in the high dimensional setting where the number of features of the given data is significantly greater than the number of observations. To accomplish this task, we propose a heuristic, called sparse zero-variance discriminant analysis (SZVD), for simultaneously performing linear …
A new LDA variant improves multi-label classification performance.
problem Improving multi-label classification performance.
method Saliency-based weights redefine between-class and within-class scatter matrices for multi-label classification.
result The proposed method leads to performance improvements in various multi-label classification problems.
We consider the high-dimensional discriminant analysis problem. For this problem, different methods have been proposed and justified by establishing exact convergence rates for the classification risk, as well as the l2 convergence results to the discriminative rule. However, sharp theoretical analysis for the variable…
In this paper we formulate a probabilistic model for class-specific discriminant subspace learning. The proposed model can naturally incorporate the multi-modal structure of the negative class, which is neglected by existing class-specific methods. Moreover, it can be directly used to define a class-specific probabilis…
Unequal Covariance Awareness for Fisher Discriminant Analysis and Its Variants in Classificationcs.LG
A new classification rule for FDA improves classification performance by accounting for unequal covariance matrices.
problem Unequal covariance matrices in practical situations affect the performance of FDA and its variants.
method Proposes a novel classification rule for FDA that accounts for unequal covariance matrices, applicable to many FDA variants.
result The new classification rule improves classification performance compared to original FDA and variants.
We present a novel approach to the formulation and the resolution of sparse Linear Discriminant Analysis (LDA). Our proposal, is based on penalized Optimal Scoring. It has an exact equivalence with penalized LDA, contrary to the multi-class approaches based on the regression of class indicator that have been proposed s…
A new QDA classifier for high-dimensional data with spiked covariance.
problem Classifying high-dimensional data with distinct covariance matrices.
method Proposes a novel quadratic classification technique with parameters chosen to maximize the fisher-discriminant ratio.
result The proposed classifier outperforms classical R-QDA and requires lower computational complexity.
DNLL loss improves deep LDA accuracy and consistency.
problem Pathological solutions in unconstrained Deep LDA.
method Introducing Discriminative Negative Log-Likelihood (DNLL) loss.
result Deep LDA trained with DNLL produces clean latent spaces and better calibrated probabilities.
We present an alternative to the pseudo-inverse method for determining the hidden to output weight values for Extreme Learning Machines performing classification tasks. The method is based on linear discriminant analysis and provides Bayes optimal single point estimates for the weight values.
This paper discusses a new type of discriminant analysis based on the orthogonal projection of data onto a generalized difference subspace (GDS). In our previous work, we have demonstrated that GDS projection works as the quasi-orthogonalization of class subspaces, which is an effective feature extraction for subspace …
Unified framework for robust discriminant analysis overcomes Gaussian assumptions.
problem Challenges in linear and quadratic discriminant analysis with non-Gaussian or contaminated data.
method FEMDA framework considers arbitrary Elliptically Symmetrical (ES) distributions with flexible scale parameters.
result Maximum-likelihood parameter estimation and classification are robust and efficient.
Semi-supervised learning is an important and active topic of research in pattern recognition. For classification using linear discriminant analysis specifically, several semi-supervised variants have been proposed. Using any one of these methods is not guaranteed to outperform the supervised classifier which does not t…
Develops MGQDA for multi-group classification with theoretical guarantees and practical applications.
problem Complex multi-group classification problems with nonlinear decision boundaries and group-specific covariance patterns.
method MGQDA, a method based on quadratic discriminant analysis that projects predictors onto a lower-dimensional subspace.
result MGQDA achieves competitive or improved predictive performance compared to existing methods.
Paper introduces a method to improve survival model calibration without sacrificing discrimination.
problem Survival models struggle to balance accurate ranking and event prediction.
method Uses conformal regression to enhance calibration without compromising discrimination.
result The approach improves model calibration across 11 real-world datasets.
A new algorithm improves Wasserstein discriminant analysis for better data classification.
problem Improving data classification in machine learning.
method Bi-level nonlinear eigenvector algorithm (WDA-nepv) for optimal transport and trace ratio optimizations.
result WDA-nepv enhances classification accuracy and scalability.
Improved LDA with capped l_{2,1}-norm reduces outlier sensitivity.
problem Outliers and noise sensitivity in classical LDA.
method Introducing capped l_{2,1}-norm and proposing CLDA.
result CLDA effectively removes outliers and suppresses noise.
Robust GQDA improves classification accuracy in non-Normal data.
problem Non-robustness of GQDA under data contamination.
method Introduced robust estimators for mean vector and dispersion matrix.
result Robust GQDA classifiers perform significantly better in real data applications.
The kernel matrix used in kernel methods encodes all the information required for solving complex nonlinear problems defined on data representations in the input space using simple, but implicitly defined, solutions. Spectral analysis on the kernel matrix defines an explicit nonlinear mapping of the input data represen…
SEDA improves RLDA for high-dimensional data.
problem Inconsistent performance of RLDA in high-dimensional scenarios.
method Developed a non-asymptotic approximation of misclassification rate, derived new theoretical results on eigenvectors, and proposed SEDA algorithm.
result SEDA achieves higher classification accuracy and dimensionality reduction compared to existing LDA methods.
We investigate the difference between using an penalty versus an constraint in generalized eigenvalue problems, such as principal component analysis and discriminant analysis. Our main finding is that an penalty may fail to provide very sparse solutions; a severe disadvantage for variable sel…
The paper analyzes an ensemble of randomly projected linear discriminants for high-dimensional data.
problem Classification issues in small samples of high-dimensional data.
method Asymptotic analysis using random matrix theory.
result The ensemble offers a performance advantage under certain conditions.
In this paper, we study the modeling and the classification of functional data presenting regime changes over time. We propose a new model-based functional mixture discriminant analysis approach based on a specific hidden process regression model that governs the regime changes over time. Our approach is particularly a…
Fisher's linear discriminant analysis (FLDA) is an important dimension reduction method in statistical pattern recognition. It has been shown that FLDA is asymptotically Bayes optimal under the homoscedastic Gaussian assumption. However, this classical result has the following two major limitations: 1) it holds only fo…