This tutorial explains Linear Discriminant Analysis (LDA) and Quadratic Discriminant Analysis (QDA) as two fundamental classification methods in statistical and probabilistic learning. We start with the optimization of decision boundary on which the posteriors are equal. Then, LDA and QDA are derived for binary and mul…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Fisher's LDA”
Classification is an important tool with many useful applications. Among the many classification methods, Fisher's Linear Discriminant Analysis (LDA) is a traditional model-based approach which makes use of the covariance information. However, in the high-dimensional, low-sample size setting, LDA cannot be directly dep…
Paper improves matrix-valued data classification using nonparametric LDA.
problem Classification of matrix-valued data in neuroimaging and signal processing.
method Nonparametric LDA based on NPMLE for vectorized and scaled matrices.
result Improves classification performance across various data structures.
Supervised Dimensionality Reduction Revisited: Why LDA on Frozen CNN Features Deserves a Second Lookcs.LG
LDA improves image classification accuracy with fewer features.
problem Fine-grained image classification with pretrained features.
method Supervised dimensionality reduction with LDA before linear probing.
result LDA improves accuracy over full features in 11 out of 12 configurations.
Linear Discriminant Analysis (LDA) is a well-known method for dimensionality reduction and classification. Previous studies have also extended the binary-class case into multi-classes. However, many applications, such as object detection and keyframe extraction cannot provide consistent instance-label pairs, while LDA …
It is well known that in a supervised classification setting when the number of features is smaller than the number of observations, Fisher's linear discriminant rule is asymptotically Bayes. However, there are numerous modern applications where classification is needed in the high-dimensional setting. Naive implementa…
Deep generative LDAcs.LG
Reinterprets DNF as a deep generative LDA model for complex data.
problem Limited applicability of LDA in complex data scenarios.
method Proposes a discriminative normalization flow (DNF) model and interprets it as a deep generative LDA.
result DNF and its subspace version outperform conventional LDA in modeling complex data.
A new algorithm improves Wasserstein discriminant analysis for better data classification.
problem Improving data classification in machine learning.
method Bi-level nonlinear eigenvector algorithm (WDA-nepv) for optimal transport and trace ratio optimizations.
result WDA-nepv enhances classification accuracy and scalability.
A new QDA classifier for high-dimensional data with spiked covariance.
problem Classifying high-dimensional data with distinct covariance matrices.
method Proposes a novel quadratic classification technique with parameters chosen to maximize the fisher-discriminant ratio.
result The proposed classifier outperforms classical R-QDA and requires lower computational complexity.
Despite many years of research into latent Dirichlet allocation (LDA), applying LDA to collections of non-categorical items is still challenging. Yet many problems with much richer data share a similar structure and could benefit from the vast literature on LDA. We propose logistic LDA, a novel discriminative variant o…
In latent Dirichlet allocation (LDA), topics are multinomial distributions over the entire vocabulary. However, the vocabulary usually contains many words that are not relevant in forming the topics. We adopt a variable selection method widely used in statistical modeling as a dimension reduction tool and combine it wi…
A 'Gibbs-Newton' Technique for Enhanced Inference of Multivariate Polya Parameters and Topic Modelscs.LG
Hyper-parameters play a major role in the learning and inference process of latent Dirichlet allocation (LDA). In order to begin the LDA latent variables learning process, these hyper-parameters values need to be pre-determined. We propose an extension for LDA that we call 'Latent Dirichlet allocation Gibbs Newton' (LD…
Simple Deep LDA models achieve accuracy competitive with softmax baselines.
problem Training Deep LDA models by maximum likelihood estimation leads to overlapping or collapsed class clusters.
method Proposed a constrained Deep LDA formulation with geometric constraints to fix class means and covariance.
result MLE becomes stable under geometric constraints, yielding well-separated class clusters.
E-LDA: Toward Interpretable LDA Topic Models with Strong Guarantees in Logarithmic Parallel Timecs.LG
E-LDA offers faster, interpretable LDA topic models.
problem Inferring topics in LDA topic models with strong guarantees.
method Non-gradient combinatorial approach for faster convergence.
result Logarithmic parallel computation time and interpretability.
LDA-GO improves LDA for high-dimensional data via gradient optimization.
problem LDA struggles in high-dimensional settings due to unreliable covariance matrix estimation.
method LDA-GO learns a low-rank precision matrix via gradient optimization, automatically selecting between Gaussian likelihood and cross-entropy loss.
result LDA-GO outperforms other LDA variants in sparse-signal high-dimensional regimes.
Latent Dirichlet Allocation (LDA) is a popular topic modeling technique for discovery of hidden semantic architecture of text datasets, and plays a fundamental role in many machine learning applications. However, like many other machine learning algorithms, the process of training a LDA model may leak the sensitive inf…
A new LDA variant improves multi-label classification performance.
problem Improving multi-label classification performance.
method Saliency-based weights redefine between-class and within-class scatter matrices for multi-label classification.
result The proposed method leads to performance improvements in various multi-label classification problems.
New method improves reliability of LDA topic modeling by assessing stability across replicated runs.
problem LDA's reproducibility issues due to initial values and Gibbs sampling.
method Cluster replicated LDA runs using modified Jaccard coefficient and pruning algorithm.
result New measure S-CLOP quantifies LDA topic stability, improving reproducibility.
MILDA uses unlabelled data to compute LDA projections.
problem Training LDA models with unlabelled data.
method Minimal prior information to compute LDA projection vector.
result MILDA closely matches supervised LDA performance and adapts to non-stationary data.
Derives VMP for LDA, simplifying inference for topic modeling.
problem Manual derivation of VMP equations for LDA is challenging and time-consuming.
method Detailed derivation of VMP update equations for LDA.
result Enables easier implementation of VMP for LDA models.
Dense Distributions from Sparse Samples: Improved Gibbs Sampling Parameter Estimators for LDAstat.ML
We introduce a novel approach for estimating Latent Dirichlet Allocation (LDA) parameters from collapsed Gibbs samples (CGS), by leveraging the full conditional distributions over the latent variable assignments to efficiently average over multiple samples, for little more computational cost than drawing a single addit…
When data analysts train a classifier and check if its accuracy is significantly different from chance, they are implicitly performing a two-sample test. We investigate the statistical properties of this flexible approach in the high-dimensional setting. We prove two results that hold for all classifiers in any dimensi…
Less Discriminatory Alternative and Interpretable XGBoost Framework for Binary Classificationstat.ML
LDA-XGB1 balances fairness and accuracy in lending models.
problem Fair lending practices and model interpretability in binary classification.
method Biobjective optimization using binning and information value, leveraging XGBoost.
result Achieves effective balance between accuracy, fairness, and interpretability.
LDTA expands LDA's topic modeling capacity with tree-structured priors.
problem Limited expressiveness of Dirichlet priors in LDA for complex topic relationships.
method Introduces Latent Dirichlet-Tree Allocation (LDTA) with Dirichlet-Tree (DT) priors, and develops universal mean-field variational inference and Expectation Propagation.
result LDTA enables expressive, tree-structured priors over topic proportions, expanding modeling capacity of LDA.
This paper defines less discriminatory algorithms and explores their feasibility.
problem Creating algorithms that are less discriminatory while meeting business needs.
method Formal definition of less discriminatory algorithms, evaluation of feasibility, and search for alternatives.
result Formal definitions of less discriminatory algorithms face challenges due to lack of held-out data, necessitating a reliance on reasonableness standards.
The sBIC outperforms other model selection criteria in LDA topic modeling.
problem Selecting the optimal number of topics in Latent Dirichlet Allocation (LDA) models.
method Monte Carlo simulations comparing sBIC to other criteria.
result sBIC is superior for choosing the number of topics in LDA models.
DNLL loss improves deep LDA accuracy and consistency.
problem Pathological solutions in unconstrained Deep LDA.
method Introducing Discriminative Negative Log-Likelihood (DNLL) loss.
result Deep LDA trained with DNLL produces clean latent spaces and better calibrated probabilities.
Improved topic modeling captures temporal relationships in speech.
problem Lack of temporal information in LDA for speech analysis.
method Temporal Markov chain extension to LDA for acoustic unit discovery.
result Improved phone segmentation results compared to base LDA.
SWRLDA improves LDA for multi-class classification with edge classes.
problem LDA's vulnerability to edge classes causing biased mean and large distances.
method Self-weighted robust LDA with l21-norm distance criterion.
result SWRLDA outperforms other methods on synthetic and real-world datasets.
A new LDA model with covariates for mixed-membership clusters.
problem Modeling mixed-membership clusters in discrete data with covariates.
method Negative binomial regression embedded within LDA, slice sampling within Gibbs sampling.
result Model successfully retrieves true parameter values and predicts cluster abundances using covariates.
Streaming variational Bayes (SVB) is successful in learning LDA models in an online manner. However previous attempts toward developing online Monte-Carlo methods for LDA have little success, often by having much worse perplexity than their batch counterparts. We present a streaming Gibbs sampling (SGS) method, an onli…
A new method for LDA using randomized Kaczmarz improves accuracy for large datasets.
problem Efficiently performing LDA on large datasets.
method Randomized Kaczmarz method applied to linear discriminant analysis.
result The method achieves comparable accuracy to full data LDA.
We consider moment matching techniques for estimation in Latent Dirichlet Allocation (LDA). By drawing explicit links between LDA and discrete versions of independent component analysis (ICA), we first derive a new set of cumulant-based tensors, with an improved sample complexity. Moreover, we reuse standard ICA techni…
Paper calculates the exact error of LDA models.
problem Bayesian generalization error in Latent Dirichlet Allocation (LDA).
method Theoretical analysis of learning coefficient using algebraic geometry.
result Exact asymptotic form of LDA's generalization error.
Cross-validation in high-dimensional spaces: a lifeline for least-squares models and multi-class LDAcs.LG
Least-squares models such as linear regression and Linear Discriminant Analysis (LDA) are amongst the most popular statistical learning techniques. However, since their computation time increases cubically with the number of features, they are inefficient in high-dimensional neuroimaging datasets. Fortunately, for k-fo…
Improved LDA with capped l_{2,1}-norm reduces outlier sensitivity.
problem Outliers and noise sensitivity in classical LDA.
method Introducing capped l_{2,1}-norm and proposing CLDA.
result CLDA effectively removes outliers and suppresses noise.
EFDA extends LDA to non-Gaussian models using exponential families.
problem Classifying non-Gaussian data with LDA's limitations.
method EFDA uses exponential families to derive closed-form estimators for natural parameters and a linear decision rule.
result EFDA matches LDA's accuracy while reducing ECE by 2-6x, proving asymptotic calibration and efficiency.
Unification of HDP and LDA Models for Optimal Topic Clustering of Subject Specific Question Bankscs.IR
Unified HDP and LDA models for efficient topic clustering of online course queries.
problem Efficiently cluster and answer subject-specific online course queries.
method Use Hierarchical Dirichlet Process (HDP) to optimize topic number for Latent Dirichlet Allocation (LDA) model runs.
result Achieve optimal clustering efficiency by recursively applying LDA on effective topics.
Labeled Latent Dirichlet Allocation (LLDA) is an extension of the standard unsupervised Latent Dirichlet Allocation (LDA) algorithm, to address multi-label learning tasks. Previous work has shown it to perform in par with other state-of-the-art multi-label methods. Nonetheless, with increasing label sets sizes LLDA enc…
We introduce incremental variational inference and apply it to latent Dirichlet allocation (LDA). Incremental variational inference is inspired by incremental EM and provides an alternative to stochastic variational inference. Incremental LDA can process massive document collections, does not require to set a learning …
Topic models have emerged as fundamental tools in unsupervised machine learning. Most modern topic modeling algorithms take a probabilistic view and derive inference algorithms based on Latent Dirichlet Allocation (LDA) or its variants. In contrast, we study topic modeling as a combinatorial optimization problem, and p…
ALBU improves LDA performance on small datasets.
problem Limited data impairs variational Bayes in LDA.
method Approximate Loopy Belief message passing algorithm.
result ALBU learns latent distributions more accurately than VB, especially for smaller data sets.
We review three algorithms for Latent Dirichlet Allocation (LDA). Two of them are variational inference algorithms: Variational Bayesian inference and Online Variational Bayesian inference and one is Markov Chain Monte Carlo (MCMC) algorithm -- Collapsed Gibbs sampling. We compare their time complexity and performance.…
Topic models (e.g., pLSA, LDA, SLDA) have been widely used for segmenting imagery. These models are confined to crisp segmentation. Yet, there are many images in which some regions cannot be assigned a crisp label (e.g., transition regions between a foggy sky and the ground or between sand and water at a beach). In the…
Study on Gaussian-width complexity on statistical manifolds and its applications in learning and recovery.
problem Understanding the geometry of statistical manifolds and its implications for learning and recovery.
method Analysis of Fisher width and inverse-Fisher width, proving their complementary roles and establishing a relation between them.
result Established a sharp relation between Fisher width and inverse-Fisher width, showing they cannot reduce relative to Euclidean scale.
Fisher Auto-Encodersstat.ML
Fisher auto-encoders use Fisher divergence for more robust generative modeling.
problem Model uncertainty in generative models.
method Minimizing Fisher divergence between true and modeled joint distributions.
result Fisher auto-encoders can more accurately quantify model uncertainty.
NMF with specific constraints is equivalent to LDA.
problem Dimensionality reduction of non-negative data.
method NMF with normalization constraints and Dirichlet prior.
result NMF with these constraints is equivalent to LDA.
We propose a novel interpretation of the collapsed variational Bayes inference with a zero-order Taylor expansion approximation, called CVB0 inference, for latent Dirichlet allocation (LDA). We clarify the properties of the CVB0 inference by using the alpha-divergence. We show that the CVB0 inference is composed of two…