Naive Bayes can be used as a discriminative classifier, matching the definition of logistic regression.
problem The definition of generative and discriminative classifiers.
method Comparing Naive Bayes and logistic regression, showing they can be used in either generative or discriminative ways.
result Naive Bayes can be used as a discriminative classifier.
Smart Bayes integrates generative and discriminative features for improved classification.
problem Improving classification performance by combining generative and discriminative modeling.
method Integrates generative likelihood-ratio features into a logistic-regression-style classifier.
result Often outperforms logistic regression and Naive Bayes in simulations and real data.
A new DRM classifier for high-dimensional and imbalanced data.
problem Classification in high-dimensional and imbalanced data.
method Discriminative regression approach with iterative algorithms.
result Superior performance compared to state-of-the-art classifiers.
We study Bayesian discriminative inference given a model family $p(c,\x, θ)$ that is assumed to contain all our prior information but still known to be incorrect. This falls in between "standard" Bayesian generative modeling and Bayesian regression, where the margin $p(\x,θ)$ is known to be uninformative about $p(c|\x,…
Incorporating feature selection into a classification or regression method often carries a number of advantages. In this paper we formalize feature selection specifically from a discriminative perspective of improving classification/regression accuracy. The feature selection method is developed as an extension to the r…
Paper speeds up and extends Subclass Discriminant Analysis methods.
problem Improving efficiency and handling multi-view data in Subclass Discriminant Analysis.
method Developed a speed-up approach based on graph embedding and spectral regression, and a novel multi-view solution.
result Proposed methods achieve competitive performance and significantly decrease training time.
Paper introduces a method to improve survival model calibration without sacrificing discrimination.
problem Survival models struggle to balance accurate ranking and event prediction.
method Uses conformal regression to enhance calibration without compromising discrimination.
result The approach improves model calibration across 11 real-world datasets.
In this paper, we study the modeling and the classification of functional data presenting regime changes over time. We propose a new model-based functional mixture discriminant analysis approach based on a specific hidden process regression model that governs the regime changes over time. Our approach is particularly a…
Discriminative latent-variable models are typically learned using EM or gradient-based optimization, which suffer from local optima. In this paper, we develop a new computationally efficient and provably consistent estimator for a mixture of linear regressions, a simple instance of a discriminative latent-variable mode…
CcGAN tackles conditional image generation for continuous labels.
problem Mathematical challenges in conditioning on continuous, scalar labels.
method Proposes novel empirical losses and label input methods for continuous conditional GANs.
result CcGAN generates diverse, high-quality images from continuous labels.
Improved language models by incorporating statistical discriminators.
problem Distinguishing model-generated text from real text reliably.
method Energy-Based Model framework to incorporate discriminators.
result Improves language model performance in perplexity and human evaluation.
New model selects uncorrelated and discriminative features for unsupervised feature selection.
problem Selecting uncorrelated and discriminative features in high-dimensional data.
method Adaptive graph-based generalized regression model with uncorrelated constraint and ℓ2,1-norm regularization. result The model effectively selects uncorrelated and discriminative features, improving clustering performance.
We consider an enlarged dimension reduction space in functional inverse regression. Our operator and functional analysis based approach facilitates a compact and rigorous formulation of the functional inverse regression problem. It also enables us to expand the possible space where the dimension reduction functions bel…
EBMs improve text discrimination by generating negatives from auto-regressive models.
problem Discriminating machine-generated text from human-generated text.
method Use energy-based models to discriminate text, generating negatives using pre-trained auto-regressive language models.
result EBMs can generalize well to changes in generator architectures but are sensitive to training set.
Paper detects proxies in linear regression models causing discrimination.
problem Discrimination in machine learning models using proxies for protected attributes.
method Formulated a definition of proxy use, identified proxies via second-order cone program, and extended to justified business necessity.
result Proxies in linear regression models can be efficiently identified and removed to reduce discrimination.
We present a new mixture model-based discriminant analysis approach for functional data using a specific hidden process regression model. The approach allows for fitting flexible curve-models to each class of complex-shaped curves presenting regime changes. The model parameters are learned by maximizing the observed-da…
Proposes a novel linear discriminant analysis for matrix-valued data.
problem Classification of high-dimensional matrix-valued data from imaging studies.
method Efficient nuclear norm penalized regression with low-rank structure.
result Superior performance compared to existing methods in simulations and EEG data.
A new approach for functional data description is proposed in this paper. It consists of a regression model with a discrete hidden logistic process which is adapted for modeling curves with abrupt or smooth regime changes. The model parameters are estimated in a maximum likelihood framework through a dedicated Expectat…
Statistical approaches for Functional Data Analysis concern the paradigm for which the individuals are functions or curves rather than finite dimensional vectors. In this paper, we particularly focus on the modeling and the classification of functional data which are temporal curves presenting regime changes over time.…
Deep clustering models are shown to be equivalent to K-means under certain conditions.
problem The challenge of unsupervised deep learning and clustering.
method The study proves the equivalence of recent discriminative models and K-means under specific conditions and parameters.
result Maximizing the L2 regularized mutual information is equivalent to a soft and regularized K-means loss.
Locally Valid and Discriminative prediction intervals for deep learning models.
problem Efficient and theoretically sound uncertainty quantification for deep learning models.
method Locally Valid and Discriminative prediction intervals (LVD) using kernel regression.
result Locally Valid and Discriminative prediction intervals (LVD) offer better performance and scalability compared to existing methods.
Paper tackles expected predictions computation for arbitrary generative models.
problem Hard to compute expected predictions for arbitrary generative models.
method Identifies tractable generative and discriminative models for expected predictions.
result Tractable computation of high-order moments and expectations for classification.
We present a novel approach to the formulation and the resolution of sparse Linear Discriminant Analysis (LDA). Our proposal, is based on penalized Optimal Scoring. It has an exact equivalence with penalized LDA, contrary to the multi-class approaches based on the regression of class indicator that have been proposed s…
The paper revisits discriminative vs. generative classifiers, showing naive Bayes requires fewer samples.
problem Comparing discriminative and generative classifiers in multiclass settings.
method Theoretical analysis and simulations of naive Bayes vs. logistic regression.
result Multiclass naive Bayes requires fewer samples to approach asymptotic error compared to logistic regression.
Explains LDA and QDA for binary and multiple classes.
problem Classification methods in statistical and probabilistic learning.
method Optimization of decision boundaries, estimation of parameters, relation to other methods.
result Equivalence of LDA and Fisher discriminant analysis.
Efficient algorithm for contextual bandits with first-order guarantees.
problem Adapting to low noise in contextual bandits.
method Reduction to online regression with logarithmic loss.
result Optimal and efficient first-order guarantees for contextual bandits.
Generative adversarial networks improved for regression tasks.
problem Improving neural network training with semi-supervised data.
method Feature contrasting loss function for discriminator training.
result Semi-supervised GANs can be effectively applied to regression problems.
New method compresses large sample data for faster discriminant analysis.
problem Large sample sizes in discriminant analysis increase computational burden.
method Proposes a new compression approach for reducing training samples.
result Significant computational gains and superior predictive ability compared to random sub-sampling.
New models reduce discrimination in machine learning without sacrificing explanatory bias.
problem Discrimination and explanatory bias in fairness measures.
method Causal effect estimators using propensity score analysis.
result Theoretical and practical superiority of FairCEEs over existing models.
We provide a unified analysis of the predictive risk of ridge regression and regularized discriminant analysis in a dense random effects model. We work in a high-dimensional asymptotic regime where p,n→∞ and p/n→γ∈(0,∞), and allow for arbitrary covariance among the features. For both metho…
Multi-output Gaussian processes (MOGP) are probability distributions over vector-valued functions, and have been previously used for multi-output regression and for multi-class classification. A less explored facet of the multi-output Gaussian process is that it can be used as a generative model for vector-valued rando…
We consider a discriminative learning (regression) problem, whereby the regression function is a convex combination of k linear classifiers. Existing approaches are based on the EM algorithm, or similar techniques, without provable guarantees. We develop a simple method based on spectral techniques and a `mirroring' tr…
Proposes RLAR for efficient labeled data classification with robust margin and manifold structure.
problem Clear margin representation and data manifold structure difficulty in linear discriminant methods.
method Introduces retargeted regression for adaptive margin learning and locality-aware strategy for compact data manifold.
result RLAR outperforms state-of-the-art approaches in UCI and benchmark data sets.
Discriminative jackknife estimates deep learning uncertainty.
problem Quantifying uncertainty in deep learning models.
method Discriminative jackknife using influence functions of loss.
result DJ satisfies frequentist coverage and discriminative accuracy.
Slow feature analysis (SFA) is an unsupervised learning algorithm that extracts slowly varying features from a time series. Graph-based SFA (GSFA) is a supervised extension that can solve regression problems if followed by a post-processing regression algorithm. A training graph specifies arbitrary connections between …
Improved SVRG for quadratic functions achieves better performance and running times.
problem Minimizing quadratic functions with a specific type of Hessian matrix.
method Variant of SVRG algorithm for quadratic functions with improved analysis.
result Improved performance and running times for quadratic functions compared to state-of-the-art methods.
The paper analyzes the statistical properties of GANs using f-divergence.
problem Understanding the statistical behavior of GANs and comparing different f-divergences. method Asymptotic analysis of f-divergence GANs, including Kullback-Leibler divergence. result Asymptotically equivalent GANs with the same discriminator classes for correctly specified models.
New method detects and prevents unfairness in few-shot regression models.
problem Fairness issues in supervised few-shot meta-learning models.
method Causal Bayesian knowledge graph for dependency visualization, risk difference quantification, and fast-adapted bias-control approach.
result Efficiently detects and mitigates unfairness in model predictions.
Study predicts MSE survival using feature selection with missing data imputation methods.
problem Predicting survival of micro and small businesses (MSE).
method Feature selection with missing data imputation methods (MI, KNN, EM). Comparison of data mining techniques (logistic regression, naive Bayes, LDA, SVM).
result Developed a model to predict MSE survival.
Study on merging predictors in causal and anticausal directions using CMAXENT.
problem Comparing merging predictors in causal and anticausal directions.
method Using CMAXENT as inductive bias, study differences in merging predictors.
result CMAXENT solution reduces to logistic regression in causal direction and LDA in anticausal direction.
Several dihedral angles prediction methods were developed for protein structure prediction and their other applications. However, distribution of predicted angles would not be similar to that of real angles. To address this we employed generative adversarial networks (GAN). Generative adversarial networks are composed …
Paper proposes a method to identify and treat latent discriminating features in machine learning models.
problem Fairness issues in machine learning models trained on historical data containing sensitive attributes.
method A novel algorithm that identifies and treats latent discriminating features, agnostic of the learning algorithm.
result Experimental results show near-ideal fairness measurement compared to other methods.
Paper compares generative and discriminative models in uncertainty quantification.
problem Comparing generative and discriminative approaches in uncertainty quantification.
method Analysis of generative and discriminative models, focusing on posterior predictive distribution and prior distributions.
result Discriminative models struggle with imbalanced datasets, while generative models offer a more flexible prior distribution.
cMIM improves representation learning without positive-pair augmentations.
problem Learning robust representations for diverse tasks.
method Contrastive Mutual Information Machine (cMIM) framework.
result cMIM outperforms MIM and InfoNCE on classification and regression tasks.
Automatic image annotation (AIA) raises tremendous challenges to machine learning as it requires modeling of data that are both ambiguous in input and output, e.g., images containing multiple objects and labeled with multiple semantic tags. Even more challenging is that the number of candidate tags is usually huge (as …
This paper compares machine learning algorithms for ecological data.
problem Classifying ecological datasets into subsets with common patterns.
method Applied eight machine learning classification algorithms to ecological data.
result Linear Discriminant Analysis and k-nearest neighbors are the best methods.
The paper bounds neural networks' approximation error and applies it to regression and GANs.
problem Bounding the approximation error of norm-constrained neural networks.
method Proved upper and lower bounds on approximation error using Rademacher complexity.
result Obtained convergence rates for over-parameterized neural networks and optimal GAN learning rates.
Develops experimental design for discovering missing physics in bioreactors.
problem Discovering missing physics in incomplete model structures of process systems.
method Combines universal differential equations and symbolic regression with sequential experimental design.
result Successfully recovered true model structure of a bioreactor using machine learning techniques.