We study the rates of convergence from empirical surrogate risk minimizers to the Bayes optimal classifier. Specifically, we introduce the notion of \emph{consistency intensity} to characterize a surrogate loss function and exploit this notion to obtain the rate of convergence from an empirical surrogate risk minimizer…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Bayes-optimal classifier”
Unified framework for Bayes-optimal classifiers under group fairness.
problem Mitigating disparate impacts from algorithmic predictions in high-stakes decision-making.
method Unified framework based on Neyman-Pearson argument for deriving Bayes-optimal classifiers under group fairness constraints.
result Proposes FairBayes method that directly controls disparity and achieves optimal fairness-accuracy tradeoff.
Bayes-optimal classifiers are robust to adversarial attacks, unlike CNNs trained on the same data.
problem The vulnerability of modern CNN classifiers to adversarial examples.
method Constructing realistic image datasets and deriving analytic conditions for Bayes-optimal classifiers.
result Bayes-optimal classifiers are robust to adversarial attacks, unlike CNNs trained on the same data.
The paper tackles fair classification with multiple sensitive features.
problem Existing fair classification methods often consider a single sensitive feature, but in practice, individuals are defined by multiple sensitive features.
method Characterizes Bayes-optimal fair classifiers for multiple sensitive features under various fairness measures, proposing in-processing and post-processing algorithms.
result Bayes-optimal fair classifiers for multiple sensitive features are instance-dependent thresholding rules that rely on a weighted sum of group membership probabilities.
The paper proves statistical consistency and fairness guarantees for a plug-in algorithm.
problem Establishing statistical guarantees for fairness-aware binary classification.
method Proves statistical consistency and derives finite sample guarantees for the plug-in algorithm.
result The plug-in algorithm is statistically consistent and guarantees fairness and differential privacy.
Develops methods for fair classification under linear disparity constraints.
problem Disparate impacts of machine learning algorithms on protected groups.
method Bayes-optimal fair classification methods via pre-, in-, and post-processing.
result Explicit forms of Bayes-optimal fair classifiers under linear disparity measures.
Study on conditions for Bayes optimal classifier in adversarial robustness.
problem Existence of Bayes optimal classifier for adversarial robustness.
method General sufficient conditions for existence of Bayes optimal classifier.
result Guaranteed existence of Bayes optimal classifier under certain conditions.
Unified approach for fair classification with overlapping groups.
problem Ensuring fairness across multiple overlapping groups in prediction problems.
method Probabilistic population analysis leading to Bayes-optimal classifier, unifying existing methods.
result Outperforms baselines in fairness-performance tradeoff on real datasets.
This paper tackles fair Bayes-optimal classifiers under predictive parity, proving their limitations and proposing a new algorithm.
problem Ensuring fair Bayes-optimal classifiers under predictive parity, especially when group performance levels vary widely.
method Proving the limitations of fair Bayes-optimal classifiers under predictive parity and proposing a new adaptive thresholding algorithm, FairBayes-DPP.
result Fair Bayes-optimal classifiers under predictive parity may not hold if group performance levels vary widely, leading to within-group unfairness.
The paper explores robust classifiers for imbalanced Gaussian data.
problem Adversarial robustness in machine learning with imbalanced data.
method Developed exact and approximate Bayes-optimal robust classifiers for Gaussian classification problems.
result Revealed fundamental tradeoffs between standard and robust accuracy.
The study sets limits on how robust classifiers can be against adversarial attacks.
problem Understanding the limits of robustness in classification models against adversarial attacks.
method Utilized optimal transport theory to derive variational formulae and explicit lower-bounds on Bayes-optimal error.
result Explicit lower-bounds on the Bayes-optimal error for distance-based attacks, universal in geometry of class-conditional distributions.
Meta-learning method improves PU classification performance.
problem Improving binary classifiers from PU data in unseen tasks.
method Adapts model to PU data using related tasks and neural networks.
result Proposed method outperforms existing methods on synthetic and real-world datasets.
Adaptive classifier optimizes high-dimensional data with spiked covariance structure.
problem Classification of high-dimensional data with spiked covariance structure.
method Adaptive classifier that whitens data, screens features, and applies Fisher linear discriminant.
result The classifier is Bayes optimal under certain conditions and performs well on real and synthetic data.
Optimal classifiers derived from GMMs are approximated by deep neural networks.
problem Binary classification of high-dimensional overlapping Gaussian mixtures.
method Closed-form expressions for Bayes optimal decision boundaries derived from GMMs' eigenstructure. Empirical validation through synthetic and real-world data.
result Deep neural networks approximate optimal classifiers for GMMs, with decision thresholds related to covariance eigenvectors.
Study shows how classifiers can approach Bayes error in high-dimensional settings.
problem Generalization error in high-dimensional perceptrons.
method Proved a formula for generalization error using convex optimization and observed that logistic and hinge regression can approach Bayes error closely.
result Logistic and hinge regression can approach Bayes-optimal generalization error closely in high-dimensional settings.
Optimal graph classification uses message-passing neural networks.
problem Node classification on sparse graphs with fixed feature dimensions.
method Asymptotic local Bayes optimality, message-passing graph neural networks.
result Optimal message-passing architecture interpolates between MLP and convolution.
Improved conformal prediction for better conditional coverage of classifier predictions.
problem Achieving exact conditional coverage in finite samples for prediction sets.
method Developed a variant of conformal prediction targeting coverage conditional on confidence and trust score.
result Empirically improved conditional coverage properties compared to standard conformal prediction.
Instance- and Label-dependent label Noise (ILN) widely exists in real-world datasets but has been rarely studied. In this paper, we focus on Bounded Instance- and Label-dependent label Noise (BILN), a particular case of ILN where the label noise rates -- the probabilities that the true labels of examples flip into the …
Bad models can teach well by replicating noise.
problem Overparameterized models can replicate noise in training data.
method Knowledge distillation from noisy samplers.
result Distillation from samplers approximates Bayes optimal classifier.
We investigate the problem of multiclass classification with rejection, where a classifier can choose not to make a prediction to avoid critical misclassification. First, we consider an approach based on simultaneous training of a classifier and a rejector, which achieves the state-of-the-art performance in the binary …
New bounds show robust models can generalize well, contrary to prior theories.
problem Existing robustness-based error bounds are vacuous for the best classifier.
method Developed novel bounds that converge to the true error of the best classifier.
result New bounds converge to the true error of the best classifier, improving generalization.
This paper improves binary classification methods beyond accuracy, especially in imbalanced datasets.
problem Binary classification performance metrics often fail to reflect real-world consequences, especially in imbalanced datasets.
method Derives a generalized Bayes-optimal classifier from accuracy to any performance metric, removing assumptions and providing finite-sample statistical guarantees.
result Optimal classification performance depends on class imbalance properties, providing new insights and guarantees.
Ensemble Quantile Classifierstat.ML
Both the median-based classifier and the quantile-based classifier are useful for discriminating high-dimensional data with heavy-tailed or skewed inputs. But these methods are restricted as they assign equal weight to each variable in an unregularized way. The ensemble quantile classifier is a more flexible regularize…
Focal loss improves classification but not class-posterior probability estimation.
problem Improving class-posterior probability estimation from focal loss.
method Proved classification-calibration and derived a transformation to recover true class-posterior probabilities.
result A transformation of the confidence score from focal loss minimization allows recovery of true class-posterior probabilities.
We analyze the problem of learning a single user's preferences in an active learning setting, sequentially and adaptively querying the user over a finite time horizon. Learning is conducted via choice-based queries, where the user selects her preferred option among a small subset of offered alternatives. These queries …
Regularization helps improve classification of noisy high-dimensional data.
problem Classifying high-dimensional noisy Gaussian mixture with limited oracle knowledge.
method Analysis of regularized convex classifiers including ridge, hinge, and logistic regression.
result Regularization can reach Bayes-optimal performance under certain conditions.
Robust Vertex Classificationstat.ML
For random graphs distributed according to stochastic blockmodels, a special case of latent position graphs, adjacency spectral embedding followed by appropriate vertex classification is asymptotically Bayes optimal; but this approach requires knowledge of and critically depends on the model dimension. In this paper, w…
Bayesian model fuses multiple classifiers with explicit correlation modeling.
problem Combining outputs of multiple classifiers with explicit correlation.
method Hierarchical Bayesian model with correlated Dirichlet distribution.
result Fused classifier performance can be Bayes optimal even for highly correlated base classifiers.
Paper studies fair classification of functional data.
problem Mitigating disparities in functional data classification.
method Unified framework for fairness-aware functional classification.
result Established theoretical guarantees on fairness and excess risk controls.
Analysis of Diagnostics (Part I): Prevalence, Uncertainty Quantification, and Machine Learningstat.ML
This study connects prevalence and machine learning for diagnostic testing.
problem Uncertainty quantification in machine learning for diagnostic tests.
method Developed a numerical homotopy algorithm to estimate classification boundaries and quantify uncertainty.
result The proposed method stabilizes uncertainty quantification in machine learning for diagnostic tests.
We propose a framework for constructing and analyzing multiclass and multioutput classification metrics, i.e., involving multiple, possibly correlated multiclass labels. Our analysis reveals novel insights on the geometry of feasible confusion tensors -- including necessary and sufficient conditions for the equivalence…
Paper studies multiclass classifiers from binary classifiers, proving methods and demonstrating advantages.
problem Constructing efficient multiclass classifiers from binary ones.
method Two methods: one vs. all and hierarchical classification, with a new leverage-hierarchical method introduced.
result Proves upper bounds and exact formulas for multiclass regret in terms of binary regrets.
Bayes-optimal learning of deep random networks with Gaussian weights is studied.
problem Learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights.
method Closed-form expressions for Bayes-optimal test error, ridge regression, kernel and random features regression are computed.
result Optimally regularized ridge regression and kernel regression achieve Bayes-optimal performances, while logistic loss yields a near-optimal test error for classification.
Motivated by safety-critical applications, test-time attacks on classifiers via adversarial examples has recently received a great deal of attention. However, there is a general lack of understanding on why adversarial examples arise; whether they originate due to inherent properties of data or due to lack of training …
Adversarially robust machine learning has received much recent attention. However, prior attacks and defenses for non-parametric classifiers have been developed in an ad-hoc or classifier-specific basis. In this work, we take a holistic look at adversarial examples for non-parametric classifiers, including nearest neig…
Knowing when a classifier's prediction can be trusted is useful in many applications and critical for safely using AI. While the bulk of the effort in machine learning research has been towards improving classifier performance, understanding when a classifier's predictions should and should not be trusted has received …
In this work we show that, using the eigen-decomposition of the adjacency matrix, we can consistently estimate feature maps for latent position graphs with positive definite link function , provided that the latent positions are i.i.d. from some distribution F. We then consider the exploitation task of vertex classi…
Given a graph in which a few vertices are deemed interesting a priori, the vertex nomination task is to order the remaining vertices into a nomination list such that there is a concentration of interesting vertices at the top of the list. Previous work has yielded several approaches to this problem, with theoretical re…
SPRT-TANDEM improves sequential classification accuracy with fewer samples.
problem Efficiently classifying sequential data with high accuracy and low sampling cost.
method Deep neural network-based SPRT algorithm that estimates log-likelihood ratio of two hypotheses.
result SPRT-TANDEM achieves statistically significantly better classification accuracy than other classifiers with fewer samples.
ContraBAR uses contrastive learning to learn Bayes-optimal policies in RL.
problem Learning optimal policies for unknown tasks sampled from a known distribution.
method Proposes ContraBAR, a meta RL algorithm using contrastive predictive coding (CPC) for belief inference.
result ContraBAR achieves comparable performance to state-of-the-art methods and is computationally efficient.
Bayes optimal algorithm under certain conditions doesn't achieve exponential simple regret.
problem Best arm identification with normal rewards over time.
method Fixed-budget best arm identification problem with rewards from normal distributions. Evaluates performance via simple regret.
result Bayes optimal algorithm does not yield exponential decrease in simple regret.
Self-training improves weak classifiers in mixture models.
problem Improving weak classifiers in mixture models.
method Iterative self-training algorithm using pseudolabels and unlabeled data.
result Self-training converts weak learners to strong learners in mixture models.
The paper studies binary classification and aims at estimating the underlying regression function which is the conditional expectation of the class labels given the inputs. The regression function is the key component of the Bayes optimal classifier, moreover, besides providing optimal predictions, it can also assess t…
Study of Bayes optimal learning in high-dimensional linear regression with network side information.
problem Bayes optimal learning in high-dimensional linear regression with network side information.
method Introduce a Reg-Graph model and an iterative AMP algorithm for Bayes optimality under general conditions.
result Characterization of the limiting mutual information between latent signal and data observed.
The study analyzes multi-class teacher-student perceptron performance and generalization errors.
problem Analyzing multi-class classification with the teacher-student perceptron.
method Deriving asymptotic expressions for Bayes-optimal and empirical risk minimization (ERM) generalization errors.
result Regularised cross-entropy minimization yields close-to-optimal accuracy for multi-class classification.
Minimizes indecisions in selective classification to control misclassification rates.
problem Controlling misclassification rates in high-risk scenarios.
method Using indecisions to control misclassification rates, even below Bayes optimal.
result Control of misclassification rates to any user-specified level, even below Bayes optimal.
Bayes-optimal learning of a neural network with quadratic activations is achieved with GAMP-RIE.
problem Learning a neural network with quadratic activations from quadratic samples.
method Combining approximate message passing with rotationally invariant matrix denoising.
result Derives a closed-form expression for Bayes-optimal test error.
In this work we show that, using the eigen-decomposition of the adjacency matrix, we can consistently estimate latent positions for random dot product graphs provided the latent positions are i.i.d. from some distribution. If class labels are observed for a number of vertices tending to infinity, then we show that the …