New uniqueness concept for adversarial Bayes classifier.
problem Understanding adversarial Bayes classifiers in binary classification.
method Developed a new notion of uniqueness and analyzed it for a family of one-dimensional data distributions.
result Improved regularity of adversarial Bayes classifiers as perturbation radius increases.
The study compares clustering risk in Hidden Markov and i.i.d. models, showing the Bayes classifier is nearly optimal.
problem Comparing clustering risk in Hidden Markov and i.i.d. models.
method Analysis of Bayes risk, theoretical bounds, and simulations.
result The Bayes classifier is nearly optimal for clustering in both Hidden Markov and i.i.d. models.
Adversarial consistency depends on the uniqueness of adversarial Bayes classifiers.
problem Consistency of adversarial surrogate losses is not guaranteed.
method Connected consistency of adversarial surrogate losses to the uniqueness of adversarial Bayes classifiers.
result A convex surrogate loss is statistically consistent for adversarial learning if and only if the adversarial Bayes classifier is unique.
Improved Naive Bayes classifier with neural network models.
problem Limited complexity handling and independence assumption in Naive Bayes.
method Introducing Neural Naive Bayes and Neural Pooled Markov Chain models.
result Error rate reduced by 4.5 on IMDB dataset.
Bayes classifier cannot be learned from noisy labels without knowing noise distribution.
problem Learning a Bayes classifier from noisy labels when the noise distribution is unknown.
method Demonstrates the identifiability issues and proposes a simple algorithm for learning the Bayes decision rule.
result The Bayes decision rule is generally unidentified and cannot be learned without knowing the noise distribution.
The paper compares one-hot encoding to Naïve Bayes for categorical variables.
problem Incorrect one-hot encoding affects Naïve Bayes performance.
method Mathematical and experimental analysis of PoB vs. categorical Naïve Bayes.
result Posterior probabilities are usually greater in the PoB case, but agree on the maximum a posteriori class label.
Study on the structure of classifier boundaries in DNA sequencing.
problem Understanding the structure of boundaries in a Bayes classifier for DNA sequencing.
method Examined the structure of the boundary in a Bayes classifier applied to DNA sequencing data. Introduced a new measure of uncertainty, Neighbor Similarity.
result The boundary is large and complex, and Neighbor Similarity effectively measures classifier uncertainty.
We study the rates of convergence from empirical surrogate risk minimizers to the Bayes optimal classifier. Specifically, we introduce the notion of \emph{consistency intensity} to characterize a surrogate loss function and exploit this notion to obtain the rate of convergence from an empirical surrogate risk minimizer…
Proposes a non-convex optimization method for a parsimonious weighted naive Bayes classifier.
problem Improving naïve Bayes classifier performance with a large number of input variables.
method Sparse regularization of model log-likelihood for direct estimation of variable weights.
result Optimization-based weighted naïve Bayes classifiers achieve equivalent performance to averaging-based classifiers.
Naive Bayes can be used as a discriminative classifier, matching the definition of logistic regression.
problem The definition of generative and discriminative classifiers.
method Comparing Naive Bayes and logistic regression, showing they can be used in either generative or discriminative ways.
result Naive Bayes can be used as a discriminative classifier.
Bayesian network classifiers are used in many fields, and one common class of classifiers are naive Bayes classifiers. In this paper, we introduce an approach for reasoning about Bayesian network classifiers in which we explicitly convert them into Ordered Decision Diagrams (ODDs), which are then used to reason about t…
Uniform Closure Method and Bayes classifier perform similarly in classifying open knots.
problem Classifying knots in open macromolecular chains.
method Used the Bayes MAP classifier and compared it to the Uniform Closure Method.
result Both methods have comparable accuracy and positive predictive value.
The paper derives a formula for factorizing categorical data to improve Bayes classifiers.
problem Improving the accuracy of Bayes classifiers by effectively factoring multidimensional data.
method Derives an explicit formula for calculating the marginal likelihood of a factorized categorical dataset.
result The derived formula can be used to select the best factorization for constructing a Bayes classifier.
Unified framework for Bayes-optimal classifiers under group fairness.
problem Mitigating disparate impacts from algorithmic predictions in high-stakes decision-making.
method Unified framework based on Neyman-Pearson argument for deriving Bayes-optimal classifiers under group fairness constraints.
result Proposes FairBayes method that directly controls disparity and achieves optimal fairness-accuracy tradeoff.
This paper proposes an efficient method for calculating Shapley values in Naive Bayes classifiers.
problem The need for explaining machine learning model decisions.
method An exact analytic expression of Shapley values for Naive Bayes classifiers.
result The proposed Shapley values provide informative results with low complexity and low computation time.
In this paper we present a new Bayesian network model for classification that combines the naive-Bayes (NB) classifier and the finite-mixture (FM) classifier. The resulting classifier aims at relaxing the strong assumptions on which the two component models are based, in an attempt to improve on their classification pe…
There is a rising interest in studying the robustness of deep neural network classifiers against adversaries, with both advanced attack and defence techniques being actively developed. However, most recent work focuses on discriminative classifiers, which only model the conditional distribution of the labels given the …
We show that a simple modification of the 1-nearest neighbor classifier yields a strongly Bayes consistent learner. Prior to this work, the only strongly Bayes consistent proximity-based method was the k-nearest neighbor classifier, for k growing appropriately with sample size. We will argue that a margin-regularized 1…
In this paper, we empirically evaluate algorithms for learning four types of Bayesian network (BN) classifiers - Naive-Bayes, tree augmented Naive-Bayes, BN augmented Naive-Bayes and general BNs, where the latter two are learned using two variants of a conditional-independence (CI) based BN-learning algorithm. Experime…
New algorithms explain Naive Bayes classifiers in polynomial time and delay.
problem Computing explanations for Naive Bayes classifiers efficiently.
method Developed log-linear time and polynomial delay algorithms for PI-explanations.
result Efficiently computed PI-explanations for linear classifiers.
PAC-Bayesian framework for fairness in stochastic and deterministic classifiers.
problem Theoretical guarantees on fairness for balancing predictive risk and fairness constraints.
method PAC-Bayesian framework for both stochastic and deterministic classifiers, covering a broad class of fairness measures.
result Derives generalization bounds for fairness, demonstrating tightness with empirical evaluation.
The presence of noisy instances in mobile phone data is a fundamental issue for classifying user phone call behavior (i.e., accept, reject, missed and outgoing), with many potential negative consequences. The classification accuracy may decrease and the complexity of the classifiers may increase due to the number of re…
Study on conditions for Bayes optimal classifier in adversarial robustness.
problem Existence of Bayes optimal classifier for adversarial robustness.
method General sufficient conditions for existence of Bayes optimal classifier.
result Guaranteed existence of Bayes optimal classifier under certain conditions.
Paper estimates FPR of Bayes classifier using soft labels.
problem Determining optimal classifier performance.
method Uses soft labels and denoising technique.
result Consistent and unbiased FPR estimator developed.
The paper revisits discriminative vs. generative classifiers, showing naive Bayes requires fewer samples.
problem Comparing discriminative and generative classifiers in multiclass settings.
method Theoretical analysis and simulations of naive Bayes vs. logistic regression.
result Multiclass naive Bayes requires fewer samples to approach asymptotic error compared to logistic regression.
New method certifies deep graph classifiers with tighter risk bounds.
problem Certifying the reliability of deep graph classifiers.
method Linearized deep assignment flows with random initial conditions, using PAC-Bayes risk certification.
result Computes tighter out-of-sample risk certificates efficiently.
Paper explores Bayes rule for Gaussian mixtures with missing data, outperforming supervised classifiers.
problem Improving classification accuracy in partially classified samples with missing data.
method Generative model framework with missing-data mechanism, Bayes rule allocation.
result Bayes rule classifier with missing-data mechanism outperforms fully supervised classifiers in various conditions.
The paper tackles fair classification with multiple sensitive features.
problem Existing fair classification methods often consider a single sensitive feature, but in practice, individuals are defined by multiple sensitive features.
method Characterizes Bayes-optimal fair classifiers for multiple sensitive features under various fairness measures, proposing in-processing and post-processing algorithms.
result Bayes-optimal fair classifiers for multiple sensitive features are instance-dependent thresholding rules that rely on a weighted sum of group membership probabilities.
Two new Hie-TAN and Hie-TAN-Lite algorithms improve TAN for hierarchical feature spaces.
problem Learning dependencies in hierarchical feature spaces.
method Exploits hierarchical parent-child relationships as constraints to learn a dependency tree.
result Hie-TAN-Lite outperforms Hie-TAN and other methods in predictive accuracy.
We propose an empirical Bayes estimator based on Dirichlet process mixture model for estimating the sparse normalized mean difference, which could be directly applied to the high dimensional linear classification. In theory, we build a bridge to connect the estimation error of the mean difference and the misclassificat…
We compare in this paper several feature selection methods for the Naive Bayes Classifier (NBC) when the data under study are described by a large number of redundant binary indicators. Wrapper approaches guided by the NBC estimation of the classification error probability out-perform filter approaches while retaining …
The paper proves statistical consistency and fairness guarantees for a plug-in algorithm.
problem Establishing statistical guarantees for fairness-aware binary classification.
method Proves statistical consistency and derives finite sample guarantees for the plug-in algorithm.
result The plug-in algorithm is statistically consistent and guarantees fairness and differential privacy.
We address the problem of learning to benchmark the best achievable classifier performance. In this problem the objective is to establish statistically consistent estimates of the Bayes misclassification error rate without having to learn a Bayes-optimal classifier. Our learning to benchmark framework improves on previ…
Since the discovery of adversarial examples - the ability to fool modern CNN classifiers with tiny perturbations of the input, there has been much discussion whether they are a "bug" that is specific to current neural architectures and training methods or an inevitable "feature" of high dimensional geometry. In this pa…
Study shows how classifiers can approach Bayes error in high-dimensional settings.
problem Generalization error in high-dimensional perceptrons.
method Proved a formula for generalization error using convex optimization and observed that logistic and hinge regression can approach Bayes error closely.
result Logistic and hinge regression can approach Bayes-optimal generalization error closely in high-dimensional settings.
Despite its simplicity, the naive Bayes classifier has surprised machine learning researchers by exhibiting good performance on a variety of learning problems. Encouraged by these results, researchers have looked to overcome naive Bayes primary weakness - attribute independence - and improve the performance of the algo…
Study generalization of voting classifiers using margin-based bounds.
problem Understanding the generalization of ensemble classifiers like voting.
method Proved margin-based generalization bounds using PAC-Bayes theory and Dirichlet posteriors.
result Provided state-of-the-art guarantees on classification tasks.
New method computes discriminative classifiers from generative models.
problem Discriminative vs generative classifiers are often seen as distinct, but this work shows they can be equivalent.
method General theoretical result showing generative classifiers can be computed discriminatively.
result Bayesian Maximum Posterior classifier from generative models matches discriminative classifier definition.
Classical clustering algorithms typically either lack an underlying probability framework to make them predictive or focus on parameter estimation rather than defining and minimizing a notion of error. Recent work addresses these issues by developing a probabilistic framework based on the theory of random labeled point…
Pac-Bayes bounds are among the most accurate generalization bounds for classifiers learned from independently and identically distributed (IID) data, and it is particularly so for margin classifiers: there have been recent contributions showing how practical these bounds can be either to perform model selection (Ambrol…
The paper studies a stochastic majority vote approach to improve classifier accuracy.
problem Improving classifier accuracy over ensembles of classifiers.
method Minimizing a PAC-Bayes generalization bound with Dirichlet distributions.
result Achieves state-of-the-art accuracy and tight generalization bounds.
Improved conformal prediction for better conditional coverage of classifier predictions.
problem Achieving exact conditional coverage in finite samples for prediction sets.
method Developed a variant of conformal prediction targeting coverage conditional on confidence and trust score.
result Empirically improved conditional coverage properties compared to standard conformal prediction.
A new method estimates Bayes error for deep networks, suggesting they may have reached the limit.
problem Evaluating the performance of deep learning models and detecting overfitting.
method A simple and direct Bayes error estimator based on uncertainty of class assignments.
result Deep networks may have reached the Bayes error limit for benchmark datasets.
The concept of refinement from probability elicitation is considered for proper scoring rules. Taking directions from the axioms of probability, refinement is further clarified using a Hilbert space interpretation and reformulated into the underlying data distribution setting where connections to maximal marginal diver…
Develops methods for fair classification under linear disparity constraints.
problem Disparate impacts of machine learning algorithms on protected groups.
method Bayes-optimal fair classification methods via pre-, in-, and post-processing.
result Explicit forms of Bayes-optimal fair classifiers under linear disparity measures.
In machine learning, classification models need to be trained in order to predict class labels. When the training data contains personal information about individuals, collecting training data becomes difficult due to privacy concerns. Local differential privacy is a definition to measure the individual privacy when th…
Unified approach for fair classification with overlapping groups.
problem Ensuring fairness across multiple overlapping groups in prediction problems.
method Probabilistic population analysis leading to Bayes-optimal classifier, unifying existing methods.
result Outperforms baselines in fairness-performance tradeoff on real datasets.
Proposes a sparse Naïve Bayes classifier to improve performance and interpretability.
problem Naïve Bayes assumes feature independence, which is violated in real data.
method Integrates feature correlation and performance measures for feature selection.
result Competitive results in accuracy, sparsity, and running times for balanced datasets.