Random Hyperboxes is a simple yet effective ensemble classifier.
problem Improving classification accuracy using ensemble methods.
method Random subsets of sample and feature spaces are used to train individual hyperbox-based classifiers, which are then combined into an ensemble.
result The proposed classifier outperforms other fuzzy min-max neural networks and ensemble methods on 20 datasets.
Random projections improve classifier generalization without needing to choose the best threshold.
problem Improving classifier generalization without choosing the best threshold.
method Thresholding a random one-dimensional feature after random projection of data.
result Generalization gap is significantly smaller than linear classifiers.
Several recent works have shown that state-of-the-art classifiers are vulnerable to worst-case (i.e., adversarial) perturbations of the datapoints. On the other hand, it has been empirically observed that these same classifiers are relatively robust to random noise. In this paper, we propose to study a \textit{semi-ran…
The paper verifies the robustness of classifier ensembles against randomized attacks.
problem Ensuring classifier ensembles are robust against arbitrary randomized attacks.
method Formal verification procedure using SMT and MILP encodings to assess robustness.
result Proves the NP-hardness of the robustness-checking problem and provides upper bounds on attack sets.
Rule-based classifiers quantify uncertainty using Bernoulli random variables.
problem Quantifying the uncertainty of precision estimates for rule-based text classifiers.
method Treat partitions of sub-strings as Bernoulli random variables, compare means using statistical tests, and combine classifiers using Dempster-Shafer theory.
result The approach can be used to combine binary classifiers into a multi-label classifier.
RS-Del provides robustness for sequence classifiers against edit distance attacks.
problem Certifying robustness of discrete sequence classifiers against edit distance attacks.
method Randomized deletion (RS-Del) for discrete sequence classifiers, focusing on edit distance-bounded adversaries.
result Achieved a certified accuracy of 91% at an edit distance radius of 128 bytes on malware detection.
Randomized classifiers outperform deterministic ones in robustness against adversarial attacks.
problem Ensuring optimal robustness against all adversarial attacks.
method Game-theoretic approach, focusing on the non-existence of Nash equilibrium in deterministic classifiers and demonstrating the superiority of randomized classifiers.
result Randomized classifiers can outperform deterministic ones in robustness against adversarial attacks.
Consider a binary decision making process where a single machine learning classifier replaces a multitude of humans. We raise questions about the resulting loss of diversity in the decision making process. We study the potential benefits of using random classifier ensembles instead of a single classifier in the context…
Randomized classifiers improve strategic classification efficiency.
problem Designing optimal classifiers in strategic classification games.
method Investigation of randomized classifiers and their efficiency in strategic classification.
result Randomized classifiers are necessary for maximizing classification efficiency.
Study improves image classifier robustness to random p-norm corruptions.
problem Improving robustness of image classifiers to real-world imperceptible corruptions.
method Training and testing with random p-norm corruptions, evaluating robustness against different p-norms.
result Training with a combination of p-norm corruptions significantly improves robustness.
Unhinged loss minimization fails to improve classifier accuracy for simple data.
problem Accuracy of classifiers minimizing the unhinged loss.
method Minimizing the unhinged loss function.
result Minimizing the unhinged loss yields classifiers with accuracy no better than random guessing for simple data.
We propose a new randomized ensemble technique with a provable security guarantee against black-box transfer attacks. Our proof constructs a new security problem for random binary classifiers which is easier to empirically verify and a reduction from the security of this new model to the security of the ensemble classi…
Random forests classify Pokemon names based on evolutionary status.
problem Classifying Pokemon names into pre-evolution and post-evolution categories.
method Trained random forests using sound symbolism from Pokemon names, conducted experiments with human participants.
result Random forests outperform human participants in classifying Pokemon names.
Boosting improves accuracy by combining weak learners into a voting classifier.
problem Boosting's theoretical performance is sub-optimal, especially for voting classifiers.
method Proposes a randomized boosting algorithm that outputs voting classifiers with a single logarithmic dependency on sample size.
result Randomized boosting achieves a generalization error with a single logarithmic dependency on the sample size.
There is a large literature explaining why AdaBoost is a successful classifier. The literature on AdaBoost focuses on classifier margins and boosting's interpretation as the optimization of an exponential likelihood function. These existing explanations, however, have been pointed out to be incomplete. A random forest …
Enhances robustness of deep neural networks with randomized smoothing.
problem Improving robustness of deep neural networks against noisy inputs and adversarial attacks.
method Introduces a variance-margin trade-off approach to increase certified robust radius using pre-trained models.
result Significant improvement in certified accuracy compared to state-of-the-art methods.
A bias classifier is introduced to resist adversarial attacks.
problem Resisting adversarial attacks on deep neural networks (DNNs).
method Introducing the bias part of a DNN with Relu as the activation function as a classifier, and adding a random first-degree part to make it information-theoretically safe.
result The bias classifier is more robust than DNNs of similar size against adversarial attacks.
A new method detects changes in multivariate data using random forests.
problem Detecting changes in multivariate data.
method A computationally feasible search method using random forests and class probability predictions.
result Consistently locates change points in simulations.
We study the performance of data-driven, a priori and random approaches to label space partitioning for multi-label classification with a Gaussian Naive Bayes classifier. Experiments were performed on 12 benchmark data sets and evaluated on 5 established measures of classification quality: micro and macro averaged F1 s…
Both neural networks and decision trees are popular machine learning methods and are widely used to solve problems from diverse domains. These two classifiers are commonly used base classifiers in an ensemble framework. In this paper, we first present a new variant of oblique decision tree based on a linear classifier,…
We study the robustness of classifiers to various kinds of random noise models. In particular, we consider noise drawn uniformly from the ℓ_p ball for p∈[1,∞] and Gaussian noise with an arbitrary covariance matrix. We characterize this robustness to random noise in terms of the distance to the decisio…
Work on making classifiers robust against adversarial attacks for top-k predictions.
problem Vulnerability of classifiers to adversarial perturbations, especially for top-k predictions.
method Randomized smoothing to turn any classifier into a robust one, using Gaussian noise.
result Derives a tight robustness in ℓ2 norm for top-k predictions, achieving 62.8% certified top-5 accuracy on ImageNet.
Unified approach adjusts classifiers to meet system-level constraints.
problem Multi-class classification under system-level constraints.
method Post-processing approach using linearly constrained stochastic program and entropic regularization.
result Finite-sample guarantees for risk and constraint satisfaction.
We show how to turn any classifier that classifies well under Gaussian noise into a new classifier that is certifiably robust to adversarial perturbations under the ℓ2 norm. This "randomized smoothing" technique has been proposed recently in the literature, but existing guarantees are loose. We prove a tight robu…
In this article, a large data set containing every course taken by every undergraduate student in a major university in Canada over 10 years is analysed. Modern machine learning algorithms can use large data sets to build useful tools for the data provider, in this case, the university. In this article, two classifiers…
A new adaptive kNN classifier outperforms Random Forests.
problem Improving classification accuracy using nearest neighbors.
method Finding discriminant subspaces for efficient nearest neighbor classification, leveraging bagging for diversity.
result The proposed method outperforms Random Forests and other nearest neighbors ensembles.
Study examines how classifier performance is affected by training data quality.
problem How classifier performance is affected by training data quality.
method Extensive numerical experiments with four classifiers (Bayes, neural nets, partition models, random forests) on metagenomic assembly data.
result Classifier performance degrades as training data quality degrades, leading to breakdown-like behavior.
New method improves certified robustness for classifier confidence.
problem Certifying confidence in classifier predictions.
method Randomized smoothing with modified Neyman-Pearson lemma.
result Certified radii for prediction confidence improved.
Enhances functional classifier performance with new tree-based methods and unbiased feature importance assessment.
problem Challenges of high-dimensional functional data and biased feature importance assessment.
method Augmented functional classification trees and random forests with ad-hoc conditional permutations for unbiased feature importance.
result Significant enhancement in predictive power of functional classifiers through new feature importance assessment.
New classifier robust to adversarial perturbations from high-accuracy models.
problem Adversarial attacks on high-accuracy machine learning models.
method Randomized smoothing framework with random partitions and doubling dimension.
result Constructs a robust classifier with high accuracy and bounded adversarial error.
This study improves scalability of randomized smoothing for certifying classifier robustness.
problem Certifying machine learning classifiers against adversarial attacks is challenging and scalable solutions are needed.
method The study reviews and explores randomized smoothing and its derivatives, focusing on scalability.
result The study provides theoretical guarantees and discusses scalability challenges of randomized smoothing.
RaSE ensemble framework improves sparse classification accuracy.
problem Sparse classification challenges in high-dimensional data.
method Random Subspace Ensemble (RaSE) framework with subspace selection via RIC.
result RaSE achieves low misclassification rates and accurate feature ranking.
In this paper we propose strategies for estimating performance of a classifier when labels cannot be obtained for the whole test set. The number of test instances which can be labeled is very small compared to the whole test data size. The goal then is to obtain a precise estimate of classifier performance using as lit…
From a fresh data science perspective, this thesis discusses the prediction of coronary artery disease based on genetic variations at the DNA base pair level, called Single-Nucleotide Polymorphisms (SNPs), collected from the Ontario Heart Genomics Study (OHGS). First, the thesis explains two commonly used supervised le…
A new method for efficient BNC parameter estimation outperforms HDP smoothing.
problem Efficiently estimating parameters for Bayesian network classifiers to match or exceed random forest performance.
method Uses log-linear regression to approximate hierarchical Dirichlet process (HDP) smoothing, making the approach simpler and faster.
result Our method outperforms HDP smoothing while being orders of magnitude faster and competitive with random forests.
Test assesses if a linear classifier is random or significant.
problem Determining if a linear classifier captures meaningful differences between classes.
method Proposes a homogeneity test related to linear separability, establishes upper bounds for p-values.
result Upper bounds for p-values are highly accurate for normally distributed samples.
Adversarial perturbations dramatically decrease the accuracy of state-of-the-art image classifiers. In this paper, we propose and analyze a simple and computationally efficient defense strategy: inject random Gaussian noise, discretize each pixel, and then feed the result into any pre-trained classifier. Theoretically,…
We employ random geometric digraphs to construct semi-parametric classifiers. These data-random digraphs are from parametrized random digraph families called proximity catch digraphs (PCDs). A related geometric digraph family, class cover catch digraph (CCCD), has been used to solve the class cover problem by using its…
Study optimizes classifiers for credit card mail campaigns and default prediction.
problem Optimizing classifiers for credit card mail campaigns and default prediction.
method Three distinct models: response, risk, and response-risk. Optimized various performance metrics.
result Random Forest classifier achieves highest accuracy (83.2%) in multi-class response-risk model.
Recent works have shown the effectiveness of randomized smoothing as a scalable technique for building neural network-based classifiers that are provably robust to ℓ2-norm adversarial perturbations. In this paper, we employ adversarial training to improve the performance of randomized smoothing. We design an adap…
New method certifies deep graph classifiers with tighter risk bounds.
problem Certifying the reliability of deep graph classifiers.
method Linearized deep assignment flows with random initial conditions, using PAC-Bayes risk certification.
result Computes tighter out-of-sample risk certificates efficiently.
Better calibration for small datasets improves probability estimates.
problem Limited data hinders traditional calibration methods.
method Generating additional calibration data improves classifier performance.
result The proposed approach enhances calibration for small datasets.
When randomized ensembles such as bagging or random forests are used for binary classification, the prediction error of the ensemble tends to decrease and stabilize as the number of classifiers increases. However, the precise relationship between prediction error and ensemble size is unknown in practice. In the standar…
New method makes machine learning models robust to label flipping attacks.
problem Machine learning models are vulnerable to label flipping attacks.
method Randomized smoothing over arbitrary functions to build certifiably robust classifiers.
result Linear classifiers are robust to label flipping attacks with deterministic bounds.
SWEEN improves certified robustness via weighted ensembling of smoothed classifiers.
problem Finding optimal base classifiers for randomized smoothing.
method Smoothed WEighted ENsembling (SWEEN) scheme.
result SWEEN achieves optimal certified robustness under mild assumptions.
Denoised smoothing defends pretrained classifiers against adversarial attacks.
problem Adversarial attacks on pretrained classifiers.
method Prepending a denoiser to any off-the-shelf classifier using randomized smoothing.
result Guaranteed ℓp-robustness to adversarial examples without modifying the pretrained classifier. In this paper, we address a problem of machine learning system vulnerability to adversarial attacks. We propose and investigate a Key based Diversified Aggregation (KDA) mechanism as a defense strategy. The KDA assumes that the attacker (i) knows the architecture of classifier and the used defense strategy, (ii) has an…
LC-CRFs are equivalent to HMMs, and MPM/MAP classifiers can be reformulated as CRFs.
problem Comparing and reformulating HMMs and CRFs.
method Demonstrating equivalence and reformulation of classifiers.
result LC-CRFs are equivalent to HMMs, and MPM/MAP classifiers can be reformulated as CRFs.