Study examines how classifier performance is affected by training data quality.
problem How classifier performance is affected by training data quality.
method Extensive numerical experiments with four classifiers (Bayes, neural nets, partition models, random forests) on metagenomic assembly data.
result Classifier performance degrades as training data quality degrades, leading to breakdown-like behavior.
Advances adversarial training of smoothed classifiers for robust deep learning.
problem Building neural network classifiers robust to adversarial attacks.
method Adversarial training of randomized smoothed classifiers with an adapted attack.
result Significantly outperforms existing provably robust classifiers on ImageNet and CIFAR-10.
Study proves existence of robust classifiers in multiclass adversarial training.
problem Proves existence of robust classifiers in multiclass adversarial training.
method Three models of adversarial training in multiclass classification, proving existence of Borel measurable robust classifiers.
result Proves existence of Borel measurable robust classifiers in each model.
Generative GAN improves classifier performance with limited training data.
problem Limited training data for black-box API attacks.
method Generative adversarial network (GAN) to generate synthetic training data.
result Improves classifier performance with synthetic data.
Distillation speeds up classifier training and provides insights into its success.
problem Empirical success of knowledge distillation without theoretical explanation.
method Study of linear and deep linear classifiers, proving a generalization bound.
result Three key factors for distillation success: data geometry, optimization bias, strong monotonicity.
Develops fair classifiers robust to training distribution perturbations.
problem Ensuring fairness in classifiers robust to training data perturbations.
method Formulates a min-max objective function to minimize distributionally robust training loss while maintaining fairness for perturbed distributions. Uses an iterative online learning algorithm to find a fair and robust classifier.
result Our classifier maintains fairness and accuracy for a wide range of perturbations compared to state-of-the-art fair classifiers.
We compare and contrast two approaches to validating a trained classifier while using all in-sample data for training. One is simultaneous validation over an organized set of hypotheses (SVOOSH), the well-known method that began with VC theory. The other is withhold and gap (WAG). WAG withholds a validation set, trains…
A meta-learning framework for dynamic ensemble selection.
problem Improving dynamic ensemble selection for classification tasks.
method Proposes a three-step framework using meta-learning to select competent classifiers.
result Problem-dependent scenario yields the best performance.
Framework to mitigate adversarial attacks by allowing classifiers to abstain.
problem Vulnerability of classifiers to adversarial examples.
method Adversarial training with a rejection option.
result ATRO framework improves classifier reliability against adversarial attacks.
Detects backdoors in trained classifiers without access to training data.
problem Post-training detection of backdoor attacks in DNN image classifiers.
method Purely unsupervised anomaly detection (AD) approach.
result Detects whether a classifier has been backdoor-attacked and infers source and target classes.
Gradient descent-based adversarial training converges to robust classifiers on linearly separable data.
problem Understanding the inductive bias of adversarial training for robustness.
method Gradient descent on binary classification tasks with linearly separable data, focusing on inductive bias and convergence rates.
result Gradient descent-based adversarial training converges to the maximum margin classifier at a faster rate than clean data training.
META-DES.H selects competent classifiers using meta-learning and dynamic weighting.
problem Selecting competent classifiers in dynamic ensemble selection.
method META-DES framework using meta-learning and dynamic weighting.
result Improvements in recognition accuracy on 30 datasets.
This work generates training-time adversarial data using auto-encoders to manipulate classifiers.
problem Manipulating the behavior of trained classifiers during test time with bounded perturbation.
method An auto-encoder-like network generates perturbations, learning to update weights to produce harmful noise.
result The method can manipulate classifiers effectively, showing good transferability.
Paper studies deep learning attacks on online APIs with limited data.
problem Adversarial machine learning threats on online APIs with strict rate limitations.
method Develops an active learning approach to build adversarial classifiers with limited training data.
result Active learning can build adversarial classifiers with small statistical difference from target classifiers using limited data.
A new method combines classifiers using possibility distributions and adaptive t-norms.
problem Aggregating predictions from multiple classifiers trained on overlapping datasets.
method Proposes a new approach to aggregate classifier predictions using possibility theory and adaptive t-norms.
result Proves the proposed approach possesses desirable robustness properties.
Machine Learning (ML) algorithms are used to train computers to perform a variety of complex tasks and improve with experience. Computers learn how to recognize patterns, make unintended decisions, or react to a dynamic environment. Certain trained machines may be more effective than others because they are based on mo…
New method reconstructs significant parts of training data from neural networks.
problem Understanding and reconstructing training data from neural networks.
method Proposes a novel reconstruction scheme based on recent theoretical results about neural network training.
result Shows that a significant fraction of training data can be reconstructed from neural network parameters.
We study the effect of imperfect training data labels on the performance of classification methods. In a general setting, where the probability that an observation in the training dataset is mislabelled may depend on both the feature vector and the true label, we bound the excess risk of an arbitrary classifier trained…
The problem of detecting whether a test sample is from in-distribution (i.e., training distribution by a classifier) or out-of-distribution sufficiently different from it arises in many real-world machine learning applications. However, the state-of-art deep neural networks are known to be highly overconfident in their…
A new classifier method detects out-of-distribution samples by minimizing KL divergence.
problem Detecting out-of-distribution samples in neural networks.
method Training a confident-classifier by minimizing KL divergence and maximizing entropy, or adding a reject class.
result The confident-classifier still yields high confidence for OOD samples far from the in-distribution.
Tricks adversarial attacks to target specific classes, improving classifier accuracy.
problem Recent adversarial defense approaches have failed to protect classifiers from untargeted attacks.
method Target Training defense tricks untargeted attacks into targeted attacks on designated classes, then derives the real class.
result 86.2% accuracy for CW-L2 (confidence=0) in CIFAR10, outperforming unsecured classifiers.
GanDef uses GANs to defend against adversarial examples in neural networks.
problem Defending against adversarial examples in neural networks.
method GAN-based adversarial training defense using a competition game to regulate feature selection.
result GanDef trains a classifier to defend against adversarial examples with high accuracy.
Contingency Training improves classifier accuracy and robustness against irrelevant variables.
problem Feature selection leaves irrelevant variables in high-dimensional datasets, reducing classifier performance.
method Subsampling and creating constraints to find proper feature importance weights.
result Contingency Training outperforms traditional training methods, especially with irrelevant variables.
Localized adversarial training improves image classifiers' robustness.
problem State-of-the-art image classifiers fail on carefully manipulated adversarial images.
method Developed a localized adversarial attack and used it to train a robust classifier.
result Localized adversarial training increases robustness against adversarial inputs.
Paper proposes a classifier to improve medical image classification with limited data.
problem Limited training data leads to overfitting in medical image classification.
method Uses reinforcement learning to update classifier parameters with generalization feedback from a subset of training data.
result Improves classification performance and demonstrates generalized learning.
ICE improves classification performance by leveraging internal patterns among instances.
problem Inconsistent results for different MCS algorithms on specific problems.
method ICE groups training data into overlapping clusters, builds classifiers for each cluster, and predicts class labels by averaging predictions from top-performing models.
result ICE provides a stable improvement on a significant proportion of datasets over existing MCS methods.
Develops method to train classifiers on incomplete feature datasets.
problem Training classifiers on datasets with missing feature subsets.
method Simultaneous training of neural networks and sparse coding.
result Classifier trained on incomplete features correctly separates original data.
Improves classification accuracy with noisy labels using generative classifiers.
problem Handling noisy labels in large-scale datasets.
method Robust Generative Classifier (RoG) on top of pre-trained DNNs.
result Significantly improves classification accuracy with no re-training of the deep model.
StylEx trains a GAN to explain classifier decisions in StyleSpace.
problem Creating meaningful image-specific explanations for classifier decisions.
method Training a StyleGAN to learn a classifier-specific StyleSpace, incorporating the classifier model.
result StylEx finds attributes that align with semantic ones and generates human-interpretable explanations.
A new metric estimates classifier accuracy using only training data.
problem Assessing classifier accuracy without cross-validation.
method Bayesian Area Under the ROC Curve (CBAUC) metric for linear classifiers.
result The CBAUC is faster and more accurate than conventional AUC estimators.
New method uses off-the-shelf classifiers to improve diffusion generation without extra training.
problem Improving sample quality and controllability in conditional diffusion generation.
method Leveraging off-the-shelf classifiers in a training-free fashion with calibration and pre-conditioning techniques.
result Significant performance improvements (up to 20%) over existing guidance schemes.
New method improves robustness of smoothed classifiers.
problem Improving accuracy and robustness of smoothed classifiers.
method Regularized risk with adaptive regularization.
result Tighter robustness bounds with high probability.
We present a novel algorithm (Principal Sensitivity Analysis; PSA) to analyze the knowledge of the classifier obtained from supervised machine learning techniques. In particular, we define principal sensitivity map (PSM) as the direction on the input space to which the trained classifier is most sensitive, and use anal…
Adversarial training helps classifiers resist universal perturbations.
problem Vulnerability of classifiers to universal perturbations.
method Adversarial training with shared adversarial examples.
result Adversarial training reduces sensitivity to universal perturbations.
This work addresses fairness in ML models by training and evaluating attribute classifiers under uncertain and incomplete data.
problem Challenges in fairness metrics due to uncertain and incomplete data.
method Developed a theoretical and empirical analysis to understand and improve bias estimation in the data-scarce regime.
result The test accuracy of the attribute classifier is not always correlated with its effectiveness in bias estimation.
Expands pre-trained deep networks to classify new classes with minimal data.
problem Learning new classes with limited data.
method Hard distillation with a compact generative model.
result Low-shot network expansion is possible with minimal memory and data.
Bayes-optimal classifiers are robust to adversarial attacks, unlike CNNs trained on the same data.
problem The vulnerability of modern CNN classifiers to adversarial examples.
method Constructing realistic image datasets and deriving analytic conditions for Bayes-optimal classifiers.
result Bayes-optimal classifiers are robust to adversarial attacks, unlike CNNs trained on the same data.
New method controls classifier guidance in diffusion models.
problem Improving classifier guidance in diffusion models.
method Cross-entropy control of classifier gradients.
result Effective guidance vectors with mean squared error O(dε). We consider the task of training classifiers without labels. We propose a weakly supervised method---adversarial label learning---that trains classifiers to perform well against an adversary that chooses labels for training data. The weak supervision constrains what labels the adversary can choose. The method therefore…
We train a network to generate mappings between training sets and classification policies (a 'classifier generator') by conditioning on the entire training set via an attentional mechanism. The network is directly optimized for test set performance on an training set of related tasks, which is then transferred to unsee…
Causal influence measures for machine learnt classifiers shed light on the reasons behind classification, and aid in identifying influential input features and revealing their biases. However, such analyses involve evaluating the classifier using datapoints that may be atypical of its training distribution. Standard me…
Stochastic defense improves natural classifiers against adversarial attacks.
problem Vulnerability of deep networks to adversarial attacks.
method Long-run MCMC sampling with Energy-Based Model for adversarial purification.
result Balancing memoryless and metastable behavior leads to effective purification and robust classification.
A new method aggregates generative classifiers to resist adversarial attacks.
problem Adversarial attacks on deep neural networks.
method Rank-aggregating ensemble of generative classifiers trained on intermediate layer responses.
result The ensemble of generative classifiers shows robustness to adversarial attacks.
Disagreement-based approaches generate multiple classifiers and exploit the disagreement among them with unlabeled data to improve learning performance. Co-training is a representative paradigm of them, which trains two classifiers separately on two sufficient and redundant views; while for the applications where there…
Paper tackles training binary classifiers from unlabeled data with minimal supervision.
problem Training arbitrary binary classifiers from only unlabeled data is impossible without supervision.
method Proposes an ERM-based learning method from two sets of unlabeled data with different class priors.
result The proposed method is consistent and outperforms state-of-the-art methods.
Self-training algorithm improves classifier performance with labeled and unlabeled data.
problem Improving classifier performance with limited labeled data.
method Iterative learning of halfspaces, exploration and pruning phases.
result Misclassification error is bounded and never degrades compared to initial labeled set.
Adversarial training can lead to overfitting without compromising robustness.
problem Explaining benign overfitting in adversarially robust linear classification.
method Theoretical analysis and numerical experiments on adversarial training.
result Adversarially trained linear classifiers can achieve near-optimal risks despite overfitting noisy data.
New insights into how linear classifiers and leaky ReLU networks can overfit without harming generalization.
problem Understanding conditions for benign overfitting in linear classifiers and leaky ReLU networks.
method Utilizing Karush--Kuhn--Tucker (KKT) conditions for margin maximization.
result Satisfaction of KKT conditions leads to benign overfitting in linear classifiers and leaky ReLU networks.