RayS attack improves hard-label adversarial attacks by reducing query complexity and identifying false robust models.
problem Challenges in hard-label adversarial attacks, especially in terms of effectiveness and efficiency.
method Reformulates continuous problem into discrete problem without gradient estimation and uses a fast check step to eliminate unnecessary searches.
result Significantly reduces the number of queries needed for hard-label attacks and identifies false robust models.
We study the problem of attacking a machine learning model in the hard-label black-box setting, where no model information is revealed except that the attacker can make queries to probe the corresponding hard-label decisions. This is a very challenging problem since the direct extension of state-of-the-art white-box at…
Efficient method for generating adversarial examples with limited query budget.
problem Developing black-box adversarial attacks with limited information.
method Bayesian Optimization in a structured low-dimensional subspace.
result Significantly higher attack success rate with fewer queries.
New PL method improves ASR accuracy without pseudo-labels.
problem Improving ASR accuracy with limited labeled data.
method End-to-end continuous pseudo-labeling with soft-labels.
result Soft-labels can lead to model collapse, but regularization can mitigate this.
Rotation invariant algorithms fail with hard labels sampled from sparse targets.
problem Rotation invariant algorithms fail to learn from hard labels sampled from sparse targets.
method Proving the excess risk of rotation invariant algorithms and proposing a simple non-rotation invariant algorithm.
result Rotation invariant algorithms incur an excess risk of $Ω\left(\frac{d-1}{n}
ight)$, while non-rotation invariant algorithms have an excess risk of $O\left(\frac{s\log d}{n}
ight).
We study the most practical problem setup for evaluating adversarial robustness of a machine learning system with limited access: the hard-label black-box attack setting for generating adversarial examples, where limited model queries are allowed and only the decision is provided to a queried data input. Several algori…
The goal of semi-supervised learning is to improve supervised classifiers by using additional unlabeled training examples. In this work we study a simple self-learning approach to semi-supervised learning applied to the least squares classifier. We show that a soft-label and a hard-label variant of self-learning can be…
Distillation works even with hard labels from overparameterized teacher, leading to better performance.
problem Improving model performance with hard labels from overparameterized teacher.
method Training a student model on a large held-out dataset labeled by a highly overparameterized teacher.
result Student model outperforms traditional approaches due to double descent phenomenon.
We study the problem of finding a universal (image-agnostic) perturbation to fool machine learning (ML) classifiers (e.g., neural nets, decision tress) in the hard-label black-box setting. Recent work in adversarial ML in the white-box setting (model parameters are known) has shown that many state-of-the-art image clas…
Learning using privileged information is an attractive problem setting that helps many learning scenarios in the real world. A state-of-the-art method of Gaussian process classification (GPC) with privileged information is GPC+, which incorporates privileged information into a noise term of the likelihood. A drawback o…
Adversarial example generation becomes a viable method for evaluating the robustness of a machine learning model. In this paper, we consider hard-label black-box attacks (a.k.a. decision-based attacks), which is a challenging setting that generates adversarial examples based on only a series of black-box hard-label que…
Retraining with predicted labels improves model accuracy in noisy settings.
problem Improving model accuracy with noisy or corrupted labels.
method Retraining with predicted hard labels in a linearly separable binary classification setting.
result Retraining with predicted labels can increase model accuracy, as proven theoretically.
Paper estimates optimal classification error with soft labels and calibration.
problem Estimating the optimal classification error with soft labels and calibration.
method Extends previous work on soft labels to estimate Bayes error, addressing bias and corrupted labels.
result The method provides a statistically consistent estimator of the Bayes error, even with imperfectly calibrated soft labels.
Proposes a new method for deep ensembles that improves accuracy and calibration.
problem Improving accuracy and calibration of deep ensembles.
method Estimates confusion matrices of ensemble members and weighs them according to their inferred performance.
result Empirically shows superiority of soft Dawid Skene over ensemble averaging.
Self-training with noisy student-teacher boosts keyword spotting accuracy.
problem Robust keyword spotting in challenging conditions.
method Aggressive data augmentation and self-training with noisy student-teacher approach.
result Significant accuracy improvement in difficult conditions, up to 60%.
Efficient WKNN-Shapley computation improves data valuation accuracy.
problem Efficient computation of Data Shapley for WKNN algorithm.
method Reframed WKNN-Shapley as a counting problem, introduced quadratic-time algorithm.
result Quadratic-time WKNN-Shapley computation, improving from O(NK). Proposes a robust VIB approach using soft labels and mutual info estimation.
problem Improving robustness of VIB to adversarial perturbations.
method Refines categorical class information with soft labels from a reference network, relaxes Gaussian posterior assumption.
result Significantly outperforms benchmarked models on MNIST and CIFAR-10.
IDS improves RLHF by smoothing reward data, enhancing model performance.
problem Reward model performance degrades and overoptimization hinders true objective.
method Iterative Data Smoothing (IDS) updates model and data labels during each epoch.
result IDS outperforms traditional methods in RLHF.
Spanning attack improves black-box attacks with unlabeled data.
problem Query inefficiency in black-box attacks due to high input space dimensionality.
method Proposes spanning attack by constraining adversarial perturbations in a low-dimensional subspace via an auxiliary unlabeled dataset.
result Significantly improves query efficiency of black-box attacks.
New methods interpret clustering outcomes without altering data structure.
problem Post-processing methods destroy data integrity and obscure interpretations.
method Algorithm-agnostic interpretation methods using permutation feature importance, individual conditional expectation, and partial dependence.
result Preserves original feature structure and explains clustering outcomes.
Scarcity of labeled data is a bottleneck for supervised learning models. A paradigm that has evolved for dealing with this problem is data programming. An existing data programming paradigm allows human supervision to be provided as a set of discrete labeling functions (LF) that output possibly noisy labels to input in…
The paper analyzes knowledge distillation in wide neural networks, providing theoretical insights and practical implications.
problem Lack of theoretical understanding of knowledge distillation in wide neural networks.
method Theoretical analysis of knowledge distillation in a linearized model of a wide neural network, introducing a metric of task training difficulty.
result For a perfect teacher, a high ratio of teacher's soft labels can be beneficial. For imperfect teacher, hard labels can correct wrong predictions.
Voting ensemble of robust models improves robustness.
problem Improving robustness of defensive models against adversarial attacks.
method Hard-label voting ensemble of pretrained robust models.
result Voting ensemble can boost robust error over individual models.
Dataset distillation is a method for reducing dataset sizes by learning a small number of synthetic samples containing all the information of a large dataset. This has several benefits like speeding up model training, reducing energy consumption, and reducing required storage space. Currently, each synthetic sample is …
The paper characterizes the efficiency of transferring knowledge from a teacher to a student classifier over finite domains.
problem Characterizing the statistical efficiency of knowledge transfer over finite domains.
method Three progressive levels of privileged information: hard labels, teacher probabilities, and soft labels. Novel empirical loss functions used to achieve the fundamental limits.
result Achieving the fundamental limits of knowledge transfer through specific levels of privileged information and novel loss functions.
The study examines when to trust confidence thresholding in pseudo-labelling regression.
problem Calibrated probabilities from classifiers used for pseudo-labelling need careful handling to avoid bias in downstream regression.
method Developed a diagnostic apparatus to predict and bound the bias induced by confidence thresholding, derived a closed-form expression for the attenuation bias.
result The bias can be predicted from the residual score variance V∗, motivating a structural separation between classifier features and downstream controls. Mixup~\cite{zhang2017mixup} is a recently proposed method for training deep neural networks where additional samples are generated during training by convexly combining random pairs of images and their associated labels. While simple to implement, it has been shown to be a surprisingly effective method of data augmenta…
Proposes IFCDA framework to improve cross-domain adaptation.
problem Negative transfer and difficulty in handling category-irrelevant losses in DA.
method Importance filtered mechanism to generate filtered soft labels, combined with graph-based label propagation.
result Significantly improves performance in both Closed-Set and Open-Set DA scenarios.