Study shows flipping a small subset of labels can severely damage machine learning models.
problem Adversarial attacks on distributed machine learning models.
method Formalized label flipping attacks, proposed a greedy algorithm, demonstrated with logistic regression models.
result A budget of only 0.1% of labels at each training step can reduce model accuracy by 6%, and some models can perform worse than random guessing when up to 25% of labels are flipped.
Efficiently poisons offline RLHF models by flipping preference labels.
problem Vulnerability of offline RLHF models to preference label flipping attacks.
method Developed two attack methods: BAL-A and BMP-A, solving a structured binary sparse approximation problem.
result Demonstrated that flipping one preference label induces a parameter-independent shift in the DPO gradient, enabling structured binary sparse approximation.
Many machine learning systems rely on data collected in the wild from untrusted sources, exposing the learning algorithms to data poisoning. Attackers can inject malicious data in the training dataset to subvert the learning process, compromising the performance of the algorithm producing errors in a targeted or an ind…
Deep Partition Aggregation defends against poisoning attacks with provable certificates.
problem Adversarial poisoning attacks corrupt classifier test-time behavior.
method Deep Partition Aggregation (DPA) is an ensemble method using hash partitions and base models trained on these partitions.
result DPA can certify >= 50% of test images against over 500 poison image insertions on MNIST, and nine insertions on CIFAR-10.
New method makes machine learning models robust to label flipping attacks.
problem Machine learning models are vulnerable to label flipping attacks.
method Randomized smoothing over arbitrary functions to build certifiably robust classifiers.
result Linear classifiers are robust to label flipping attacks with deterministic bounds.
Paper defends against malware detection attacks using clustering and deep learning.
problem Label flipping attacks on malware detection systems in IoT environments.
method Developed a Silhouette clustering-based attack mechanism and two CNN-type deep learning algorithms for defense.
result Proposed algorithms LSD and CSD improve malware detection accuracy by up to 19%.
Federated learning is protected against adversarial attacks with residual-based reweighting.
problem Adversarial attacks on federated learning's aggregation process.
method Residual-based reweighting combined with iteratively reweighted least squares.
result Our aggregation algorithm outperforms other methods in label-flipping and backdoor attacks.
Novel defense algorithm improves SVMs against data poisoning attacks.
problem Vulnerability of SVMs to targeted training data manipulations like poisoning attacks.
method Developed a weighted SVM using K-LID to de-emphasize suspicious data samples.
result Significant reduction in classification error rates (10% on average) with the proposed defense.
Adversarial examples are important for understanding the behavior of neural models, and can improve their robustness through adversarial training. Recent work in natural language processing generated adversarial examples by assuming white-box access to the attacked model, and optimizing the input directly against it (E…
T-BFA targets and misleads specific DNN inputs to a chosen output.
problem Targeted attack on DNN weight parameters to hijack function.
method Identifies critical weight bits, ranks them by class dependence, and flips them to mislead inputs.
result Successfully misclassifies images from 'Hen' to 'Goose' class with 100% success rate, maintaining 59.35% validation accuracy.
Unified framework for data poisoning attacks in graph-based semi-supervised learning.
problem Data poisoning attacks on graph-based semi-supervised learning.
method Unified formula for data poisoning attacks, specialized algorithms for regression and classification tasks.
result Data poisoning can be effective even with minimal perturbations.
Proposes SOVR loss to improve adversarial robustness by increasing logit margins.
problem Adversarial training's difficulty in robustness against sophisticated attacks.
method Introduces SOVR loss function that switches from cross-entropy to one-vs-the-rest loss for important samples.
result SOVR loss increases logit margins of important samples, improving robustness against Auto-Attack.
Paper proposes NeuroAttack to undermine SNNs security through bit-flips.
problem Security and reliability issues in SNNs.
method Cross-layer attack exploiting low-level reliability issues via adversarial input noise.
result Serious integrity threat to SNNs and DNNs.
Paper proposes a black-box adversarial attack method for graph embedding models.
problem Robustness of graph embedding models against adversarial attacks.
method GF-Attack constructs a generalized adversarial attacker by the graph filter and feature matrix, performing the attack on the graph filter in a black-box fashion.
result GF-Attack can consistently make strong attacks on different graph embedding models even with small perturbations.
Machine learning (ML) over distributed multi-party data is required for a variety of domains. Existing approaches, such as federated learning, collect the outputs computed by a group of devices at a central aggregator and run iterative algorithms to train a globally shared model. Unfortunately, such approaches are susc…
Semantic paraphrases can fool financial sentiment classifiers due to geometric shifts in model representations.
problem Semantic paraphrase attacks on financial sentiment classifiers
method Developing a continuous local model of semantic paraphrase perturbations
result The worst-case local displacement of the target representation is governed by the largest generalised eigenvalue of a matrix pencil constructed from the Jacobians of the two embedding maps.
New attacks can infer model training membership using only label predictions, not confidence.
problem Inferring whether a data point was used to train a machine learning model.
method Evaluate model's predicted labels under perturbations to infer membership.
result Label-only attacks perform as well as confidence-based attacks and break defenses that rely on confidence masking.
A double pants decomposition of a 2-dimensional surface is a collection of two pants decomposition of this surface introduced in arXiv:1005.0073v2. There are two natural operations acting on double pants decompositions: flips and handle twists. It is shown in arXiv:1005.0073v2 that the groupoid generated by flips and h…
RayS attack improves hard-label adversarial attacks by reducing query complexity and identifying false robust models.
problem Challenges in hard-label adversarial attacks, especially in terms of effectiveness and efficiency.
method Reformulates continuous problem into discrete problem without gradient estimation and uses a fast check step to eliminate unnecessary searches.
result Significantly reduces the number of queries needed for hard-label attacks and identifies false robust models.
Bullseye Polytope improves clean-label poisoning attacks in transfer learning.
problem Poisoning neural networks with correctly labeled data.
method Creates poison images with centers close to target images in feature space.
result Improves attack success rate by 26.75% in end-to-end transfer learning.
Develops a framework for assessing adversarial robustness in deep learning models.
problem Adversarial examples can falsely flip deep learning models' predictions with imperceptible perturbations.
method Defines adversarial robustness as a locally adaptive measure and develops a data-augmentation framework.
result Proves that the adaptive data-augmentation maintains consistency of 1-nearest neighbor classification under deterministic labels.
New attacks reveal training label proportions in federated learning.
problem Adversaries can infer training label proportions from limited data.
method Proposed three types of attacks: Class Sniffing, Quantity Inference, Whole Determination.
result Our attacks work well on various tasks and datasets.
The paper solves pentagon equations using triangulations and edge transformations.
problem Solving pentagon equations with triangulations and edge transformations.
method General data and transformation rule method applied to triangulations.
result Recovery of initial data after transformations.
It has been a long-standing problem to efficiently learn a halfspace using as few labels as possible in the presence of noise. In this work, we propose an efficient Perceptron-based algorithm for actively learning homogeneous halfspaces under the uniform distribution over the unit sphere. Under the bounded noise condit…
Efficiently creates label-consistent backdoor attacks without obvious mislabeling.
problem Vulnerability of deep neural networks to backdoor attacks that can be triggered by a backdoor trigger.
method Uses adversarial perturbations and generative models to inject inputs that are consistent with their labels.
result Demonstrates the feasibility of creating undetectable backdoor attacks by maintaining label-consistency.
New attacks reveal membership in label-only ML models.
problem Vulnerability of ML models to membership inference attacks.
method Developed decision-based membership inference attacks.
result Label-only exposures are vulnerable to membership leakage.
Efficient method for generating adversarial examples with limited query budget.
problem Developing black-box adversarial attacks with limited information.
method Bayesian Optimization in a structured low-dimensional subspace.
result Significantly higher attack success rate with fewer queries.
Clean-label poisoning attacks inject innocuous looking (and "correctly" labeled) poison images into training data, causing a model to misclassify a targeted image after being trained on this data. We consider transferable poisoning attacks that succeed without access to the victim network's outputs, architecture, or (i…
Proposes a new model for noisy labels considering multiple labelers and adversarial attacks.
problem Real-world noisy label models with multiple labelers and adversarial attacks.
method Labeler-dependent noise model with adversarial attack vectors.
result State-of-the-art approaches for learning from noisy labels are defeated by adversarial label attacks.
New insights on robust learning under strong noise models.
problem Challenging label-noise models in robust learning.
method Extending statistical query framework to more general noise models and using evolutionary algorithms.
result First polynomial time algorithm for learning linear threshold functions with arbitrarily small excess error in presence of Tsybakov noise.
Headless attacks bypass classification heads to fool transfer learning models.
problem Adversarial attacks against transfer learning models without access to the classification head.
method Label-blind adversarial attacks that do not require class-label information.
result Transfer attack lowers ResNet18 accuracy on CIFAR10 by over 40%.
Data poisoning is an attack on machine learning models wherein the attacker adds examples to the training set to manipulate the behavior of the model at test time. This paper explores poisoning attacks on neural nets. The proposed attacks use "clean-labels"; they don't require the attacker to have any control over the …
Study online learner attacks by manipulating labels, revealing critical thresholds.
problem Adversarial attacks on online learners, especially label manipulation.
method Control-theoretical perspective, theoretical analysis, empirical study.
result Discontinuous transition in learner's accuracy occurs at a critical threshold.
We study the problem of attacking a machine learning model in the hard-label black-box setting, where no model information is revealed except that the attacker can make queries to probe the corresponding hard-label decisions. This is a very challenging problem since the direct extension of state-of-the-art white-box at…
Domain knowledge helps detect adversarial examples in multi-label classification.
problem Detecting adversarial examples in multi-label classification.
method Convert domain knowledge into constraints and inject them into a semi-supervised learning problem.
result Domain-knowledge constraints help detect adversarial examples effectively.
Deep Neural Networks (DNNs) are vulnerable to adversarial attacks, especially white-box targeted attacks. One scheme of learning attacks is to design a proper adversarial objective function that leads to the imperceptible perturbation for any test image (e.g., the Carlini-Wagner (C&W) method). Most methods address targ…
Spanning attack improves black-box attacks with unlabeled data.
problem Query inefficiency in black-box attacks due to high input space dimensionality.
method Proposes spanning attack by constraining adversarial perturbations in a low-dimensional subspace via an auxiliary unlabeled dataset.
result Significantly improves query efficiency of black-box attacks.
Paper explores how poisoning data can increase privacy risks in machine learning models.
problem Increasing privacy risks of benign training samples through data poisoning attacks.
method Proposes generic and optimization-based attacks to amplify membership exposure.
result Demonstrates substantial increase in membership inference precision with minimal model performance degradation.
This paper investigates a class of attacks targeting the confidentiality aspect of security in Deep Reinforcement Learning (DRL) policies. Recent research have established the vulnerability of supervised machine learning models (e.g., classifiers) to model extraction attacks. Such attacks leverage the loosely-restricte…
Backdoor attacks are possible in feature-partitioned collaborative learning, even without labels.
problem Backdoor attacks in feature-partitioned collaborative learning.
method Demonstrated the feasibility of backdoor attacks and introduced defense techniques.
result Backdoor attacks can be blocked without significantly affecting main task accuracy.
Study reveals how high-dimensional models are vulnerable to consistent adversarial attacks.
problem Understanding the vulnerability of high-dimensional linear classifiers to adversarial attacks.
method Introducing a new error metric to quantify model vulnerability, and rigorously characterizing these metrics in asymptotic settings.
result As models become more overparameterized, their vulnerability to label-preserving perturbations increases.
Machine learning has been applied to a broad range of applications and some of them are available online as application programming interfaces (APIs) with either free (trial) or paid subscriptions. In this paper, we study adversarial machine learning in the form of back-box attacks on online classifier APIs. We start w…
Study flip graphs for surfaces of infinite type, finding uncountably many connected components.
problem Understanding relationships between triangulations of infinite type surfaces via flips.
method Associate triangulations to flip graphs and study sequences of simultaneous flips.
result Flip graphs for infinite type surfaces have uncountably many connected components.
Deep neural networks (DNNs) have transformed several artificial intelligence research areas including computer vision, speech recognition, and natural language processing. However, recent studies demonstrated that DNNs are vulnerable to adversarial manipulations at testing time. Specifically, suppose we have a testing …
Instance- and Label-dependent label Noise (ILN) widely exists in real-world datasets but has been rarely studied. In this paper, we focus on Bounded Instance- and Label-dependent label Noise (BILN), a particular case of ILN where the label noise rates -- the probabilities that the true labels of examples flip into the …
Tricks adversarial attacks to target specific classes, improving classifier accuracy.
problem Recent adversarial defense approaches have failed to protect classifiers from untargeted attacks.
method Target Training defense tricks untargeted attacks into targeted attacks on designated classes, then derives the real class.
result 86.2% accuracy for CW-L2 (confidence=0) in CIFAR10, outperforming unsecured classifiers.
Sign-OPT uses fewer queries to generate adversarial examples.
problem Efficiently generating adversarial examples with limited model queries.
method Adopting zeroth order optimization with sign of gradient estimation.
result Sign-OPT requires 5X to 10X fewer queries than state-of-the-art approaches.
New framework improves adversarial robustness in one-stage L2D.
problem Adversarial robustness in one-stage Learning-to-Defer (L2D).
method Formalizes attacks, proposes cost-sensitive adversarial surrogate losses, establishes theoretical guarantees.
result Improves robustness against untargeted and targeted attacks while preserving clean performance.