Unified benchmarks assess data poisoning and backdoor attacks.
problem Unclear danger and effectiveness of data poisoning methods.
method Developed standardized benchmarks for data poisoning and backdoor attacks.
result Existing methods may not generalize to realistic settings.
MetaPoison poisons neural networks by learning to craft imperceptible changes to training data.
problem Data poisoning attacks on neural networks.
method MetaPoison uses meta-learning to approximate bilevel optimization for crafting poisons.
result MetaPoison outperforms previous methods and works in various scenarios.
This paper evaluates targeted data poisoning attacks by focusing on the hardest samples, improving evaluation and defense strategies.
problem The effectiveness of targeted data poisoning attacks is often overestimated due to average evaluation methods.
method The paper introduces metrics to identify the hardest and easiest to poison samples based on clean model information.
result The proposed metrics reliably stratify samples by poisoning vulnerability, enabling rigorous worst-case evaluation and proactive defense.
Temporal threat model defends against data poisoning with timestamps.
problem Adversaries can poison more samples than expected, rendering existing defenses ineffective.
method Leverage timestamps to define earliness and duration metrics for temporal robustness.
result Temporal aggregation provides provable temporal robustness against data poisoning.
Paper proves mathematically that poisoning datasets can be detected.
problem Detecting data poisoning attacks in datasets.
method Mathematical definition and Conformal Separability Test.
result Dataset poisoning can be effectively detected.
Linear models can be poisoned by shifting a fraction of one class's data, revealing scaling laws and weight alignment.
problem Understanding and quantifying data poisoning in linear models.
method Analysis of ridge least squares with an unpenalized intercept, using resolvent techniques and random matrix theory.
result Closed-form limits for the poisoned score, revealing scaling laws and weight alignment with the poisoning direction.
Clean-label poisoning attacks inject innocuous looking (and "correctly" labeled) poison images into training data, causing a model to misclassify a targeted image after being trained on this data. We consider transferable poisoning attacks that succeed without access to the victim network's outputs, architecture, or (i…
Poisoning attack is identified as a severe security threat to machine learning algorithms. In many applications, for example, deep neural network (DNN) models collect public data as the inputs to perform re-training, where the input data can be poisoned. Although poisoning attack against support vector machines (SVM) h…
Aggregation defenses improve deep learning models' robustness against data poisoning attacks.
problem Data poisoning attacks manipulate deep learning models with malicious training samples.
method Deep Partition Aggregation, efficiency improvements, data-to-complexity ratio, poisoning overfitting phenomenon.
result Aggregation defenses boost poisoning robustness through the poisoning overfitting phenomenon.
Research evaluates data poisoning attacks on regression learning and introduces a new defense strategy.
problem Data poisoning attacks on regression learning threaten model integrity in critical systems.
method Realistic scenarios, novel black-box attack, and evaluation on 26 datasets.
result Mean squared error (MSE) increases to 150% with only 2% poisoned samples.
This paper proposes a framework for certifying neural network defenses against data poisoning attacks.
problem Vulnerability of neural networks to data poisoning attacks.
method Random selection based defenses that average predictions on sub-datasets sampled from the training set.
result The certified radius of bagging derived by the framework is tighter than previous work.
Efficient poisoning attack converges to any target classifier with provable convergence.
problem Inducing a corrupted model that misbehaves in favor of an adversary.
method Online convex optimization to find poisoning points incrementally.
result Provably converges to any attainable target classifier.
New defence against data-poisoning attacks in neural networks.
problem Data-poisoning attacks can evade existing defences and increase model efficacy.
method Proved geometric mechanism and identified near clone regime in input space.
result Regularisation and data augmentation reduce data fitting capacity and prevent poisoning.
The Lethal Dose Conjecture limits how much poisoned data can be tolerated.
problem Limiting the impact of poisoned data in machine learning models.
method Theoretical analysis and provable defenses (DPA, FA) based on majority voting.
result DPA and FA are asymptotically optimal defenses against data poisoning.
Paper shows large models are vulnerable to poisoning attacks.
problem Vulnerability of large machine learning models to poisoning attacks.
method Analyzed isotropic random feature vectors and geometric median as robust gradient aggregator rule.
result Linear and logistic regressions with D≥169H2/P2 parameters are vulnerable to arbitrary manipulation by poisoners. Data poisoning is an attack on machine learning models wherein the attacker adds examples to the training set to manipulate the behavior of the model at test time. This paper explores poisoning attacks on neural nets. The proposed attacks use "clean-labels"; they don't require the attacker to have any control over the …
Paper explores how poisoning data can increase privacy risks in machine learning models.
problem Increasing privacy risks of benign training samples through data poisoning attacks.
method Proposes generic and optimization-based attacks to amplify membership exposure.
result Demonstrates substantial increase in membership inference precision with minimal model performance degradation.
Improved defense against data poisoning attacks by aggregating smaller subsets.
problem Mitigating the impact of poisoned data on model robustness.
method Finite Aggregation method that combines duplicates of smaller disjoint subsets for training.
result Consistent improvement in certified robustness bounds, up to 4.77% on GTSRB.
Data poisoning attacks can severely degrade FL models, especially targeting specific classes.
problem Data poisoning attacks against federated learning systems.
method Demonstrated targeted attacks on FL systems, analyzed attack longevity, and proposed a defense strategy.
result Data poisoning attacks can cause substantial drops in classification accuracy and recall with a small percentage of malicious participants.
Many machine learning systems rely on data collected in the wild from untrusted sources, exposing the learning algorithms to data poisoning. Attackers can inject malicious data in the training dataset to subvert the learning process, compromising the performance of the algorithm producing errors in a targeted or an ind…
Data poisoning attacks -- where an adversary can modify a small fraction of training data, with the goal of forcing the trained classifier to high loss -- are an important threat for machine learning in many applications. While a body of prior work has developed attacks and defenses, there is not much general understan…
Subpopulation attacks poison data to misclassify naturally distributed points.
problem Improving accuracy of machine learning predictions through adversarial data modification.
method Introducing a novel subpopulation attack framework, using influence functions and gradient optimization.
result Subpopulation attacks are effective and stealthy, making them difficult to defend against.
Defense against backdoor attacks by clustering incompatible data subsets.
problem Backdoor poisoning attacks on deep neural networks.
method Incompatibility clustering of data subsets during training.
result Successfully reduces attack success rate to below 1%.
Paper shows data poisoning and Byzantine attacks are equivalent, impacting federated learning security.
problem Resilience of federated learning systems to adversarial attacks.
method Proved equivalence between data poisoning and Byzantine gradient attacks.
result Equivalence between data poisoning and Byzantine attacks in federated learning.
New study shows data-oblivious attacks can outperform data-aware ones.
problem Comparing effectiveness of data-oblivious and data-aware poisoning attacks.
method Theoretical study of feature selection with LASSO, focusing on separation between full-information and oblivious attackers.
result Data-oblivious attacks can achieve the same results as full-information attacks for feature selection with LASSO.
Data poisoning attacks compromise the integrity of machine-learning models by introducing malicious training samples to influence the results during test time. In this work, we investigate backdoor data poisoning attack on deep neural networks (DNNs) by inserting a backdoor pattern in the training images. The resulting…
Machine learning models trained on data from the outside world can be corrupted by data poisoning attacks that inject malicious points into the models' training sets. A common defense against these attacks is data sanitization: first filter out anomalous training points before training the model. In this paper, we deve…
New research shows Byzantine failures hurt generalization more than data poisoning in robust distributed learning.
problem Generalization in robust distributed learning algorithms under Byzantine failures vs. data poisoning.
method Algorithmic stability analysis of robust distributed learning algorithms.
result Byzantine failures yield strictly worse generalization rates than data poisoning.
In recent years, a variety of effective neural network-based methods for anomaly and cyber attack detection in industrial control systems (ICSs) have been demonstrated in the literature. Given their successful implementation and widespread use, there is a need to study adversarial attacks on such detection methods to b…
Trustworthy AI is a critical issue in machine learning where, in addition to training a model that is accurate, one must consider both fair and robust training in the presence of data bias and poisoning. However, the existing model fairness techniques mistakenly view poisoned data as an additional bias to be fixed, res…
Poisoning attacks can undermine fairness in machine learning models.
problem Poisoning attacks can introduce classification disparities among different groups in data.
method Developed a gradient-based poisoning attack framework to target algorithmic fairness.
result Demonstrated the effectiveness of poisoning attacks in both white-box and black-box scenarios.
Machine learning (ML) over distributed multi-party data is required for a variety of domains. Existing approaches, such as federated learning, collect the outputs computed by a group of devices at a central aggregator and run iterative algorithms to train a globally shared model. Unfortunately, such approaches are susc…
Paper proposes a game-theoretic approach to generate unlearnable examples.
problem Generating imperceptible perturbations to degrade deep learning models.
method Formulated as a Stackelberg game, proposing a novel attack method GUE.
result GUE effectively poisons models with minimal training data and generalizes well.
We consider data poisoning attacks, a class of adversarial attacks on machine learning where an adversary has the power to alter a small fraction of the training data in order to make the trained classifier satisfy certain objectives. While there has been much prior work on data poisoning, most of it is in the offline …
Efficient active learning method defends against malicious mislabeling and data poisoning attacks.
problem Malicious mislabeling and data poisoning attacks on deep neural networks.
method Adversarial retraining and active learning with random sampling strategy.
result The proposed method achieves 89% accuracy with only one-third of the labeled data, significantly outperforming the baseline method.
Poisoning datasets can reveal private details of other users' training points.
problem Integrity and privacy of machine learning training data.
method Active inference attacks that poison a small fraction of the training dataset.
result Poisoning as little as 0.1% of the training dataset can significantly boost inference attacks.
In big data era, machine learning is one of fundamental techniques in intrusion detection systems (IDSs). However, practical IDSs generally update their decision module by feeding new data then retraining learning models in a periodical way. Hence, some attacks that comprise the data for training or testing classifiers…
We study data poisoning attacks in the online setting where training items arrive sequentially, and the attacker may perturb the current item to manipulate online learning. Importantly, the attacker has no knowledge of future training items nor the data generating distribution. We formulate online data poisoning attack…
Bayesian models can be tricked into believing false data.
problem Vulnerability of Bayesian inference to data poisoning attacks.
method Developed attacks to manipulate Bayesian posterior through deletion and replication of data.
result Demonstrated that Bayesian inference can be steered to target distributions.
New method defends RL agents from poisoning attacks without MDP knowledge.
problem Poisoning attacks on RL systems can cause learning failures.
method Generic poisoning framework for online RL, Vulnerability-Aware Adversarial Critic Poison (VA2C-P).
result Successfully prevents RL agents from learning good policies or converging to target policies.
Machine learning algorithms are vulnerable to poisoning attacks: An adversary can inject malicious points in the training dataset to influence the learning process and degrade the algorithm's performance. Optimal poisoning attacks have already been proposed to evaluate worst-case scenarios, modelling attacks as a bi-le…
New defense mechanism detects and mitigates poisoned regression data.
problem Vulnerability of regression models to targeted data poisoning attacks.
method Introduces N-LID, a measure of local intrinsic dimensionality to distinguish poisoned samples.
result N-LID based defense outperforms state-of-the-art methods in prediction accuracy and runtime.
Paper introduces class-oriented poisoning attacks to improve adversarial availability attacks.
problem Improving adversarial availability attacks on neural networks.
method Gradient-based framework for crafting poisoned images at the class level.
result Demonstrated effectiveness of class-oriented poisoning attacks on various models and datasets.
Study defends shallow neural networks from data-poisoning attacks.
problem Protecting shallow neural networks from adversarial attacks during training.
method Developed a non-gradient stochastic algorithm for depth-2 neural networks, proving near-optimal trade-offs.
result Demonstrated improved performance over stochastic gradient descent under various data distributions.
New method optimizes ML models under poisoned data, improving robustness.
problem Vulnerability of ML models to poisoned data attacks.
method Multiobjective bilevel optimization to consider hyperparameter learning and attack effects.
result Current approaches underestimate model robustness and regularization benefits.
Paper proposes a framework to detect adversarial concept drifts under poisoning attacks.
problem Adversarial concept drift in data streams.
method Augmented Restricted Boltzmann Machine with improved gradient computation and energy function.
result High robustness and efficacy of the proposed drift detection framework in adversarial scenarios.
Bullseye Polytope improves clean-label poisoning attacks in transfer learning.
problem Poisoning neural networks with correctly labeled data.
method Creates poison images with centers close to target images in feature space.
result Improves attack success rate by 26.75% in end-to-end transfer learning.
Paper shows adversarial training can be fooled by new type of noise.
problem Adversarial training can be fooled by new types of noise.
method Designing ADVIN, a new type of inducing noise.
result ADVIN can degrade adversarial training robustness by 99.9%.