This paper proposes a framework for certifying neural network defenses against data poisoning attacks.
problem Vulnerability of neural networks to data poisoning attacks.
method Random selection based defenses that average predictions on sub-datasets sampled from the training set.
result The certified radius of bagging derived by the framework is tighter than previous work.
This paper evaluates targeted data poisoning attacks by focusing on the hardest samples, improving evaluation and defense strategies.
problem The effectiveness of targeted data poisoning attacks is often overestimated due to average evaluation methods.
method The paper introduces metrics to identify the hardest and easiest to poison samples based on clean model information.
result The proposed metrics reliably stratify samples by poisoning vulnerability, enabling rigorous worst-case evaluation and proactive defense.
Linear models can be poisoned by shifting a fraction of one class's data, revealing scaling laws and weight alignment.
problem Understanding and quantifying data poisoning in linear models.
method Analysis of ridge least squares with an unpenalized intercept, using resolvent techniques and random matrix theory.
result Closed-form limits for the poisoned score, revealing scaling laws and weight alignment with the poisoning direction.
The Lethal Dose Conjecture limits how much poisoned data can be tolerated.
problem Limiting the impact of poisoned data in machine learning models.
method Theoretical analysis and provable defenses (DPA, FA) based on majority voting.
result DPA and FA are asymptotically optimal defenses against data poisoning.
New defense against DNN model stealing attacks by poisoning predictions.
problem Stealing DNN models through black-box access attacks.
method Active perturbation of predictions to mislead attacker's training.
result Effective against a wide range of DNN stealing attacks, reducing attacker's success rate significantly.
Detects poisoned training samples in deep neural networks.
problem Data poisoning attacks on deep neural networks.
method Two approaches: parametric probability distributions and Bayesian deep neural networks.
result Uncertainty estimates from trained models can discriminate clean from poisoned samples.
New defense mechanism detects and mitigates poisoned regression data.
problem Vulnerability of regression models to targeted data poisoning attacks.
method Introduces N-LID, a measure of local intrinsic dimensionality to distinguish poisoned samples.
result N-LID based defense outperforms state-of-the-art methods in prediction accuracy and runtime.
Improved defense against data poisoning attacks by aggregating smaller subsets.
problem Mitigating the impact of poisoned data on model robustness.
method Finite Aggregation method that combines duplicates of smaller disjoint subsets for training.
result Consistent improvement in certified robustness bounds, up to 4.77% on GTSRB.
Paper introduces class-oriented poisoning attacks to improve adversarial availability attacks.
problem Improving adversarial availability attacks on neural networks.
method Gradient-based framework for crafting poisoned images at the class level.
result Demonstrated effectiveness of class-oriented poisoning attacks on various models and datasets.
Backdoors can be implanted in neural models of source code, and we detect and remove them.
problem Vulnerability of neural models to backdoors in source code.
method Defined and implemented various backdoor classes, adapted robust statistics algorithms, and detected poisoned data through spectral signatures.
result Demonstrated the ease of injecting and removing backdoors in neural models of source code.
Subpopulation attacks poison data to misclassify naturally distributed points.
problem Improving accuracy of machine learning predictions through adversarial data modification.
method Introducing a novel subpopulation attack framework, using influence functions and gradient optimization.
result Subpopulation attacks are effective and stealthy, making them difficult to defend against.
New method certifies regression robustness without data distribution assumptions.
problem Certifying robustness of regression models against poisoning attacks.
method Reduces certified regression to certified classification using median decision function.
result Proposes six new provably-robust regression models.
Paper evaluates robustness of NAS against poisoning attacks.
problem Robustness of Neural Architecture Search (NAS) against poisoning attacks.
method Evaluation of Efficient NAS (ENAS) against carefully designed ineffective operations in poisoning attacks.
result Demonstrates how poisoning attacks exploit design flaws in ENAS controller.
We study data poisoning attacks in the online setting where training items arrive sequentially, and the attacker may perturb the current item to manipulate online learning. Importantly, the attacker has no knowledge of future training items nor the data generating distribution. We formulate online data poisoning attack…
Paper studies autoencoder-based anomaly detectors' robustness to adversarial poisoning attacks.
problem Adversarial poisoning attacks on online-trained autoencoder-based anomaly detectors in ICSs.
method Proposes two algorithms for generating poison samples and evaluates them on synthetic and real-world ICS data.
result Autoencoder detectors are resilient to poisoning in the face of all relevant attacks in the SWaT dataset.
Timber targets decision trees, outperforming existing attacks.
problem Poisoning decision trees to manipulate model predictions.
method Greedy attack strategy using sub-tree retraining for efficiency.
result Timber outperforms existing attacks in effectiveness and efficiency.
Researchers show how to manipulate Partial Dependence plots to deceive explanations of predictive models.
problem The robustness and trustworthiness of Partial Dependence (PD) explanations are compromised.
method Data poisoning using genetic and gradient algorithms to manipulate PD plots.
result PD explanations can be fooled and manipulated to mislead understanding of predictive models.
Paper proves mathematically that poisoning datasets can be detected.
problem Detecting data poisoning attacks in datasets.
method Mathematical definition and Conformal Separability Test.
result Dataset poisoning can be effectively detected.
New method defends RL agents from poisoning attacks without MDP knowledge.
problem Poisoning attacks on RL systems can cause learning failures.
method Generic poisoning framework for online RL, Vulnerability-Aware Adversarial Critic Poison (VA2C-P).
result Successfully prevents RL agents from learning good policies or converging to target policies.
UM-GNN improves GNN robustness against poisoning attacks.
problem Vulnerability of GNNs to poisoning attacks.
method UM-GNN uses epistemic uncertainties from message passing to build a surrogate predictor.
result UM-GNN achieves significantly improved robustness against poisoning attacks.
Efficient poisoning attack converges to any target classifier with provable convergence.
problem Inducing a corrupted model that misbehaves in favor of an adversary.
method Online convex optimization to find poisoning points incrementally.
result Provably converges to any attainable target classifier.
Unified benchmarks assess data poisoning and backdoor attacks.
problem Unclear danger and effectiveness of data poisoning methods.
method Developed standardized benchmarks for data poisoning and backdoor attacks.
result Existing methods may not generalize to realistic settings.
MetaPoison poisons neural networks by learning to craft imperceptible changes to training data.
problem Data poisoning attacks on neural networks.
method MetaPoison uses meta-learning to approximate bilevel optimization for crafting poisons.
result MetaPoison outperforms previous methods and works in various scenarios.
Clean-label poisoning attacks inject innocuous looking (and "correctly" labeled) poison images into training data, causing a model to misclassify a targeted image after being trained on this data. We consider transferable poisoning attacks that succeed without access to the victim network's outputs, architecture, or (i…
Bullseye Polytope improves clean-label poisoning attacks in transfer learning.
problem Poisoning neural networks with correctly labeled data.
method Creates poison images with centers close to target images in feature space.
result Improves attack success rate by 26.75% in end-to-end transfer learning.
Temporal threat model defends against data poisoning with timestamps.
problem Adversaries can poison more samples than expected, rendering existing defenses ineffective.
method Leverage timestamps to define earliness and duration metrics for temporal robustness.
result Temporal aggregation provides provable temporal robustness against data poisoning.
Machine learning has become an important component for many systems and applications including computer vision, spam filtering, malware and network intrusion detection, among others. Despite the capabilities of machine learning algorithms to extract valuable information from data and produce accurate predictions, it ha…
Poisoning attack is identified as a severe security threat to machine learning algorithms. In many applications, for example, deep neural network (DNN) models collect public data as the inputs to perform re-training, where the input data can be poisoned. Although poisoning attack against support vector machines (SVM) h…
Data poisoning is an attack on machine learning models wherein the attacker adds examples to the training set to manipulate the behavior of the model at test time. This paper explores poisoning attacks on neural nets. The proposed attacks use "clean-labels"; they don't require the attacker to have any control over the …
Aggregation defenses improve deep learning models' robustness against data poisoning attacks.
problem Data poisoning attacks manipulate deep learning models with malicious training samples.
method Deep Partition Aggregation, efficiency improvements, data-to-complexity ratio, poisoning overfitting phenomenon.
result Aggregation defenses boost poisoning robustness through the poisoning overfitting phenomenon.
Paper models game theory for defending against data poisoning attacks.
problem Defending against data poisoning attacks using game theory.
method Modeling attacker-defender game, proving non-existence of pure Nash Equilibrium, proposing mixed strategy approach, and developing an algorithm to approximate Nash Equilibrium.
result Demonstrated effectiveness of mixed strategy defense in experiments.
Researchers develop method to protect against 'weight poisoning' attacks on pre-trained models.
problem The security threat of downloading untrusted pre-trained weights that can be manipulated after fine-tuning.
method RIPPLe regularization method and Embedding Surgery initialization procedure.
result Demonstrated that weight poisoning attacks are possible even with limited knowledge of the dataset and fine-tuning procedure.
Paper shows large models are vulnerable to poisoning attacks.
problem Vulnerability of large machine learning models to poisoning attacks.
method Analyzed isotropic random feature vectors and geometric median as robust gradient aggregator rule.
result Linear and logistic regressions with D≥169H2/P2 parameters are vulnerable to arbitrary manipulation by poisoners. Research evaluates data poisoning attacks on regression learning and introduces a new defense strategy.
problem Data poisoning attacks on regression learning threaten model integrity in critical systems.
method Realistic scenarios, novel black-box attack, and evaluation on 26 datasets.
result Mean squared error (MSE) increases to 150% with only 2% poisoned samples.
New defence against data-poisoning attacks in neural networks.
problem Data-poisoning attacks can evade existing defences and increase model efficacy.
method Proved geometric mechanism and identified near clone regime in input space.
result Regularisation and data augmentation reduce data fitting capacity and prevent poisoning.
Generative model creates poisoning attacks against machine learning classifiers.
problem Poisoning attacks degrade machine learning classifier performance.
method Generative Adversarial Net with generator, discriminator, and classifier components.
result Effectively compromises machine learning classifiers, including deep networks.
Paper explores how poisoning data can increase privacy risks in machine learning models.
problem Increasing privacy risks of benign training samples through data poisoning attacks.
method Proposes generic and optimization-based attacks to amplify membership exposure.
result Demonstrates substantial increase in membership inference precision with minimal model performance degradation.
FR-Train improves fair and robust AI training by detecting and reducing poisoned data.
problem Training AI models that are fair and robust in the presence of data bias and poisoning.
method Mutual information-based adversarial training with an additional discriminator.
result FR-Train maintains fairness and accuracy even in the presence of poisoned data.
PA-GNN enhances GNN robustness against poisoning attacks using clean graph knowledge.
problem Improving robustness of GNNs against poisoning attacks.
method PA-GNN uses a penalized aggregation mechanism and meta-optimization to transfer robustness from clean graphs.
result PA-GNN significantly improves GNN robustness against poisoning attacks on real-world graphs.
Framework detects and mitigates data-poisoning attacks in causal effect estimation.
problem Vulnerability to append-only attacks in observational causal analyses.
method Develops a data-poisoning audit for augmented inverse-probability-weighted estimation.
result Proposes a greedy scan to compute exact worst-case movement at every append budget.
Data poisoning attacks can severely degrade FL models, especially targeting specific classes.
problem Data poisoning attacks against federated learning systems.
method Demonstrated targeted attacks on FL systems, analyzed attack longevity, and proposed a defense strategy.
result Data poisoning attacks can cause substantial drops in classification accuracy and recall with a small percentage of malicious participants.
Poisoning attacks can undermine fairness in machine learning models.
problem Poisoning attacks can introduce classification disparities among different groups in data.
method Developed a gradient-based poisoning attack framework to target algorithmic fairness.
result Demonstrated the effectiveness of poisoning attacks in both white-box and black-box scenarios.
Making learners robust to adversarial perturbation at test time (i.e., evasion attacks) or training time (i.e., poisoning attacks) has emerged as a challenging task. It is known that for some natural settings, sublinear perturbations in the training phase or the testing phase can drastically decrease the quality of the…
Many machine learning systems rely on data collected in the wild from untrusted sources, exposing the learning algorithms to data poisoning. Attackers can inject malicious data in the training dataset to subvert the learning process, compromising the performance of the algorithm producing errors in a targeted or an ind…
Researchers develop methods to protect reinforcement learning and control systems from policy poisoning attacks.
problem Attacks on reinforcement learning and control systems that manipulate learned policies.
method Unified framework for solving policy poisoning attacks, demonstrating global optimality and feasibility.
result Policy poisoning attacks are feasible and can be defended with a convex optimization approach.
Deep Partition Aggregation defends against poisoning attacks with provable certificates.
problem Adversarial poisoning attacks corrupt classifier test-time behavior.
method Deep Partition Aggregation (DPA) is an ensemble method using hash partitions and base models trained on these partitions.
result DPA can certify >= 50% of test images against over 500 poison image insertions on MNIST, and nine insertions on CIFAR-10.
Regularization helps protect machine learning models from poisoning attacks.
problem Mitigating the impact of poisoned data on machine learning models.
method Distributionally-robust optimization using Wasserstein distance to find an upper bound for worst-case fitness.
result The regularizer is equal to the dual norm of the model parameters for regression models.
Regularisation improves ML classifier stability against poisoning attacks.
problem Poisoning attacks degrade ML algorithms' performance; current attacks ignore hyperparameters.
method Proposed a multiobjective bilevel optimisation problem to consider hyperparameter effects.
result L2 regularisation enhances learning algorithm stability and mitigates poisoning attacks.