New method defends RL agents from poisoning attacks without MDP knowledge.
problem Poisoning attacks on RL systems can cause learning failures.
method Generic poisoning framework for online RL, Vulnerability-Aware Adversarial Critic Poison (VA2C-P).
result Successfully prevents RL agents from learning good policies or converging to target policies.
This paper evaluates targeted data poisoning attacks by focusing on the hardest samples, improving evaluation and defense strategies.
problem The effectiveness of targeted data poisoning attacks is often overestimated due to average evaluation methods.
method The paper introduces metrics to identify the hardest and easiest to poison samples based on clean model information.
result The proposed metrics reliably stratify samples by poisoning vulnerability, enabling rigorous worst-case evaluation and proactive defense.
Paper shows large models are vulnerable to poisoning attacks.
problem Vulnerability of large machine learning models to poisoning attacks.
method Analyzed isotropic random feature vectors and geometric median as robust gradient aggregator rule.
result Linear and logistic regressions with D≥169H2/P2 parameters are vulnerable to arbitrary manipulation by poisoners. Poisoning attacks can undermine fairness in machine learning models.
problem Poisoning attacks can introduce classification disparities among different groups in data.
method Developed a gradient-based poisoning attack framework to target algorithmic fairness.
result Demonstrated the effectiveness of poisoning attacks in both white-box and black-box scenarios.
Machine learning algorithms are vulnerable to poisoning attacks: An adversary can inject malicious points in the training dataset to influence the learning process and degrade the algorithm's performance. Optimal poisoning attacks have already been proposed to evaluate worst-case scenarios, modelling attacks as a bi-le…
Bayesian models can be tricked into believing false data.
problem Vulnerability of Bayesian inference to data poisoning attacks.
method Developed attacks to manipulate Bayesian posterior through deletion and replication of data.
result Demonstrated that Bayesian inference can be steered to target distributions.
Backdoors can be implanted in neural models of source code, and we detect and remove them.
problem Vulnerability of neural models to backdoors in source code.
method Defined and implemented various backdoor classes, adapted robust statistics algorithms, and detected poisoned data through spectral signatures.
result Demonstrated the ease of injecting and removing backdoors in neural models of source code.
Designing models that are robust to small adversarial perturbations of their inputs has proven remarkably difficult. In this work we show that the reverse problem---making models more vulnerable---is surprisingly easy. After presenting some proofs of concept on MNIST, we introduce a generic tilting attack that injects …
Efficient active learning method defends against malicious mislabeling and data poisoning attacks.
problem Malicious mislabeling and data poisoning attacks on deep neural networks.
method Adversarial retraining and active learning with random sampling strategy.
result The proposed method achieves 89% accuracy with only one-third of the labeled data, significantly outperforming the baseline method.
Regularization can improve machine learning models' robustness against poisoning attacks.
problem Poisoning attacks degrade machine learning models' performance by manipulating a fraction of the training data.
method Proposes a novel optimal attack formulation considering the effect of hyperparameters on regularization, leading to better evaluation of robustness.
result Demonstrates that L2 regularization can help mitigate the impact of poisoning attacks. This paper proposes a framework for certifying neural network defenses against data poisoning attacks.
problem Vulnerability of neural networks to data poisoning attacks.
method Random selection based defenses that average predictions on sub-datasets sampled from the training set.
result The certified radius of bagging derived by the framework is tighter than previous work.
Study reveals mutual reinforcement between adversarial inputs and poisoned models.
problem Understanding and mitigating vulnerabilities of deep learning models.
method Developed a new attack model to jointly optimize adversarial inputs and poisoned models.
result Mutual reinforcement effects between adversarial inputs and poisoned models significantly amplify each other's effectiveness.
Machine learning (ML) over distributed multi-party data is required for a variety of domains. Existing approaches, such as federated learning, collect the outputs computed by a group of devices at a central aggregator and run iterative algorithms to train a globally shared model. Unfortunately, such approaches are susc…
Regularisation improves ML classifier stability against poisoning attacks.
problem Poisoning attacks degrade ML algorithms' performance; current attacks ignore hyperparameters.
method Proposed a multiobjective bilevel optimisation problem to consider hyperparameter effects.
result L2 regularisation enhances learning algorithm stability and mitigates poisoning attacks. New defense mechanism detects and mitigates poisoned regression data.
problem Vulnerability of regression models to targeted data poisoning attacks.
method Introduces N-LID, a measure of local intrinsic dimensionality to distinguish poisoned samples.
result N-LID based defense outperforms state-of-the-art methods in prediction accuracy and runtime.
Interpolating label noise makes models vulnerable to adversarial attacks.
problem Adversarial vulnerability of models trained on noisy labels.
method Theoretical analysis of label noise and adversarial risk relationship.
result Uniform label noise induces adversarial risk similar to worst-case poisoning.
New method optimizes ML models under poisoned data, improving robustness.
problem Vulnerability of ML models to poisoned data attacks.
method Multiobjective bilevel optimization to consider hyperparameter learning and attack effects.
result Current approaches underestimate model robustness and regularization benefits.
Recent work has identified that classification models implemented as neural networks are vulnerable to data-poisoning and Trojan attacks at training time. In this work, we show that these training-time vulnerabilities extend to deep reinforcement learning (DRL) agents and can be exploited by an adversary with access to…
Dynamic defense against Byzantine poisoning in federated learning.
problem Byzantine poisoning attacks on federated learning models.
method Dynamic federated aggregation operator to filter out adversarial clients.
result Dynamic selection of clients improves global learning model performance.
Transferability captures the ability of an attack against a machine-learning model to be effective against a different, potentially unknown, model. Empirical evidence for transferability has been shown in previous work, but the underlying reasons why an attack transfers or not are not yet well understood. In this paper…
Efficiently poisons offline RLHF models by flipping preference labels.
problem Vulnerability of offline RLHF models to preference label flipping attacks.
method Developed two attack methods: BAL-A and BMP-A, solving a structured binary sparse approximation problem.
result Demonstrated that flipping one preference label induces a parameter-independent shift in the DPO gradient, enabling structured binary sparse approximation.
UM-GNN improves GNN robustness against poisoning attacks.
problem Vulnerability of GNNs to poisoning attacks.
method UM-GNN uses epistemic uncertainties from message passing to build a surrogate predictor.
result UM-GNN achieves significantly improved robustness against poisoning attacks.
Machine learning has become an important component for many systems and applications including computer vision, spam filtering, malware and network intrusion detection, among others. Despite the capabilities of machine learning algorithms to extract valuable information from data and produce accurate predictions, it ha…
The goal of network representation learning is to learn low-dimensional node embeddings that capture the graph structure and are useful for solving downstream tasks. However, despite the proliferation of such methods, there is currently no study of their robustness to adversarial attacks. We provide the first adversari…
Researchers develop method to protect against 'weight poisoning' attacks on pre-trained models.
problem The security threat of downloading untrusted pre-trained weights that can be manipulated after fine-tuning.
method RIPPLe regularization method and Embedding Surgery initialization procedure.
result Demonstrated that weight poisoning attacks are possible even with limited knowledge of the dataset and fine-tuning procedure.
Clustering algorithms have become a popular tool in computer security to analyze the behavior of malware variants, identify novel malware families, and generate signatures for antivirus systems. However, the suitability of clustering algorithms for security-sensitive settings has been recently questioned by showing tha…
New framework detects and removes malicious model updates in federated learning.
problem Vulnerability of federated learning systems to malicious client attacks.
method Central server learns to detect and remove malicious model updates using a detection model.
result Ensures robust federated learning resilient to Byzantine and targeted model poisoning attacks.
Deep learning models are vulnerable to various adversarial manipulations of their training data, parameters, and input sample. In particular, an adversary can modify the training data and model parameters to embed backdoors into the model, so the model behaves according to the adversary's objective if the input contain…
This work develops secure distributed algorithms for machine learning to protect against data poisoning and network attacks.
problem Vulnerability of distributed machine learning algorithms to cyber threats.
method Game-theoretic framework to capture conflicting goals of a learner and an attacker, iterative distributed algorithm.
result Distributed SVM is prone to fail in different types of attacks, with impact depending on network structure and attack capabilities.
Framework detects and mitigates data-poisoning attacks in causal effect estimation.
problem Vulnerability to append-only attacks in observational causal analyses.
method Develops a data-poisoning audit for augmented inverse-probability-weighted estimation.
result Proposes a greedy scan to compute exact worst-case movement at every append budget.
Backdoor attacks can poison federated meta-learning models, even with few examples.
problem Backdoor attacks on federated meta-learning models.
method Proposed a defense mechanism inspired by matching networks.
result Backdoor attacks can be very successful and persist after additional training in federated meta-learning.
Researchers analyze backdoor data poisoning attacks and identify a memorization capacity parameter.
problem Understanding and mitigating backdoor data poisoning attacks in machine learning models.
method Formal theoretical framework, statistical and computational analysis, explicit constructions, and algorithm design.
result Identified a memorization capacity parameter to assess vulnerability to backdoor attacks and developed algorithms to detect and mitigate them.
Develops a robust training framework to detect backdoor attacks in DNNs.
problem Vulnerability of DNNs to backdoor attacks by poisoned training data.
method Collider framework selects prominent samples based on geometric structures and coreset selection objective.
result Significantly reduces backdoor success rate in various poisoned datasets.
Recommender system is an important component of many web services to help users locate items that match their interests. Several studies showed that recommender systems are vulnerable to poisoning attacks, in which an attacker injects fake data to a given system such that the system makes recommendations as the attacke…
Novel defense algorithm improves SVMs against data poisoning attacks.
problem Vulnerability of SVMs to targeted training data manipulations like poisoning attacks.
method Developed a weighted SVM using K-LID to de-emphasize suspicious data samples.
result Significant reduction in classification error rates (10% on average) with the proposed defense.
While machine learning is going through an era of celebrated success, concerns have been raised about the vulnerability of its backbone: stochastic gradient descent (SGD). Recent approaches have been proposed to ensure the robustness of distributed SGD against adversarial (Byzantine) workers sending poisoned gradients …
Study on reward poisoning attacks on CMAB, revealing attackability depends on adversary's knowledge.
problem Reward poisoning attacks on Combinatorial Multi-Armed Bandits (CMAB).
method Provided a sufficient and necessary condition for attackability, devised an attack algorithm.
result Attackability of CMAB depends on adversary's knowledge of the bandit instance.
Cronus securely transfers model parameters to protect federated learning from poisoning attacks.
problem Federated learning's privacy and security issues, especially with large models.
method Cronus uses black-box knowledge transfer to reduce parameter dimensions and control information leakage.
result Cronus is the only secure federated learning method against poisoning attacks.
New attack targets speech-based AI models via stock market data.
problem Vulnerabilities in speech-based AI models using LLMs.
method Backdoor attack using acoustic data poisoning.
result Shows possible vulnerabilities in speech-based transformers.
Paper proves mathematically that poisoning datasets can be detected.
problem Detecting data poisoning attacks in datasets.
method Mathematical definition and Conformal Separability Test.
result Dataset poisoning can be effectively detected.
Fair machine learning models can be vulnerable to adversarial attacks that reduce their accuracy and fairness.
problem Fairness constraints in machine learning models can compromise their robustness against adversarial attacks.
method Analysis of data poisoning attacks on group-based fair machine learning models, focusing on equalized odds.
result Adversaries can significantly reduce the test accuracy of fair machine learning models and widen their fairness gap.
The susceptibility of deep learning to adversarial attack can be understood in the framework of the Renormalisation Group (RG) and the vulnerability of a specific network may be diagnosed provided the weights in each layer are known. An adversary with access to the inputs and outputs could train a second network to clo…
Efficient poisoning attack converges to any target classifier with provable convergence.
problem Inducing a corrupted model that misbehaves in favor of an adversary.
method Online convex optimization to find poisoning points incrementally.
result Provably converges to any attainable target classifier.
Linear models can be poisoned by shifting a fraction of one class's data, revealing scaling laws and weight alignment.
problem Understanding and quantifying data poisoning in linear models.
method Analysis of ridge least squares with an unpenalized intercept, using resolvent techniques and random matrix theory.
result Closed-form limits for the poisoned score, revealing scaling laws and weight alignment with the poisoning direction.
Unified benchmarks assess data poisoning and backdoor attacks.
problem Unclear danger and effectiveness of data poisoning methods.
method Developed standardized benchmarks for data poisoning and backdoor attacks.
result Existing methods may not generalize to realistic settings.
MetaPoison poisons neural networks by learning to craft imperceptible changes to training data.
problem Data poisoning attacks on neural networks.
method MetaPoison uses meta-learning to approximate bilevel optimization for crafting poisons.
result MetaPoison outperforms previous methods and works in various scenarios.
Clean-label poisoning attacks inject innocuous looking (and "correctly" labeled) poison images into training data, causing a model to misclassify a targeted image after being trained on this data. We consider transferable poisoning attacks that succeed without access to the victim network's outputs, architecture, or (i…
Bullseye Polytope improves clean-label poisoning attacks in transfer learning.
problem Poisoning neural networks with correctly labeled data.
method Creates poison images with centers close to target images in feature space.
result Improves attack success rate by 26.75% in end-to-end transfer learning.