Reward-poisoning attacks can force RL agents to learn bad policies, and we categorize and quantify their feasibility.
problem Reward-poisoning attacks can manipulate RL agents to learn undesirable policies.
method Categorize attacks by infinity-norm constraint, provide thresholds for feasibility, and develop adaptive attack strategies.
result Adaptive reward-poisoning attacks can achieve the nefarious policy in polynomial steps, while non-adaptive attacks require exponential steps.
New adaptive attacks bypass many defenses to adversarial examples.
problem Adversarial example defenses are not adequately evaluated using adaptive attacks.
method Detailed analysis of thirteen defenses, demonstrating their vulnerabilities to adaptive attacks.
result Adversarial example defenses are more vulnerable to adaptive attacks than previously thought.
Automated discovery of adaptive attacks improves adversarial defense evaluation.
problem Challenges in reliably evaluating adversarial defenses.
method Formalizes adaptive attacks as reusable building blocks in a search space for automatic discovery.
result Our tool discovers significantly stronger attacks than AutoAttack, improving adversarial defense evaluation.
Stochastic multi-armed bandits form a class of online learning problems that have important applications in online recommendation systems, adaptive medical treatment, and many others. Even though potential attacks against these learning algorithms may hijack their behavior, causing catastrophic loss in real-world appli…
The problem of spoofing attacks is increasingly relevant as digital systems are becoming more ubiquitous. Thus the detection of such attacks and the localisation of attackers have been objects of recent study. After an attack has been detected, various algorithms have been proposed in order to localise the attacker. In…
Simple adaptive attacks can jailbreak state-of-the-art safety-aligned language models.
problem Vulnerability of safety-aligned language models to simple adaptive attacks.
method Designing adversarial prompt templates, random search on suffixes, and transfer/prefilling attacks.
result 100% attack success rate on various models including GPT-4, Vicuna, Mistral, and Claude.
Neural networks are vulnerable to adversarially-constructed perturbations of their inputs. Most research so far has considered perturbations of a fixed magnitude under some lp norm. Although studying these attacks is valuable, there has been increasing interest in the construction of (and robustness to) unrestricted…
ADAPT improves robustness of Vision Transformers without full model fine-tuning.
problem Vulnerability of Vision Transformers to adversarial attacks.
method Parameter-efficient prompt tuning with ADAPT framework for adaptive adversarial training.
result ADAPT achieves robust accuracy of ~40% w.r.t. SOTA methods using only ~1% of the parameters.
A new attack for probabilistic classifiers adapts to noise levels.
problem Adversarial examples can mislead probabilistic classifiers.
method Adapts HopSkipJump attack for probabilistic classifiers, adjusting queries based on noise levels.
result Decision-based attacks are effective against probabilistic classifiers, even with noise.
AFD learns features resistant to adversarial attacks.
problem Vulnerability of neural networks to adversarial attacks.
method AFD learns features invariant to adversarial perturbations through a game of predictive and robust features.
result AFD effectively resists a wide range of attack types and strengths.
Meta learning can adapt fast but is vulnerable to adversarial attacks.
problem Vulnerability of meta learning to adversarial attacks.
method Formal definition of adversarial attacks unique to meta learning, proposing an attacking algorithm.
result Meta learning is vulnerable to adversarial attacks.
Study robustness of early-stopping GD for linear regression attacks.
problem Robustness of gradient-descent methods to adversarial attacks.
method Early-stopping strategies, gradient-descent, Mahalanobis attacks, feature-dependent learning rates, data transformations.
result Early-stopped GD is optimally robust to Euclidean-norm attacks but sub-optimal for Mahalanobis attacks.
Backdoor attacks can poison federated meta-learning models, even with few examples.
problem Backdoor attacks on federated meta-learning models.
method Proposed a defense mechanism inspired by matching networks.
result Backdoor attacks can be very successful and persist after additional training in federated meta-learning.
Study membership inference under skewed priors and adaptive thresholds, improving attack accuracy.
problem Membership inference in imbalanced settings with selective thresholding.
method Developed PPV metric for skewed priors, threshold selection procedure, and a new inference attack.
result Improved inference attack accuracy in imbalanced settings.
We study offline data poisoning attacks in contextual bandits, a class of reinforcement learning problems with important applications in online recommendation and adaptive medical treatment, among others. We provide a general attack framework based on convex optimization and show that by slightly manipulating rewards i…
Deep Neural Networks (DNNs) are susceptible to model stealing attacks, which allows a data-limited adversary with no knowledge of the training dataset to clone the functionality of a target model, just by using black-box query access. Such attacks are typically carried out by querying the target model using inputs that…
This paper improves model robustness against adversarial attacks using optimal transport.
problem Adversarial attacks can mislead deep learning models with imperceptible perturbations.
method Exploits optimal transport theory to align adversarial and original image distributions.
result SAT (Sinkhorn Adversarial Training) leads to more robust models compared to state-of-the-art.
AdvMind detects adversary intent in black-box attacks with high accuracy.
problem Detecting adversary intent in black-box adversarial attacks is challenging.
method AdvMind accounts for adversary adaptiveness and synthesizes queries to expose intent.
result AdvMind detects adversary intent with over 75% accuracy after observing less than 3 query batches.
Paper proposes a method to improve building extraction from aerial images by adapting CNN models.
problem Limited generalization of CNN-based segmentation models for unseen images.
method Combines domain transfer and adversarial attack concepts to adapt input images to target images.
result Improves overall IoU and outperforms other methods in cross-dataset experiments.
Recent advances in smart cities applications enforce security threads such as node replication attacks. Such attack is take place when the attacker plants a replicated network node within the network. Vehicular Ad hoc networks are connecting sensors that have limited resources and required the response time to be as lo…
Deep neural networks are vulnerable to adversarial examples, i.e., carefully-perturbed inputs aimed to mislead classification. This work proposes a detection method based on combining non-linear dimensionality reduction and density estimation techniques. Our empirical findings show that the proposed approach is able to…
Paper introduces timing-based adversarial attacks on DRL-based navigation systems.
problem Vulnerability of DRL-based navigation systems to adversarial attacks.
method Timing-based adversarial strategies using physical noise patterns.
result Adversarial timing attacks significantly degrade DRL-based navigation performance.
First robust bandit algorithm for contextual bandits with sub-linear regret.
problem Vulnerability of linear contextual bandit algorithms to adversarial attacks.
method Proposes a robust bandit algorithm for stochastic linear contextual bandits under fully adaptive and omniscient attacks.
result Sub-linear regret under various attacks without requiring attack information.
New attacks inflate earnings while reducing fraud scores, potentially millions at stake.
problem Manipulating financial reports to hide distress and gain.
method Maximum Violated Multi-Objective (MVMO) attacks that adapt search direction.
result Inflation of earnings by 100-200% while reducing fraud scores by 15% in 50% of cases.
In this appraisal paper, we evaluate the efficacy of SHIELD, a compression-based defense framework for countering adversarial attacks on image classification models, which was published at KDD 2018. Here, we consider alternative threat models not studied in the original work, where we assume that an adaptive adversary …
Deep learning classifiers are known to be vulnerable to adversarial examples. A recent paper presented at ICML 2019 proposed a statistical test detection method based on the observation that logits of noisy adversarial examples are biased toward the true class. The method is evaluated on CIFAR-10 dataset and is shown t…
New research shows many recent defenses against adversarial examples are ineffective against black-box attacks.
problem The robustness of recent defenses against adversarial examples is insufficient, especially against black-box attacks.
method Evaluation of nine defenses on two black-box adversarial models and six attacks on CIFAR-10 and Fashion-MNIST datasets.
result Most recent defenses provide only marginal improvements in security (<25%) compared to undefended networks. Recently, deep neural networks have significant progress and successful application in various fields, but they are found vulnerable to attack instances, e.g., adversarial examples. State-of-art attack methods can generate attack images by adding small perturbation to the source image. These attack images can fool the …
New attack reveals memorization patterns in pre-trained LLMs.
problem Determining if a data point was part of a pre-trained LLM's training set.
method Adapts MIA statistical tests to LLM's perplexity dynamics of subsequences.
result Significantly outperforms prior approaches in membership inference attacks.
Recent studies show that Deep Reinforcement Learning (DRL) models are vulnerable to adversarial attacks, which attack DRL models by adding small perturbations to the observations. However, some attacks assume full availability of the victim model, and some require a huge amount of computation, making them less feasible…
Paper tackles adversarial attacks on nonparametric regression models.
problem Vulnerability of machine learning models to adversarial attacks in nonparametric regression.
method Establishes minimax rate and proposes adaptive estimators for robust nonparametric regression under adversarial Lq-risks. result Achieves minimax optimality and provides adaptive estimators for robust nonparametric regression.
New NIDS uses hypergraphs for real-time detection of evolving port scans.
problem Challenges in offline NIDS for auto-generated port scans.
method Hypergraph-based metrics for real-time ML ensemble NIDS.
result High accuracy, precision, and recall in detecting port scans and attacks.
Neural networks have been proven to be vulnerable to a variety of adversarial attacks. From a safety perspective, highly sparse adversarial attacks are particularly dangerous. On the other hand the pixelwise perturbations of sparse attacks are typically large and thus can be potentially detected. We propose a new black…
TAROT enhances robustness and domain adaptability with domain-invariant features.
problem Developing models robust to adversarial attacks across diverse domains.
method Derives a new generalization bound and proposes TAROT algorithm.
result TAROT outperforms state-of-the-art methods in accuracy and robustness.
Deep neural networks (DNN) are known to be vulnerable to adversarial attacks. Numerous efforts either try to patch weaknesses in trained models, or try to make it difficult or costly to compute adversarial examples that exploit them. In our work, we explore a new "honeypot" approach to protect DNN models. We intentiona…
VarDetect monitors API queries to detect model extraction attacks.
problem Attackers can extract ML models through API queries.
method VarDetect uses a modified variational autoencoder to track and detect attacker samples.
result VarDetect successfully detects and raises alarms for model extraction attacks.
Deep learning models are vulnerable to adversarial examples crafted by applying human-imperceptible perturbations on benign inputs. However, under the black-box setting, most existing adversaries often have a poor transferability to attack other defense models. In this work, from the perspective of regarding the advers…
Convolutional neural networks (CNNs) are known for their good performance and generalization in vision-related tasks and have become state-of-the-art in both application and research-based domains. However, just like other neural network models, they suffer from a susceptibility to noise and adversarial attacks. An adv…
New method improves privacy risk evaluation of machine learning models.
problem Machine learning models can be vulnerable to membership inference attacks.
method Proposed new inference attack method based on prediction entropy, and introduced privacy risk score metric.
result Existing defense approaches are not as effective as previously reported.
New algorithms tackle adversarial multi-player bandits with forced-collision communication.
problem No-sensing adversarial multi-player multi-armed bandits (MP-MAB) problem.
method Adversary-Adaptive Collision-Communication (A2C2) algorithms, attackability-aware and unaware settings, information-theoretic tools, error-correction coding.
result Asymptotic attackability-dependent sublinear regret achieved, with or without knowing attackability.
SmoothLLM defends LLMs from jailbreaking attacks by randomly perturbing inputs.
problem Adversaries can fool large language models into generating objectionable content.
method SmoothLLM randomly perturbs multiple copies of a prompt and aggregates predictions to detect adversarial inputs.
result SmoothLLM sets the state-of-the-art for robustness against various jailbreak attacks.
Graph neural networks are vulnerable to adversarial attacks by manipulating graph structure.
problem Vulnerability of Graph Neural Networks to adversarial attacks.
method Categorization and review of existing attacks and defenses.
result Developed a repository for empirical studies on graph adversarial attacks and defenses.
Adversarial examples pose a threat to deep neural network models in a variety of scenarios, from settings where the adversary has complete knowledge of the model and to the opposite "black box" setting. Black box attacks are particularly threatening as the adversary only needs access to the input and output of the mode…
The evaluation of robustness against adversarial manipulation of neural networks-based classifiers is mainly tested with empirical attacks as methods for the exact computation, even when available, do not scale to large networks. We propose in this paper a new white-box adversarial attack wrt the lp-norms for $p \in…
SAPAG attacks distributed learning by reconstructing true training data from gradients.
problem Privacy attacks on distributed learning systems through gradients.
method SAPAG uses a Gaussian kernel-based gradient difference distance measure.
result SAPAG can reconstruct training data on various DNNs and at different training phases.
New defense method inspired by encryption improves visual classification accuracy.
problem Conventional defenses reduce accuracy and are defeated by obfuscated gradients.
method Block-wise pixel shuffling with secret key for training and test images.
result Achieves high accuracy (91.55%) on clean images and (89.66%) on adversarial examples.
Paper proposes a framework to detect adversarial concept drifts under poisoning attacks.
problem Adversarial concept drift in data streams.
method Augmented Restricted Boltzmann Machine with improved gradient computation and energy function.
result High robustness and efficacy of the proposed drift detection framework in adversarial scenarios.
Efficiently poisons offline RLHF models by flipping preference labels.
problem Vulnerability of offline RLHF models to preference label flipping attacks.
method Developed two attack methods: BAL-A and BMP-A, solving a structured binary sparse approximation problem.
result Demonstrated that flipping one preference label induces a parameter-independent shift in the DPO gradient, enabling structured binary sparse approximation.