Reward-poisoning attacks can force RL agents to learn bad policies, and we categorize and quantify their feasibility.
problem Reward-poisoning attacks can manipulate RL agents to learn undesirable policies.
method Categorize attacks by infinity-norm constraint, provide thresholds for feasibility, and develop adaptive attack strategies.
result Adaptive reward-poisoning attacks can achieve the nefarious policy in polynomial steps, while non-adaptive attacks require exponential steps.
New adaptive attacks bypass many defenses to adversarial examples.
problem Adversarial example defenses are not adequately evaluated using adaptive attacks.
method Detailed analysis of thirteen defenses, demonstrating their vulnerabilities to adaptive attacks.
result Adversarial example defenses are more vulnerable to adaptive attacks than previously thought.
Automated discovery of adaptive attacks improves adversarial defense evaluation.
problem Challenges in reliably evaluating adversarial defenses.
method Formalizes adaptive attacks as reusable building blocks in a search space for automatic discovery.
result Our tool discovers significantly stronger attacks than AutoAttack, improving adversarial defense evaluation.
Stochastic multi-armed bandits form a class of online learning problems that have important applications in online recommendation systems, adaptive medical treatment, and many others. Even though potential attacks against these learning algorithms may hijack their behavior, causing catastrophic loss in real-world appli…
The problem of spoofing attacks is increasingly relevant as digital systems are becoming more ubiquitous. Thus the detection of such attacks and the localisation of attackers have been objects of recent study. After an attack has been detected, various algorithms have been proposed in order to localise the attacker. In…
Simple adaptive attacks can jailbreak state-of-the-art safety-aligned language models.
problem Vulnerability of safety-aligned language models to simple adaptive attacks.
method Designing adversarial prompt templates, random search on suffixes, and transfer/prefilling attacks.
result 100% attack success rate on various models including GPT-4, Vicuna, Mistral, and Claude.
Neural networks are vulnerable to adversarially-constructed perturbations of their inputs. Most research so far has considered perturbations of a fixed magnitude under some lp norm. Although studying these attacks is valuable, there has been increasing interest in the construction of (and robustness to) unrestricted…
ADAPT improves robustness of Vision Transformers without full model fine-tuning.
problem Vulnerability of Vision Transformers to adversarial attacks.
method Parameter-efficient prompt tuning with ADAPT framework for adaptive adversarial training.
result ADAPT achieves robust accuracy of ~40% w.r.t. SOTA methods using only ~1% of the parameters.
A new attack for probabilistic classifiers adapts to noise levels.
problem Adversarial examples can mislead probabilistic classifiers.
method Adapts HopSkipJump attack for probabilistic classifiers, adjusting queries based on noise levels.
result Decision-based attacks are effective against probabilistic classifiers, even with noise.
Adaptive Misinformation defends against model stealing attacks by sending incorrect predictions for OOD queries.
problem Model stealing attacks clone target models using black-box query access and a surrogate dataset.
method Selective sending of incorrect predictions for Out-Of-Distribution (OOD) queries to degrade attacker's clone model accuracy.
result Our defense reduces attacker's clone model accuracy by up to 40% while maintaining benign user accuracy under 0.5%.
A new fast adversarial attack finds minimal perturbations to change class.
problem Evaluating robustness of neural networks against adversarial attacks.
method White-box adversarial attack minimizing perturbation size.
result Outperforms or matches state-of-the-art attacks.
AFD learns features resistant to adversarial attacks.
problem Vulnerability of neural networks to adversarial attacks.
method AFD learns features invariant to adversarial perturbations through a game of predictive and robust features.
result AFD effectively resists a wide range of attack types and strengths.
Meta learning can adapt fast but is vulnerable to adversarial attacks.
problem Vulnerability of meta learning to adversarial attacks.
method Formal definition of adversarial attacks unique to meta learning, proposing an attacking algorithm.
result Meta learning is vulnerable to adversarial attacks.
Study robustness of early-stopping GD for linear regression attacks.
problem Robustness of gradient-descent methods to adversarial attacks.
method Early-stopping strategies, gradient-descent, Mahalanobis attacks, feature-dependent learning rates, data transformations.
result Early-stopped GD is optimally robust to Euclidean-norm attacks but sub-optimal for Mahalanobis attacks.
Backdoor attacks can poison federated meta-learning models, even with few examples.
problem Backdoor attacks on federated meta-learning models.
method Proposed a defense mechanism inspired by matching networks.
result Backdoor attacks can be very successful and persist after additional training in federated meta-learning.
Study membership inference under skewed priors and adaptive thresholds, improving attack accuracy.
problem Membership inference in imbalanced settings with selective thresholding.
method Developed PPV metric for skewed priors, threshold selection procedure, and a new inference attack.
result Improved inference attack accuracy in imbalanced settings.
We study offline data poisoning attacks in contextual bandits, a class of reinforcement learning problems with important applications in online recommendation and adaptive medical treatment, among others. We provide a general attack framework based on convex optimization and show that by slightly manipulating rewards i…
This paper improves model robustness against adversarial attacks using optimal transport.
problem Adversarial attacks can mislead deep learning models with imperceptible perturbations.
method Exploits optimal transport theory to align adversarial and original image distributions.
result SAT (Sinkhorn Adversarial Training) leads to more robust models compared to state-of-the-art.
AdvMind detects adversary intent in black-box attacks with high accuracy.
problem Detecting adversary intent in black-box adversarial attacks is challenging.
method AdvMind accounts for adversary adaptiveness and synthesizes queries to expose intent.
result AdvMind detects adversary intent with over 75% accuracy after observing less than 3 query batches.
Paper proposes a method to improve building extraction from aerial images by adapting CNN models.
problem Limited generalization of CNN-based segmentation models for unseen images.
method Combines domain transfer and adversarial attack concepts to adapt input images to target images.
result Improves overall IoU and outperforms other methods in cross-dataset experiments.
Recent advances in smart cities applications enforce security threads such as node replication attacks. Such attack is take place when the attacker plants a replicated network node within the network. Vehicular Ad hoc networks are connecting sensors that have limited resources and required the response time to be as lo…
Deep RL enhances cyber security through adaptive, responsive, and scalable defenses.
problem Complex, dynamic cyber attacks require adaptive and scalable security solutions.
method Incorporates deep learning into traditional RL for cyber defense.
result Demonstrates the potential of DRL in solving high-dimensional cyber defense problems.
RL agents learn to detect and mitigate adversarial attacks.
problem Adversarial attacks on Deep RL algorithms deployed in real-life applications.
method Meta-Learned Advantage Hierarchy (MLAH) agent using meta-learning for online robustness.
result The MLAH agent exhibits hierarchical coping behaviors and maintains higher reward distributions over time.
Deep neural networks are vulnerable to adversarial examples, i.e., carefully-perturbed inputs aimed to mislead classification. This work proposes a detection method based on combining non-linear dimensionality reduction and density estimation techniques. Our empirical findings show that the proposed approach is able to…
Paper introduces timing-based adversarial attacks on DRL-based navigation systems.
problem Vulnerability of DRL-based navigation systems to adversarial attacks.
method Timing-based adversarial strategies using physical noise patterns.
result Adversarial timing attacks significantly degrade DRL-based navigation performance.
First robust bandit algorithm for contextual bandits with sub-linear regret.
problem Vulnerability of linear contextual bandit algorithms to adversarial attacks.
method Proposes a robust bandit algorithm for stochastic linear contextual bandits under fully adaptive and omniscient attacks.
result Sub-linear regret under various attacks without requiring attack information.
New attacks inflate earnings while reducing fraud scores, potentially millions at stake.
problem Manipulating financial reports to hide distress and gain.
method Maximum Violated Multi-Objective (MVMO) attacks that adapt search direction.
result Inflation of earnings by 100-200% while reducing fraud scores by 15% in 50% of cases.
In this appraisal paper, we evaluate the efficacy of SHIELD, a compression-based defense framework for countering adversarial attacks on image classification models, which was published at KDD 2018. Here, we consider alternative threat models not studied in the original work, where we assume that an adaptive adversary …
New research shows many recent defenses against adversarial examples are ineffective against black-box attacks.
problem The robustness of recent defenses against adversarial examples is insufficient, especially against black-box attacks.
method Evaluation of nine defenses on two black-box adversarial models and six attacks on CIFAR-10 and Fashion-MNIST datasets.
result Most recent defenses provide only marginal improvements in security (<25%) compared to undefended networks. New technique crafts imperceivable sparse adversarial attacks.
problem Vulnerability of neural networks to adversarial attacks.
method Proposes a black-box technique to minimize l0-distance, integrating componentwise constraints. result Adversarial examples are almost imperceivable and non-detectable.
Recently, deep neural networks have significant progress and successful application in various fields, but they are found vulnerable to attack instances, e.g., adversarial examples. State-of-art attack methods can generate attack images by adding small perturbation to the source image. These attack images can fool the …
New attack reveals memorization patterns in pre-trained LLMs.
problem Determining if a data point was part of a pre-trained LLM's training set.
method Adapts MIA statistical tests to LLM's perplexity dynamics of subsequences.
result Significantly outperforms prior approaches in membership inference attacks.
Efficient attacks on DRL models without model access and low computation.
problem Vulnerabilities of DRL models to adversarial attacks.
method Adapting black-box attacks, introducing efficient online sequential attacks, exploring perturbations in environment dynamics, and generating robust physical perturbations.
result Demonstrated the effectiveness of proposed attacks on real-world robots.
Adaptive Bayesian method improves deep learning performance and robustness.
problem Ensuring sparsity in deep learning models to improve performance and robustness.
method Adaptive empirical Bayesian method with self-adaptive spike-and-slab priors and stochastic gradient MCMC optimization.
result The method achieves state-of-the-art performance on MNIST and Fashion MNIST and compression performance on CIFAR10.
Paper tackles adversarial attacks on nonparametric regression models.
problem Vulnerability of machine learning models to adversarial attacks in nonparametric regression.
method Establishes minimax rate and proposes adaptive estimators for robust nonparametric regression under adversarial Lq-risks. result Achieves minimax optimality and provides adaptive estimators for robust nonparametric regression.
New NIDS uses hypergraphs for real-time detection of evolving port scans.
problem Challenges in offline NIDS for auto-generated port scans.
method Hypergraph-based metrics for real-time ML ensemble NIDS.
result High accuracy, precision, and recall in detecting port scans and attacks.
New method bypasses statistical and classifier-based detection of adversarial examples.
problem Vulnerability of deep learning classifiers to adversarial examples.
method Classifier-based adaptation of statistical test method and Logit Mimicry Attack.
result Reduces detection performance to less than 2.2% TPR and 1.6% TPR for statistical test and classifier-based methods, respectively, even at 5% FPR.
TAROT enhances robustness and domain adaptability with domain-invariant features.
problem Developing models robust to adversarial attacks across diverse domains.
method Derives a new generalization bound and proposes TAROT algorithm.
result TAROT outperforms state-of-the-art methods in accuracy and robustness.
Deep neural networks (DNN) are known to be vulnerable to adversarial attacks. Numerous efforts either try to patch weaknesses in trained models, or try to make it difficult or costly to compute adversarial examples that exploit them. In our work, we explore a new "honeypot" approach to protect DNN models. We intentiona…
VarDetect monitors API queries to detect model extraction attacks.
problem Attackers can extract ML models through API queries.
method VarDetect uses a modified variational autoencoder to track and detect attacker samples.
result VarDetect successfully detects and raises alarms for model extraction attacks.
New methods improve adversarial attacks' transferability to other models.
problem Vulnerability of deep learning models to adversarial examples.
method Nesterov Iterative Fast Gradient Sign Method (NI-FGSM) and Scale-Invariant attack Method (SIM).
result NI-FGSM and SIM generate more transferable adversarial examples.
Randomizing model outputs confuses black box adversarial attacks.
problem Defending against black box adversarial attacks in deep neural networks.
method Randomization applied to model outputs to confuse attackers.
result Randomization can bound the probability of introducing errors, thwarting black box attacks.
Convolutional neural networks (CNNs) are known for their good performance and generalization in vision-related tasks and have become state-of-the-art in both application and research-based domains. However, just like other neural network models, they suffer from a susceptibility to noise and adversarial attacks. An adv…
New method improves privacy risk evaluation of machine learning models.
problem Machine learning models can be vulnerable to membership inference attacks.
method Proposed new inference attack method based on prediction entropy, and introduced privacy risk score metric.
result Existing defense approaches are not as effective as previously reported.
New algorithms tackle adversarial multi-player bandits with forced-collision communication.
problem No-sensing adversarial multi-player multi-armed bandits (MP-MAB) problem.
method Adversary-Adaptive Collision-Communication (A2C2) algorithms, attackability-aware and unaware settings, information-theoretic tools, error-correction coding.
result Asymptotic attackability-dependent sublinear regret achieved, with or without knowing attackability.
SmoothLLM defends LLMs from jailbreaking attacks by randomly perturbing inputs.
problem Adversaries can fool large language models into generating objectionable content.
method SmoothLLM randomly perturbs multiple copies of a prompt and aggregates predictions to detect adversarial inputs.
result SmoothLLM sets the state-of-the-art for robustness against various jailbreak attacks.
Graph neural networks are vulnerable to adversarial attacks by manipulating graph structure.
problem Vulnerability of Graph Neural Networks to adversarial attacks.
method Categorization and review of existing attacks and defenses.
result Developed a repository for empirical studies on graph adversarial attacks and defenses.
Advances adversarial training of smoothed classifiers for robust deep learning.
problem Building neural network classifiers robust to adversarial attacks.
method Adversarial training of randomized smoothed classifiers with an adapted attack.
result Significantly outperforms existing provably robust classifiers on ImageNet and CIFAR-10.