Generative GAN improves classifier performance with limited training data.
problem Limited training data for black-box API attacks.
method Generative adversarial network (GAN) to generate synthetic training data.
result Improves classifier performance with synthetic data.
Efficiently distills white-box adversarial attacks into black-box models.
problem Generating efficient adversarial examples for robustness.
method Train a model to emulate white-box attack behavior and distill it into a more efficient black-box model.
result Reduces adversarial example generation time by 19x-39x and transfers to black-box settings.
New method for black-box adversarial attacks using pretrained model embeddings.
problem Efficiently attacking unknown target networks with high-level semantic patterns.
method Learn a low-dimensional embedding using a pretrained model, then search within the embedding space.
result Significant reduction in the number of queries for black-box adversarial attacks.
Efficient black-box evasion attacks against deep networks with fewer queries.
problem Optimizing targeted evasion attacks with limited query budget.
method Formalized problem, evaluated benefits of using substitute models, presented two new attack strategies.
result Attack strategies are as effective as previous techniques but require significantly fewer queries.
Note that this paper is superceded by "Black-Box Adversarial Attacks with Limited Queries and Information." Current neural network-based image classifiers are susceptible to adversarial examples, even in the black-box setting, where the attacker is limited to query access without access to gradients. Previous methods -…
Simple method for black-box adversarial attacks with low query efficiency.
problem Efficient construction of adversarial images in the black-box setting.
method Randomly sampling from an orthonormal basis and adjusting images.
result Demonstrated unprecedented query efficiency in both untargeted and targeted attacks.
Efficiently attacks black-box image classifiers using biased sampling.
problem Efficiently attacking black-box image classifiers without access to confidence scores.
method Reinterpreting Boundary Attacks as a biased sampling framework.
result Combination of biases outperforms state of the art by a wide margin.
Recent studies have highlighted that deep neural networks (DNNs) are vulnerable to adversarial attacks, even in a black-box scenario. However, most of the existing black-box attack algorithms need to make a huge amount of queries to perform attacks, which is not practical in the real world. We note one of the main reas…
Current neural network-based classifiers are susceptible to adversarial examples even in the black-box setting, where the attacker only has query access to the model. In practice, the threat model for real-world systems is often more restrictive than the typical black-box model where the adversary can observe the full …
New defense against DNN model stealing attacks by poisoning predictions.
problem Stealing DNN models through black-box access attacks.
method Active perturbation of predictions to mislead attacker's training.
result Effective against a wide range of DNN stealing attacks, reducing attacker's success rate significantly.
VERA uses variational inference to jailbreak LLMs without manual optimization.
problem Lack of principled objective for gradient-based optimization in jailbreaking LLMs.
method VERA casts black-box jailbreak prompting as a variational inference problem, training a small attacker LLM to approximate the target LLM's posterior over adversarial prompts.
result VERA achieves strong performance across various target LLMs, demonstrating the value of probabilistic inference for adversarial prompt generation.
This work formalizes and solves GNN extraction, a new type of attack.
problem Extracting the underlying model from a black-box GNN with API access.
method Formalizes GNN extraction, presents a solution with preliminary results.
result Preliminary results show the feasibility of GNN extraction.
VarDetect monitors API queries to detect model extraction attacks.
problem Attackers can extract ML models through API queries.
method VarDetect uses a modified variational autoencoder to track and detect attacker samples.
result VarDetect successfully detects and raises alarms for model extraction attacks.
Paper proposes a black-box technique to generate adversarial samples.
problem Robustness of Deep Neural Networks (DNNs) to adversarial samples.
method Black-box Momentum Iterative Fast Gradient Sign Method (BMI-FGSM) using Differential Evolution to approximate gradients.
result Achieves high success rates in generating adversarial samples and misclassification.
Denoised smoothing defends pretrained classifiers against adversarial attacks.
problem Adversarial attacks on pretrained classifiers.
method Prepending a denoiser to any off-the-shelf classifier using randomized smoothing.
result Guaranteed ℓp-robustness to adversarial examples without modifying the pretrained classifier. Machine learning has been applied to a broad range of applications and some of them are available online as application programming interfaces (APIs) with either free (trial) or paid subscriptions. In this paper, we study adversarial machine learning in the form of back-box attacks on online classifier APIs. We start w…
A new adversarial attack method using structured search and contextual bandits.
problem Black-box adversarial attacks on deep learning models.
method Structured search space and Bayesian optimization for contextual bandits.
result Achieves state-of-the-art success rates and query efficiencies.
Marich extracts high-fidelity models from public data with minimal queries.
problem Creating an accurate replica of a target ML model using few queries.
method Sequentially selects informative queries to maximize entropy and reduce model mismatch.
result Extracted models achieve 60-95% of target model's accuracy with 1,000-8,500 queries.
In the past few years, Convolutional Neural Networks (CNNs) have been achieving state-of-the-art performance on a variety of problems. Many companies employ resources and money to generate these models and provide them as an API, therefore it is in their best interest to protect them, i.e., to avoid that someone else c…
Study reveals how machine translation models can leak data privacy.
problem Detecting if a specific data sample was used to train machine translation models.
method Defined membership inference problem for sequence-to-sequence models, tested on machine translation models.
result Machine translation models can leak private information about training data.
Research evaluates model extraction attacks on complex ML models and introduces a defense.
problem Model extraction attacks steal functionality of ML models through prediction APIs.
method Evaluation of Knockoff nets and introduction of a defense.
result Realistic adversaries can effectively steal complex ML models and evade known defenses.
Spanning attack improves black-box attacks with unlabeled data.
problem Query inefficiency in black-box attacks due to high input space dimensionality.
method Proposes spanning attack by constraining adversarial perturbations in a low-dimensional subspace via an auxiliary unlabeled dataset.
result Significantly improves query efficiency of black-box attacks.
Deep learning models misclassify malware with added benign features.
problem Detecting malware with deep learning when it's mixed with benign code.
method Trained a deep neural network classifier using benign and malware features. Demonstrated the impact of adding benign features to malware. Used data augmentation to improve classifier robustness.
result Adding benign features to malware significantly increases false negatives.
Machine learning (ML) models may be deemed confidential due to their sensitive training data, commercial value, or use in security applications. Increasingly often, confidential ML models are being deployed with publicly accessible query interfaces. ML-as-a-service ("predictive analytics") systems are an example: Some …
NP-Attack reduces query counts for black-box adversarial attacks.
problem Efficiency of black-box adversarial attacks is low.
method Uses Neural Process to characterize image structure and find adversarial examples.
result NP-Attack significantly decreases query counts.
ODS improves adversarial attacks by maximizing output diversity.
problem Efficiency and effectiveness of adversarial attacks, especially black-box attacks.
method Output Diversified Sampling (ODS) that maximizes diversity in model outputs.
result ODS reduces the number of queries needed for black-box attacks on ImageNet by a factor of two.
New method improves black-box attacks using pre-trained models.
problem Efficiently attacking black-box models with limited information.
method EigenBA algorithm leveraging pre-trained white-box model's Jacobian matrix.
result Optimal perturbations are related to right singular vectors of Jacobian matrix.
Paper proposes a new black-box attack approach to minimize visual distortion.
problem Constructing adversarial examples that minimize visual distortion in a black-box threat model.
method Learning the noise distribution of adversarial examples to approximate the gradient of a non-differentiable loss function.
result The proposed attack results in much lower visual distortion compared to state-of-the-art black-box attacks.
Paper proposes ZO-NGD for more efficient black-box attacks.
problem Vulnerability of state-of-the-art DNNs to adversarial attacks.
method Zeroth-order natural gradient descent (ZO-NGD) for black-box attacks.
result ZO-NGD achieves significantly lower model query complexities.
Efficient method for generating adversarial examples with limited query budget.
problem Developing black-box adversarial attacks with limited information.
method Bayesian Optimization in a structured low-dimensional subspace.
result Significantly higher attack success rate with fewer queries.
Randomizing model outputs confuses black box adversarial attacks.
problem Defending against black box adversarial attacks in deep neural networks.
method Randomization applied to model outputs to confuse attackers.
result Randomization can bound the probability of introducing errors, thwarting black box attacks.
P-BO reduces black-box adversarial attacks by 10x with Bayesian optimization and function prior.
problem Efficiently generating adversarial examples against black-box models.
method Prior-guided Bayesian Optimization (P-BO) with a function prior initialized from a surrogate model.
result Significantly reduces the number of queries needed for adversarial attacks.
Paper proposes a new black-box adversarial attack using normalizing flows.
problem Adversarial vulnerability of deep neural networks.
method Proposes a novel black-box adversarial attack using normalizing flows.
result Demonstrates competitive performance against well-known black-box adversarial attack methods.
New research shows many recent defenses against adversarial examples are ineffective against black-box attacks.
problem The robustness of recent defenses against adversarial examples is insufficient, especially against black-box attacks.
method Evaluation of nine defenses on two black-box adversarial models and six attacks on CIFAR-10 and Fashion-MNIST datasets.
result Most recent defenses provide only marginal improvements in security (<25%) compared to undefended networks. New approach generates black-box attacks on neural networks using swarm evolutionary algorithm.
problem Adversarial samples attack neural networks.
method Swarm evolutionary algorithm for black-box attacks.
result 100% success rate on MNIST and CIFAR-10 images.
AdvMind detects adversary intent in black-box attacks with high accuracy.
problem Detecting adversary intent in black-box adversarial attacks is challenging.
method AdvMind accounts for adversary adaptiveness and synthesizes queries to expose intent.
result AdvMind detects adversary intent with over 75% accuracy after observing less than 3 query batches.
We use Bayesian optimization to create efficient adversarial attacks with limited queries.
problem Creating adversarial examples with limited query access.
method Bayesian optimization for query-efficient adversarial attacks.
result Our method reduces query count by up to 80% compared to state-of-the-art methods.
New black-box attack method improves GNN defense without needing training data.
problem Vulnerability of Graph Neural Networks to adversarial attacks.
method Developed a gradient-based black-box attack algorithm, BBGA, which does not require access to training data.
result BBGA achieves stable attack performance without accessing training sets, and is effective against various defenses.
New method defends machine learning models from black-box attacks.
problem Protecting machine learning models from black-box transfer attacks.
method Randomized ensemble technique with provable security guarantee.
result Empirical evidence of security for random binary classifiers.
Depending on how much information an adversary can access to, adversarial attacks can be classified as white-box attack and black-box attack. For white-box attack, optimization-based attack algorithms such as projected gradient descent (PGD) can achieve relatively high attack success rates within moderate iterates. How…
Study examines adversarial robustness of ANN variants, revealing differences in black-box settings.
problem Adversarial robustness of alternative neural network architectures.
method Analysis of conventional, stochastic ANNs, and SNNs across three datasets; experiments in white-box and black-box settings.
result Stochastic ANNs are more robust than conventional ANNs in black-box settings, especially with surrogate attacks.
A simple patch copying method reduces black-box adversarial attack queries by 81%.
problem The effectiveness of black-box adversarial attacks depends on the initialization method.
method Copying small patches from other images as initialization points.
result Reduces the number of queries required for a state-of-the-art Boundary Attack by 81%
V-BAD is a first black-box video attack framework that successfully fools deep video recognition models.
problem Vulnerability of video recognition models to black-box adversarial attacks.
method Tentative perturbations transferred from image models and partition-based rectifications for patches of tentative perturbations.
result V-BAD can craft both untargeted and targeted attacks with high success rates using a manageable number of queries.
New attacks reduce bad queries in black-box classifiers, improving effectiveness.
problem Evasion attacks often issue many bad queries, which are costly.
method Design new attacks that minimize bad queries while increasing non-bad queries.
result Reduces the number of bad queries by 1.5-7.3 times, improving attack effectiveness.
Paper proposes a black-box adversarial attack method for graph embedding models.
problem Robustness of graph embedding models against adversarial attacks.
method GF-Attack constructs a generalized adversarial attacker by the graph filter and feature matrix, performing the attack on the graph filter in a black-box fashion.
result GF-Attack can consistently make strong attacks on different graph embedding models even with small perturbations.
This paper surveys black-box attacks on computer vision models.
problem Real-life adversarial attacks on deep learning models.
method Comprehensive comparative study of attacks and defenses.
result Significant amount of research on reliability of models.
Square Attack efficiently attacks deep models with random updates.
problem Efficiently attacking deep learning models without gradient information.
method Randomized search of localized square updates.
result Significantly more query-efficient and higher success rate compared to state-of-the-art methods.
Recent studies have shown that adversarial examples in state-of-the-art image classifiers trained by deep neural networks (DNN) can be easily generated when the target model is transparent to an attacker, known as the white-box setting. However, when attacking a deployed machine learning service, one can only acquire t…