New method defends machine learning models from black-box attacks.
problem Protecting machine learning models from black-box transfer attacks.
method Randomized ensemble technique with provable security guarantee.
result Empirical evidence of security for random binary classifiers.
Interpretable semi-supervised classifier for black-box models with two self-labeling strategies.
problem Lack of labeled data and difficulty in explaining black-box models.
method Combines black-box and white-box approaches for self-labeling and prediction.
result Superior prediction rates and interpretability compared to state-of-the-art classifiers.
Paper presents mdfa to identify victims of discrimination in black box classifiers.
problem Identifying victims of discrimination in black box classifiers.
method Reduces discrimination measurement to matching distributions and sensitive attribute coincidence prediction.
result Identifies African-American individuals at high risk of violent recidivism.
New attacks reduce bad queries in black-box classifiers, improving effectiveness.
problem Evasion attacks often issue many bad queries, which are costly.
method Design new attacks that minimize bad queries while increasing non-bad queries.
result Reduces the number of bad queries by 1.5-7.3 times, improving attack effectiveness.
Interpretable companion model for black-box classifiers.
problem Dilemma between interpretable and black-box models.
method Trains a companion model from data and black-box model predictions, optimizing a combination of accuracy and complexity.
result Companion model provides interpretable predictions with a slight accuracy loss for user choice.
Develops causal explanations for black-box classifiers.
problem Creating understandable explanations for complex models.
method Generative model with information-theoretic measures of causal influence.
result Learned latent factors influence classifier outputs.
Paper proposes a new black-box adversarial attack using normalizing flows.
problem Adversarial vulnerability of deep neural networks.
method Proposes a novel black-box adversarial attack using normalizing flows.
result Demonstrates competitive performance against well-known black-box adversarial attack methods.
Current neural network-based classifiers are susceptible to adversarial examples even in the black-box setting, where the attacker only has query access to the model. In practice, the threat model for real-world systems is often more restrictive than the typical black-box model where the adversary can observe the full …
This paper finds universal perturbations to fool black-box ML classifiers.
problem Breaking security through obscurity in black-box ML settings.
method Zeroth-order optimization for finding universal adversarial perturbations in a black-box setting.
result State-of-the-art ML classifiers can be fooled with a single imperceptible image perturbation.
Bayesian model explains and improves black-box estimators for class distribution.
problem Calibrating probabilistic classifiers and uncertainty quantification for unlabeled data.
method Introduced a Bayesian model approximating the ground-truth generative process, using efficient MCMC sampling.
result The Bayesian model is competitive and sometimes superior to established point estimators.
Paper introduces active Bayesian method for assessing black-box classifiers efficiently.
problem Need to assess performance of black-box classifiers reliably with limited labels.
method Develops inference strategies and proposes active Bayesian framework for efficient instance selection.
result Significant gains in performance assessment with fewer labels compared to traditional methods.
New method wraps black-box classifiers to reduce bias.
problem Reduction of bias in black-box predictions.
method Post-processing with α-trees and boosting algorithms.
result Demonstrated effectiveness in reducing bias across multiple fairness metrics.
This paper explores how the generalization of substitute classifiers affects the success of black-box adversarial attacks.
problem Understanding the factors driving the transferability of black-box adversarial examples.
method Max-min adversarial example game framework and theoretical generalization bounds.
result Substitute NN with better generalization behavior results in more transferable adversarial examples.
A simple patch copying method reduces black-box adversarial attack queries by 81%.
problem The effectiveness of black-box adversarial attacks depends on the initialization method.
method Copying small patches from other images as initialization points.
result Reduces the number of queries required for a state-of-the-art Boundary Attack by 81%
CLE explains any classifier predictions in plain terms.
problem Understanding complex classifier predictions.
method Model-agnostic explanation technique approximating locally using interpretable models.
result CLE provides faithful and interpretable explanations for various classifiers.
AdvFlow generates imperceptible adversarial images using normalizing flows.
problem Adversarial attacks on deep learning classifiers.
method Normalizing flows to model adversarial examples.
result Generated adversaries closely follow clean data distribution, making them harder to detect.
New research shows many recent defenses against adversarial examples are ineffective against black-box attacks.
problem The robustness of recent defenses against adversarial examples is insufficient, especially against black-box attacks.
method Evaluation of nine defenses on two black-box adversarial models and six attacks on CIFAR-10 and Fashion-MNIST datasets.
result Most recent defenses provide only marginal improvements in security (<25%) compared to undefended networks. This work proposes xGEMs or manifold guided exemplars, a framework to understand black-box classifier behavior by exploring the landscape of the underlying data manifold as data points cross decision boundaries. To do so, we train an unsupervised implicit generative model -- treated as a proxy to the data manifold. We …
ManiGen generates adversarial examples without classifier knowledge.
problem Vulnerability of neural network classifiers to adversarial examples.
method Generates adversarial examples by searching along the manifold.
result Adversarial examples generated by ManiGen are as successful as state-of-the-art generators.
Proposes method for global explanations of credit risk models.
problem Lack of interpretability in credit risk scoring models.
method Sampling decision function to learn interpretable models.
result Unified solution to approximate complex decision boundaries.
Method makes non-interpretable models more intervenable.
problem Making non-interpretable models more understandable and controllable.
method Intervenability formalization and fine-tuning of black-box models.
result Fine-tuned black-box models are more intervenable and often better-calibrated.
REST improves robustness of black-box models to geometric transformations.
problem Overconfident incorrect predictions on out-of-distribution samples.
method REinforcement Spatial Transform learner (REST) that transforms input data into in-distribution samples.
result Improves robustness to geometric transformations and sample efficiency.
Efficiently attacks black-box image classifiers using biased sampling.
problem Efficiently attacking black-box image classifiers without access to confidence scores.
method Reinterpreting Boundary Attacks as a biased sampling framework.
result Combination of biases outperforms state of the art by a wide margin.
Paper proposes a new Hessian-aware zeroth-order optimization for improving black-box adversarial attacks.
problem Improving black-box adversarial attacks on neural networks.
method Introduces a Hessian-aware zeroth-order optimization algorithm called ZO-HessAware.
result ZO-HessAware achieves improved success rates with lower query complexity.
P-BO reduces black-box adversarial attacks by 10x with Bayesian optimization and function prior.
problem Efficiently generating adversarial examples against black-box models.
method Prior-guided Bayesian Optimization (P-BO) with a function prior initialized from a surrogate model.
result Significantly reduces the number of queries needed for adversarial attacks.
Study malware evasion attacks and defenses using ML classifiers.
problem Adversarial examples can mislead ML-based malware detectors.
method Investigated white-box and grey-box evasion attacks; compared defense approaches.
result Proposed a framework for grey-box and black-box attacks.
Note that this paper is superceded by "Black-Box Adversarial Attacks with Limited Queries and Information." Current neural network-based image classifiers are susceptible to adversarial examples, even in the black-box setting, where the attacker is limited to query access without access to gradients. Previous methods -…
Predicting structured outputs can be computationally onerous due to the combinatorially large output spaces. In this paper, we focus on reducing the prediction time of a trained black-box structured classifier without losing accuracy. To do so, we train a speedup classifier that learns to mimic a black-box classifier u…
Study adversarial perturbations in classification, analyzing learning and certification.
problem Formal study of classification under adversarial perturbations from both learner and third-party perspectives.
method PAC-type semi-supervised learning framework, black-box certification under limited query budget, adversary analysis.
result Existence of a polynomial query complexity adversary implies the existence of a sample efficient robust learner.
Efficient method reduces black-box adversarial queries for neural networks.
problem Solving for adversarial examples in black-box settings with limited query access.
method Efficient combinatorial optimization surrogate for gradient estimation.
result State-of-the-art black-box attack performance with reduced query count.
Adapts example weights to optimize black-box metrics.
problem Optimizing metrics defined by black-box functions.
method Adaptive example weighting and iterative post-shifting.
result Improves classification performance compared to baselines.
A new Frank-Wolfe framework improves efficiency and effectiveness of adversarial attacks.
problem Develop efficient and effective optimization-based adversarial attack algorithms.
method Proposes a Frank-Wolfe algorithm variant for both white-box and black-box adversarial attacks.
result Demonstrates improved efficiency and effectiveness compared to existing methods.
DEceit constructs effective universal pixel-restricted perturbations for deep image classifiers.
problem Creating effective universal pixel-restricted perturbations for deep neural networks.
method DEceit algorithm for black-box feedback, targeting 10% of pixels in images.
result Perturbing only 10% of pixels achieves high Fooling Rate and visual similarity.
Faced with distribution shift between training and test set, we wish to detect and quantify the shift, and to correct our classifiers without test set labels. Motivated by medical diagnosis, where diseases (targets) cause symptoms (observations), we focus on label shift, where the label marginal p(y) changes but the …
Recent work in model-agnostic explanations of black-box machine learning has demonstrated that interpretability of complex models does not have to come at the cost of accuracy or model flexibility. However, it is not clear what kind of explanations, such as linear models, decision trees, and rule lists, are the appropr…
We study black-box attacks on machine learning classifiers where each query to the model incurs some cost or risk of detection to the adversary. We focus explicitly on minimizing the number of queries as a major objective. Specifically, we consider the problem of attacking machine learning classifiers subject to a budg…
Paper shows LIME and SHAP explanations can be fooled by adversarial attacks.
problem Reliability of post hoc explanation methods like LIME and SHAP.
method Adversarial attack technique to hide classifier biases.
result Popular explanation techniques can be fooled into generating biased explanations.
Deep-PrAE improves rare-event simulation for black-box systems.
problem Evaluating rare safety-critical events in learning-based systems.
method Combines deep neural networks with IS to create statistically guaranteed estimations.
result Deep-PrAE provides accurate bounds on safety-critical event probabilities.
Invertible networks help explain decisions and identify important features.
problem Interpreting and explaining the decisions of black-box neural networks.
method Two-stage approach: invertible transformation to feature space and linear classifier. Determining decision boundaries and feature importance using local linear models.
result Ability to explain decisions and identify important features in neural networks.
Recent studies have shown that adversarial examples in state-of-the-art image classifiers trained by deep neural networks (DNN) can be easily generated when the target model is transparent to an attacker, known as the white-box setting. However, when attacking a deployed machine learning service, one can only acquire t…
Paper generates natural adversarial examples for hyperspectral data.
problem Creating adversarial examples for black-box models.
method Modified Wasserstein GAN reweights true data distribution.
result Successfully generates adversarial hyperspectral signatures.
New method shows unfair explanations can be created for any classifier.
problem Fairness in black-box classifier explanations is not guaranteed.
method Constructs a new classifier with similar performance but altered explanations.
result Explanation methods can be manipulated to show unfair decisions.
A new algorithm for optimizing huge-scale black-box problems with reduced memory usage.
problem Optimizing huge-scale black-box problems with limited vector operations.
method ZO-BCD algorithm for zeroth-order optimization with reduced memory footprint.
result ZO-BCD achieves state-of-the-art adversarial attack success rate of 97.9%.
Proposes a method to ensure accurate estimation of rare events in AI systems.
problem Lack of efficiency guarantees in black-box systems for rare-event simulation.
method Integrates deep learning with importance sampling to create a statistically guaranteed estimator.
result Demonstrates effective estimation of rare-event probabilities in AI systems.
Paper presents strong black-box attacks on unsupervised clustering models.
problem Adversarial attacks on unsupervised machine learning models.
method Iterative black-box attack on clustering algorithms.
result Adversarial samples can fool clustering algorithms irrespective of the algorithm used.
GAMIN attacks black-box models to reveal training data, achieving high success rates.
problem Exposing sensitive information in machine learning models trained on private data.
method Generative adversarial model training to generate inputs similar to training data.
result GAMIN can extract recognizable digits and features from complex models like CNNs.
Survey on making machine learning models more understandable.
problem Making predictions from machine learning models transparent and understandable for humans.
method Conducts a survey on explainable supervised machine learning approaches.
result Classifies explainable SML approaches and discusses future directions.
Paper proposes a black-box adversarial attack method for graph embedding models.
problem Robustness of graph embedding models against adversarial attacks.
method GF-Attack constructs a generalized adversarial attacker by the graph filter and feature matrix, performing the attack on the graph filter in a black-box fashion.
result GF-Attack can consistently make strong attacks on different graph embedding models even with small perturbations.