Aggregation defenses improve deep learning models' robustness against data poisoning attacks.
problem Data poisoning attacks manipulate deep learning models with malicious training samples.
method Deep Partition Aggregation, efficiency improvements, data-to-complexity ratio, poisoning overfitting phenomenon.
result Aggregation defenses boost poisoning robustness through the poisoning overfitting phenomenon.
Improved defense against data poisoning attacks by aggregating smaller subsets.
problem Mitigating the impact of poisoned data on model robustness.
method Finite Aggregation method that combines duplicates of smaller disjoint subsets for training.
result Consistent improvement in certified robustness bounds, up to 4.77% on GTSRB.
Deep Partition Aggregation defends against poisoning attacks with provable certificates.
problem Adversarial poisoning attacks corrupt classifier test-time behavior.
method Deep Partition Aggregation (DPA) is an ensemble method using hash partitions and base models trained on these partitions.
result DPA can certify >= 50% of test images against over 500 poison image insertions on MNIST, and nine insertions on CIFAR-10.
Dynamic defense against Byzantine poisoning in federated learning.
problem Byzantine poisoning attacks on federated learning models.
method Dynamic federated aggregation operator to filter out adversarial clients.
result Dynamic selection of clients improves global learning model performance.
RESTA defends LLMs against jailbreaking attacks by adding random noise to embeddings.
problem Vulnerability of LLMs to jailbreaking attacks that generate harmful outputs.
method Adds random noise to embedding vectors and aggregates during token generation.
result RESTA achieves superior robustness versus utility tradeoffs compared to baseline defenses.
In this paper, we address a problem of machine learning system vulnerability to adversarial attacks. We propose and investigate a Key based Diversified Aggregation (KDA) mechanism as a defense strategy. The KDA assumes that the attacker (i) knows the architecture of classifier and the used defense strategy, (ii) has an…
A new method aggregates generative classifiers to resist adversarial attacks.
problem Adversarial attacks on deep neural networks.
method Rank-aggregating ensemble of generative classifiers trained on intermediate layer responses.
result The ensemble of generative classifiers shows robustness to adversarial attacks.
FDA3 defends IIoT applications against adversarial attacks by federating defense knowledge.
problem Adversarial attacks on DNNs in IIoT applications can cause devastating consequences.
method Federated learning approach to aggregate defense knowledge from different sources.
result FDA3 can resist more attacks than existing methods and prevent new attacks.
The Lethal Dose Conjecture limits how much poisoned data can be tolerated.
problem Limiting the impact of poisoned data in machine learning models.
method Theoretical analysis and provable defenses (DPA, FA) based on majority voting.
result DPA and FA are asymptotically optimal defenses against data poisoning.
PatchGuard defends against localized adversarial patches with provable robustness.
problem Localized adversarial patches induce misclassification in machine learning models.
method PatchGuard uses CNNs with small receptive fields and robust masking to detect and mask corrupted features.
result PatchGuard achieves state-of-the-art provable robust accuracy and clean accuracy.
Federated learning is vulnerable to backdoor attacks; a new defense method is proposed.
problem Backdoor attacks in federated learning that can misclassify models.
method Adjusting the learning rate based on sign information of agents' updates.
result Our defense significantly reduces or eliminates backdoor attacks in federated learning.
Graph neural network (GNN), as a powerful representation learning model on graph data, attracts much attention across various disciplines. However, recent studies show that GNN is vulnerable to adversarial attacks. How to make GNN more robust? What are the key vulnerabilities in GNN? How to address the vulnerabilities …
New aggregation method improves GNN robustness to structural perturbations.
problem Graph Neural Networks (GNNs) are vulnerable to adversarial attacks that manipulate graph structure.
method Proposes a robust aggregation function with a breakdown point of 0.5, inspired by robust statistics.
result Improves GNN robustness by a factor of 3 on Cora ML and 5.5 on Citeseer, and 8 for low-degree nodes.
New attacks and defenses for GNNs on large graphs.
problem Vulnerability of GNNs to adversarial attacks on large graphs.
method Proposed two sparsity-aware first-order optimization attacks and a robust aggregation function.
result Attacks can double in strength, and defenses are effective at all scales.
New risk factors improve stress testing accuracy.
problem Improving stress testing accuracy with new risk factors.
method Adapted PCA and autoencoders for dimension reduction and interpretation.
result Aggregated risk factors enhance stress testing outcomes.
Recently, new defense techniques have been developed to tolerate Byzantine failures for distributed machine learning. The Byzantine model captures workers that behave arbitrarily, including malicious and compromised workers. In this paper, we break two prevailing Byzantine-tolerant techniques. Specifically we show robu…
Simple aggregation of multiple methods defends against adversarial attacks on neural networks.
problem Manipulation of machine learning models to hide discriminating features.
method Aggregation of multiple explanation methods to make the model robust against adversarial attacks.
result The model remains robust even when the attacker has knowledge of the model weights and explanation methods.
Federated learning is a recently proposed paradigm that enables multiple clients to collaboratively train a joint model. It allows clients to train models locally, and leverages the parameter server to generate a global model by aggregating the locally submitted gradient updates at each round. Although the incentive mo…
Recent studies have highlighted that deep neural networks (DNNs) are vulnerable to adversarial examples. In this paper, we improve the robustness of DNNs by utilizing techniques of Distance Metric Learning. Specifically, we incorporate Triplet Loss, one of the most popular Distance Metric Learning methods, into the fra…
Robust algorithm for distributed optimization resistant to Byzantine failures.
problem Resilient optimization in the presence of unreliable agents.
method Temporal and spatial robust aggregation, gradient normalization.
result Convergence for strongly convex and non-convex functions.
Paper defends sensitive attributes in GNNs from inference attacks.
problem Protecting sensitive attributes in GNNs from inference attacks.
method Proposes adversarial training with TV and Wasserstein distance to locally filter sensitive attributes.
result Framework creates strong defense against inference attacks with minimal performance loss.
FLANDERS detects and blocks extreme model poisoning in federated learning.
problem Resilience against large-scale model poisoning attacks in federated learning.
method FLANDERS treats client updates as matrix-valued time series and identifies outliers using autoregressive forecasting.
result FLANDERS significantly improves robustness in federated learning across various attacks.
New defense method inspired by encryption improves visual classification accuracy.
problem Conventional defenses reduce accuracy and are defeated by obfuscated gradients.
method Block-wise pixel shuffling with secret key for training and test images.
result Achieves high accuracy (91.55%) on clean images and (89.66%) on adversarial examples.
Automated discovery of adaptive attacks improves adversarial defense evaluation.
problem Challenges in reliably evaluating adversarial defenses.
method Formalizes adaptive attacks as reusable building blocks in a search space for automatic discovery.
result Our tool discovers significantly stronger attacks than AutoAttack, improving adversarial defense evaluation.
Unified framework analyzes privacy risks from gradients in distributed learning.
problem Analyzing inference privacy risks from gradients in machine learning.
method Unified game-based framework for various attacks, including attribute, property, distributional, and user disclosures.
result Demonstrates inefficacy of data aggregation for privacy against inference attacks.
New research shows many recent defenses against adversarial examples are ineffective against black-box attacks.
problem The robustness of recent defenses against adversarial examples is insufficient, especially against black-box attacks.
method Evaluation of nine defenses on two black-box adversarial models and six attacks on CIFAR-10 and Fashion-MNIST datasets.
result Most recent defenses provide only marginal improvements in security (<25%) compared to undefended networks. SmoothLLM defends LLMs from jailbreaking attacks by randomly perturbing inputs.
problem Adversaries can fool large language models into generating objectionable content.
method SmoothLLM randomly perturbs multiple copies of a prompt and aggregates predictions to detect adversarial inputs.
result SmoothLLM sets the state-of-the-art for robustness against various jailbreak attacks.
Heterophily affects GNN robustness; separating ego- and neighbor-embeddings improves defense.
problem The robustness of GNNs to adversarial attacks.
method Formalized relation between heterophily and GNN robustness; empirical analysis; design principles for improved robustness.
result Separating ego- and neighbor-embeddings increases GNN robustness.
Backdoor attacks can poison federated meta-learning models, even with few examples.
problem Backdoor attacks on federated meta-learning models.
method Proposed a defense mechanism inspired by matching networks.
result Backdoor attacks can be very successful and persist after additional training in federated meta-learning.
Paper analyzes adversarial attacks and defenses using game theory.
problem Unclear conditions for optimal attacks and defenses in adversarial learning.
method Game-theoretic framework with locally linear decision boundary model.
result Fast Gradient Method attack and Randomized Smoothing defense form a Nash Equilibrium.
Paper presents certified defenses against adversarial patch attacks.
problem Certified defenses against adversarial patch attacks are needed.
method Proposes the first certified defense and faster training methods.
result Demonstrates robustness transfer across different patch shapes.
There has been extensive research on developing defense techniques against adversarial attacks; however, they have been mainly designed for specific model families or application domains, therefore, they cannot be easily extended. Based on the design philosophy of ensemble of diverse weak defenses, we propose ATHENA---…
This study evaluates adversarial attacks and defenses for chest X-ray disease classification.
problem Vulnerability of deep neural networks to adversarial examples in chest X-ray disease detection.
method Detailed introduction and evaluation of various attack and defense methods.
result Attack and defense methods perform poorly with excessive iterations and large perturbations.
Stochastic defense improves natural classifiers against adversarial attacks.
problem Vulnerability of deep networks to adversarial attacks.
method Long-run MCMC sampling with Energy-Based Model for adversarial purification.
result Balancing memoryless and metastable behavior leads to effective purification and robust classification.
Defense against small image patches using occlusions.
problem Vulnerability of deep learning to small adversarial patches.
method Partially occlude image around each patch location.
result Certified security against patch attacks of a certain size.
Deep neural networks (DNNs) are known vulnerable to adversarial attacks. That is, adversarial examples, obtained by adding delicately crafted distortions onto original legal inputs, can mislead a DNN to classify them as any target labels. This work provides a solution to hardening DNNs under adversarial attacks through…
Defense against ASR attacks using dropout uncertainty.
problem Adversarial attacks on ASR systems.
method Dropout uncertainty in neural networks.
result High detection accuracy across various ASR systems and datasets.
Survey of algorithms to correct past mistakes in prediction.
problem Improving prediction accuracy by correcting past errors.
method Defensive Forecasting as a sequential game theory approach to minimize prediction metrics.
result Simple, near-optimal algorithms for various prediction tasks.
We propose a novel defense against all existing gradient based adversarial attacks on deep neural networks for image classification problems. Our defense is based on a combination of deep neural networks and simple image transformations. While straightforward in implementation, this defense yields a unique security pro…
Machine-learning (ML) algorithms or models, especially deep neural networks (DNNs), have shown significant promise in several areas. However, researchers have recently demonstrated that ML algorithms, especially DNNs, are vulnerable to adversarial examples (slightly perturbed samples that cause misclassification). The …
High-performance Deep Neural Networks (DNNs) are increasingly deployed in many real-world applications e.g., cloud prediction APIs. Recent advances in model functionality stealing attacks via black-box access (i.e., inputs in, predictions out) threaten the business model of such applications, which require a lot of tim…
This paper benchmarks time-series adversarial defenses and attacks.
problem Adversarial attacks on time-series data.
method Detailed benchmarking of adversarial defense methods in the L∞ threat model. result Adversarial defenses offer robustness against both strong white-box and black-box attacks.
Data poisoning attacks -- where an adversary can modify a small fraction of training data, with the goal of forcing the trained classifier to high loss -- are an important threat for machine learning in many applications. While a body of prior work has developed attacks and defenses, there is not much general understan…
Correctly evaluating defenses against adversarial examples has proven to be extremely difficult. Despite the significant amount of recent work attempting to design defenses that withstand adaptive attacks, few have succeeded; most papers that propose defenses are quickly shown to be incorrect. We believe a large contri…
DeepRobust is a PyTorch library for adversarial attacks and defenses.
problem Addressing adversarial attacks and defenses in deep learning.
method Contains over 20 attack and defense algorithms for various deep learning architectures.
result Fosters research in adversarial learning with comprehensive and easy-to-use platform.
New defense method reduces adversarial attacks on neural networks.
problem Vulnerability of deep neural networks to adversarial attacks.
method Sparse coding frontend trained on clean images.
result Significantly attenuates adversarial attacks before reaching classifier.
Temporal threat model defends against data poisoning with timestamps.
problem Adversaries can poison more samples than expected, rendering existing defenses ineffective.
method Leverage timestamps to define earliness and duration metrics for temporal robustness.
result Temporal aggregation provides provable temporal robustness against data poisoning.
It has been shown that adversaries can craft example inputs to neural networks which are similar to legitimate inputs but have been created to purposely cause the neural network to misclassify the input. These adversarial examples are crafted, for example, by calculating gradients of a carefully defined loss function w…