Study on robustness in linear regression models, focusing on adversarial perturbations.
problem Understanding and improving robustness in linear regression models to adversarial perturbations.
method Developed a two-stage adversarial learning framework that incorporates model structure information.
result Proved the consistency and developed the Bahadur representation of the adversarially robust estimator.
This paper investigates how network width and depth affect adversarially robust DNNs.
problem Understanding architectural configurations for adversarially robust DNNs.
method Comprehensive investigation on the impact of network width and depth on adversarial robustness.
result Optimal architectural configuration for adversarial robustness exists and can improve robustness.
Boosts barely robust learners to be more adversarially robust.
problem Learning predictors robust to small perturbations on a small fraction of data.
method Oracle-efficient algorithm for robustness with larger perturbation set.
result Qualitative and quantitative equivalence between strongly robust and barely robust learning.
This study connects Jacobian regularization to adversarial robustness and improves generalization.
problem Adversarial attacks make deep neural networks vulnerable.
method Developed a connection between Jacobian regularization and adversarial training, and established robust generalization gaps.
result Jacobian norms are related to both standard and robust generalization.
This work explores how neural architecture search can improve adversarial robustness without adversarial training.
problem Improving adversarial robustness of neural networks without adversarial training.
method Experimented with hand-crafted and NAS-based architectures to compare robustness to PGD attacks.
result NAS-based architectures are more robust for small-scale attacks, but hand-crafted architectures are more robust for larger datasets and tasks.
SOAR improves deep networks' robustness against adversarial examples.
problem Improving deep neural networks' robustness against adversarial examples.
method Formulated adversarial robustness problem under robust optimization framework, approximated loss function using second-order Taylor series expansion.
result SOAR significantly improves robustness of networks against adversarial perturbations.
RATIO improves neural network robustness and explainability.
problem Neural networks' lack of robustness to adversarial changes and uncertainty on out-distribution samples.
method RATIO: Adversarial Training on In- and Out-distribution.
result RATIO leads to robust models with reliable confidence estimates on out-distribution samples.
Adversarial robustness has become a central goal in deep learning, both in the theory and the practice. However, successful methods to improve the adversarial robustness (such as adversarial training) greatly hurt generalization performance on the unperturbed data. This could have a major impact on how the adversarial …
CNN layers with large norms are still robust to adversarial attacks.
problem Understanding the relationship between layer norms and adversarial robustness in CNNs.
method Theoretical analysis of ℓ1 and ℓ∞ norms, norm decay method, adversarial training frameworks. result Adversarially robust CNNs can have comparable or larger layer norms than non-adversarially robust ones.
(Non-)robustness of neural networks to small, adversarial pixel-wise perturbations, and as more recently shown, to even random spatial transformations (e.g., translations, rotations) entreats both theoretical and empirical understanding. Spatial robustness to random translations and rotations is commonly attained via e…
This work introduces adversarial sparsity to measure robustness beyond adversarial accuracy.
problem Evaluating robustness to adversarial attacks beyond just accuracy.
method Adversarial sparsity, which quantifies the difficulty of finding perturbations.
result Sparsity provides valuable insights into neural networks and suggests improvements in robustness.
Robust RL improves controller robustness to dynamics variations using adversarial populations.
problem Robustness issues in RL when dynamics are perturbed.
method Adversarial population augmentation to the Robust RL formulation.
result Population-based adversarial approach yields more robust and generalizable policies.
Study reveals adversarially robust domain adaptation is harder to generalize across domains.
problem Hardness of transferring adversarial robustness across different domains.
method Analysis of adversarial Rademacher complexity over symmetric difference hypothesis space.
result Adversarial Rademacher complexity is always greater than non-adversarial, indicating intrinsic hardness.
SPAT improves adversarial robustness by preserving semantics in adversarial training.
problem Adversarial examples often have different semantics than original data, introducing unintended biases.
method Semantics-preserving adversarial training (SPAT) that encourages pixel perturbation shared among all classes.
result SPAT improves adversarial robustness and achieves state-of-the-art results in CIFAR-10 and CIFAR-100.
Previous work on adversarially robust neural networks for image classification requires large training sets and computationally expensive training procedures. On the other hand, few-shot learning methods are highly vulnerable to adversarial examples. The goal of our work is to produce networks which both perform well a…
RTFE provides adversarial robustness to multiple models.
problem Adversarial examples can transfer to other models, compromising robustness.
method Proposes RTFE, a deep learning-based pre-processing mechanism.
result RTFE provides adversarial robustness to multiple independently trained classifiers.
Robust RL with learned optimal adversary improves agent performance under adversarial state observations.
problem Ensuring reinforcement learning agents' robustness against adversarial perturbations of state observations.
method Proposed a framework of alternating training with learned adversaries (ATLA) to find optimal adversarial policies and enhance agent robustness.
result ATLA achieves state-of-the-art performance under strong adversaries in continuous control environments.
Paper develops robust Bayesian models for linear regression under adversarial perturbations.
problem Ensuring reliable machine learning models under data perturbations.
method Formulates adversarial Bregman divergence loss, computes adversarial perturbation, introduces adversarially robust posteriors, derives generalization certificates.
result Derives first rigorous generalization certificates for adversarially robust Bayesian linear regression.
Obtaining deep networks that are robust against adversarial examples and generalize well is an open problem. A recent hypothesis even states that both robust and accurate models are impossible, i.e., adversarial robustness and generalization are conflicting goals. In an effort to clarify the relationship between robust…
Neural Networks have been shown to be sensitive to common perturbations such as blur, Gaussian noise, rotations, etc. They are also vulnerable to some artificial malicious corruptions called adversarial examples. The adversarial examples study has recently become very popular and it sometimes even reduces the term "adv…
One popular hypothesis of neural network generalization is that the flat local minima of loss surface in parameter space leads to good generalization. However, we demonstrate that loss surface in parameter space has no obvious relationship with generalization, especially under adversarial settings. Through visualizing …
BN helps learn fragile features, which can improve adversarial robustness.
problem The role of batch normalization in adversarial training and its impact on robustness.
method Investigated the expressiveness of BN in learning robust features compared to random features.
result Adversarially fine-tuning BN layers can result in non-trivial adversarial robustness.
Paper proposes a method to train robust neural networks without labeled data.
problem Training robust neural networks without class labels.
method Adversarial contrastive learning framework using unlabeled data.
result Robust Contrastive Learning (RoCL) achieves comparable robust accuracy to supervised methods and significantly improved robustness.
New method enhances neural network robustness against adversarial attacks.
problem Enhancing neural network robustness against adversarial attacks.
method Variational framework with per-sample noise level selector.
result Enhanced empirical robustness and certified robustness.
New algorithm improves robustness by more regularization on less robust samples.
problem Improving robustness against adversarial attacks.
method Proposes a new adversarial training algorithm that applies more regularization to data vulnerable to adversarial attacks.
result Empirically superior to other existing algorithms, achieving state-of-the-art performance in generalization and robustness.
Given the apparent difficulty of learning models that are robust to adversarial perturbations, we propose tackling the simpler problem of developing adversarially robust features. Specifically, given a dataset and metric of interest, the goal is to return a function (or multiple functions) that 1) is robust to adversar…
Adversarially trained transformers can learn robustly across tasks with minimal tuning.
problem Adversarial attacks and the high cost of adversarial training.
method Adversarial pretraining followed by in-context learning.
result Single-layer linear transformers can generalize robustly to unseen tasks.
Study shows prior Lipschitz continuity can improve adversarial robustness of Bayesian Neural Networks.
problem Improving adversarial robustness of Bayesian Neural Networks.
method Analysis of i.i.d., zero-mean Gaussian priors and posteriors approximated via mean-field variational inference.
result Adversarial robustness is sensitive to the prior variance.
New mechanisms from primate vision improve neural network robustness.
problem Demonstrating robust neural networks to small adversarial perturbations.
method Investigated two biologically plausible mechanisms: non-uniform retina sampling and receptive field diversity.
result Non-uniform retina sampling and receptive field diversity improve adversarial robustness.
Knowledge distillation is effective for producing small, high-performance neural networks for classification, but these small networks are vulnerable to adversarial attacks. This paper studies how adversarial robustness transfers from teacher to student during knowledge distillation. We find that a large amount of robu…
This work proposes SDI regularization to improve adversarial robustness.
problem Improving adversarial robustness of deep neural networks.
method SDI regularization term to complement adversarial training.
result Combining SDI with AT variants enhances robustness and generalization.
Adversarial training improves robustness of halfspaces in noisy data.
problem Learning robust halfspaces in the presence of label noise.
method Adversarial training with binary cross-entropy or nonconvex sigmoidal loss.
result Adversarial training yields robust halfspaces with improved classification error.
ORAT improves model robustness against outliers and adversarial attacks.
problem Challenges of training data like outliers and adversarial samples.
method Bi-level optimization with robust rank-based loss function.
result ORAT achieves theoretical consistency and uniform convergence rates.
New research shows flat minima in robust loss landscapes correlate with good adversarial robustness.
problem Adversarial training leads to robust overfitting, poor robust generalization.
method Average- and worst-case metrics to measure flatness in robust loss landscapes.
result Flatness in robust loss landscapes correlates with good adversarial robustness.
In recent years, neural networks have demonstrated outstanding effectiveness in a large amount of applications.However, recent works have shown that neural networks are susceptible to adversarial examples, indicating possible flaws intrinsic to the network structures. To address this problem and improve the robustness …
Proposes using mode connectivity to improve adversarial robustness of neural networks.
problem Improving adversarial robustness of deep neural networks.
method Employing mode connectivity in loss landscapes to study adversarial robustness and propose methods for improvement.
result Path connection learned using limited bonafide data can effectively mitigate adversarial effects while maintaining original accuracy.
Paper tackles robustness in adversarial noise with a meta-optimizer.
problem Sensitivity to adversarial noise hinders machine learning deployment.
method Meta-optimizer learns to robustly optimize models using adversarial examples.
result Meta-optimizer transfers adversarial knowledge to new models without generating new examples.
Adaptive networks improve model robustness through conditional normalization.
problem Limited robustness of adversarial-trained networks due to network capacity and training samples.
method Proposes a conditional normalization module to adapt networks during adversarial training.
result Adaptive networks outperform both clean validation accuracy and robustness compared to non-adaptive counterparts.
This work improves certifiably robust models by distilling knowledge from adversarially robust teachers.
problem Certifiably robust models suffer from poor standard performance.
method Knowledge distillation from adversarially robust teachers to improve standard performance.
result Distillation from adversarially robust teachers consistently improves certified training performance.
New algorithm ATENT improves adversarial robustness in neural networks.
problem Improving neural network robustness against adversarial attacks.
method Proposes a new loss function with entropic regularization for training robust neural networks.
result ATENT achieves competitive robust classification accuracy on benchmark datasets.
Adversarial training can lead to unfair accuracy disparities between different groups.
problem Adversarial training algorithms introduce unfair accuracy disparities between different groups of data.
method Propose a Fair-Robust-Learning (FRL) framework to mitigate unfairness in adversarial defenses.
result Empirical and theoretical validation of FRL's effectiveness in mitigating unfairness.
ClusTR improves clustering-based models' robustness without adversarial training.
problem Improving clustering-based models' robustness.
method Proposes ClusTR, a clustering-based training framework for robust models without adversarial training.
result ClusTR outperforms adversarially-trained models by up to 4% under strong PGD attacks.
A new method reduces adversarial training time without overfitting.
problem Catastrophic overfitting in single-step adversarial training.
method FGSMPR: FGSM with PGD Regularization.
result Reduces the gap to multi-step adversarial training.
Study on conditions for Bayes optimal classifier in adversarial robustness.
problem Existence of Bayes optimal classifier for adversarial robustness.
method General sufficient conditions for existence of Bayes optimal classifier.
result Guaranteed existence of Bayes optimal classifier under certain conditions.
Proposes a new adversarial model to avoid accuracy vs. adversarial accuracy tradeoff.
problem Inherent tradeoff between accuracy and adversarial accuracy in existing adversarial robustness definitions.
method Introduces Voronoi-epsilon adversary that balances perturbation constraints.
result Voronoi-epsilon adversary avoids accuracy vs. adversarial accuracy tradeoff even with large ε. Adversarial attacks and defenses are currently active areas of research for the deep learning community. A recent review paper divided the defense approaches into three categories; gradient masking, robust optimization, and adversarial example detection. We divide gradient masking and robust optimization differently: (…
Adversarial robustness improved by sparsity in network weights.
problem How sparsity affects adversarial robustness in neural networks.
method Theoretical proof and experimental validation of adversarial pruning methods.
result Weights sparsity improves adversarial robustness, especially through inheritance from smaller networks.
Improves model calibration by addressing adversarially unrobust inputs.
problem Miscalibrated predictions and lack of adversarial robustness in neural networks.
method AR-AdaLS, an adaptive label smoothing method that considers adversarial robustness.
result AR-AdaLS improves model calibration, even under distributional shifts.