Proposes a low-rank PGD attack for more efficient adversarial training.
problem Efficiency and effectiveness of adversarial training methods.
method Variation of PGD to compute low-rank perturbations.
result Low-rank PGD attacks are more efficient and comparable to traditional attacks.
New method speeds up PGD for CV robustness evaluation.
problem Computational inefficiency of PGD for robustness evaluation.
method Early termination of PGD based on cycle detection.
result Large speedup factors without sacrificing robustness.
WITCHcraft improves PGD attacks with random step size, enhancing efficiency.
problem Efficiently crafting adversarial attacks on neural networks.
method Variant of PGD using random step size.
result Superior performance to classical PGD without additional computational cost.
Recent work on adversarial attack and defense suggests that PGD is a universal l∞ first-order attack, and PGD adversarial training can significantly improve network robustness against a wide range of first-order l∞-bounded attacks, represented as the state-of-the-art defense method. However, an obvious …
Anti-correlated noise improves machine learning model generalization.
problem Improving machine learning model generalization.
method Injecting anticorrelated noise into gradient descent steps.
result Anti-correlated noise leads to better model generalization than uncorrelated noise.
SPGD improves adversarial training efficiency and accuracy.
problem Improving adversarial training efficiency and accuracy with fewer steps.
method Adversarial-sample generation from a frequency domain perspective, extending PGD to the frequency domain.
result SPGD achieves greater adversarial accuracy compared to PGD with fewer attack steps.
A new method reduces adversarial training time without overfitting.
problem Catastrophic overfitting in single-step adversarial training.
method FGSMPR: FGSM with PGD Regularization.
result Reduces the gap to multi-step adversarial training.
Recent advances show that deep neural networks are not robust to deliberately crafted adversarial examples which many are generated by adding human imperceptible perturbation to clear input. Consider l2 norms attacks, Project Gradient Descent (PGD) and the Carlini and Wagner (C\&W) attacks are the two main methods, …
A parameter-free PGD algorithm for convex optimization.
problem Minimizing convex functions over convex sets.
method A fully adaptive AdaGrad variant of PGD without parameters or restarts.
result Optimal convergence rates for cumulative regret.
Matrix completion has attracted much interest in the past decade in machine learning and computer vision. For low-rank promotion in matrix completion, the nuclear norm penalty is convenient due to its convexity but has a bias problem. Recently, various algorithms using nonconvex penalties have been proposed, among whic…
PGD algorithms solve nonlinear inverse problems with generative priors using noisy measurements.
problem Signal estimation from noisy nonlinear measurements with generative priors.
method Projected gradient descent algorithms for two cases: unknown and known nonlinearity.
result PGD algorithms converge linearly to optimal statistical rates using arbitrary initialization.
Backdoor attacks are found to be effective against robust machine learning models trained with PGD.
problem Injecting and defending against backdoor attacks in robust machine learning models.
method Study and detection of backdoor attacks on PGD-trained robust models using feature clustering.
result AEGIS effectively detects PGD-trained robust DNNs infected with backdoors with 91.6% accuracy.
New approach improves adversarial robustness without sacrificing natural generalization.
problem Balancing adversarial robustness and natural generalization in machine learning.
method Friendly adversarial training (FAT) using early-stopped PGD to find least adversarial data.
result Early-stopped PGD achieves adversarial robustness without compromising natural generalization.
The paper addresses speckle noise in coherent imaging systems.
problem Speckle noise degrades image quality in coherent imaging systems.
method Theoretical and algorithmic analysis of likelihood-based approaches for multilook coherent imaging.
result Established the first theoretical upper bound on MSE of the maximum likelihood estimator.
We study the problem of finding the best linear model that can minimize least-squares loss given a data-set. While this problem is trivial in the low dimensional regime, it becomes more interesting in high dimensions where the population minimizer is assumed to lie on a manifold such as sparse vectors. We propose proje…
Proves error bounds for PGD, extending log-Sobolev and Talagrand inequalities.
problem Maximum likelihood estimation of large latent variable models.
method Extending log-Sobolev and Talagrand inequalities to models with strongly concave log-likelihoods.
result Non-asymptotic error bounds for PGD in models satisfying LSI and PŁI.
New algorithm eliminates sign function in PGD attacks, improving performance.
problem Improving robustness of neural networks against adversarial attacks.
method Proposes a new raw gradient descent (RGD) algorithm that eliminates the sign function in PGD attacks.
result The RGD algorithm outperforms PGD and other competitors in various settings.
In this work, we evaluate adversarial robustness in the context of transfer learning from a source trained on CIFAR 100 to a target network trained on CIFAR 10. Specifically, we study the effects of using robust optimisation in the source and target networks. This allows us to identify transfer learning strategies unde…
Sequential learning of multiple tasks in artificial neural networks using gradient descent leads to catastrophic forgetting, whereby previously learned knowledge is erased during learning of new, disjoint knowledge. Here, we propose a new approach to sequential learning which leverages the recent discovery of adversari…
Neural networks trained with PGD achieve sharp regression rates in interpolation spaces.
problem Nonparametric regression using over-parameterized neural networks in interpolation spaces.
method Over-parameterized two-layer neural networks trained with Preconditioned Gradient Descent (PGD) and early stopping.
result Achieves a sharp regression rate of \(\cO(n^{-\frac{2αs'}{2αs'+1}})\) in interpolation spaces \(\bth{\cH_K}^{s'}\).
Adversarial testing methods based on Projected Gradient Descent (PGD) are widely used for searching norm-bounded perturbations that cause the inputs of neural networks to be misclassified. This paper takes a deeper look at these methods and explains the effect of different hyperparameters (i.e., optimizer, step size an…
New PnP algorithm converges with relaxed proximal gradient descent.
problem Convergence issues in PnP methods with deep denoisers.
method Relaxed proximal gradient descent for PnP with weakly convex regularization.
result Proposed PnP-αPGD converges for a wider range of regularization parameters. PGD-trained models have a preferential direction in their gradients, which improves robustness.
problem Mathematical lack of clarity in the direction of preferential gradient alignment after adversarial training.
method Proposed a novel definition of preferential direction and evaluated it using a metric based on GANs.
result PGD-trained models have higher alignment with the proposed preferential direction than baseline models.
FGSM is more stable in adversarially robust transfer learning than PGD.
problem Computational efficiency in adversarially robust transfer learning.
method Revisited use of FGSM in adversarial fine-tuning.
result FGSM is more stable and efficient in adversarial fine-tuning.
Paper examines convergence rate of PGD for BP objective in inverse problems.
problem Optimizing ill-posed linear inverse problems using BP vs LS.
method Analysis of PGD convergence rate for BP objective, comparison with proximal gradient method.
result PGD converges faster for BP objective due to inherent properties.
Paper presents faster, robust adversarial training methods.
problem Increasing neural network robustness against adversarial attacks.
method Integrates FGSM with Pixelwise Noise Injection Layer (PNIL) and uniform noise.
result Achieves comparable results to PGD-based adversarial training but faster.
SOAR improves deep networks' robustness against adversarial examples.
problem Improving deep neural networks' robustness against adversarial examples.
method Formulated adversarial robustness problem under robust optimization framework, approximated loss function using second-order Taylor series expansion.
result SOAR significantly improves robustness of networks against adversarial perturbations.
In recent years, deep neural networks have demonstrated outstanding performance in many machine learning tasks. However, researchers have discovered that these state-of-the-art models are vulnerable to adversarial examples: legitimate examples added by small perturbations which are unnoticeable to human eyes. Adversari…
New method improves MMD estimation without convexity assumptions.
problem Lack of theoretical guarantees for MMD estimation algorithms.
method Preconditioned gradient descent (PGD) scheme for MMD optimization.
result PGD scheme converges globally under specific conditions.
Paper assesses adversarial robustness of MCMC and BDK methods for deep Bayesian networks.
problem Assessing adversarial robustness of deep neural networks under MCMC and BDK approximations.
method Characterizes robustness of MCMC and BDK methods to FGSM and PGD attacks.
result Full MCMC-based inference shows excellent robustness, outperforming standard point estimation.
Stochastic LWTA networks resist adversarial attacks while maintaining accuracy.
problem Adversarial robustness of neural networks.
method Replaced ReLU with stochastic LWTA activations, trained with Variational Bayesian and PGD.
result Stochastic LWTA networks achieve state-of-the-art robustness against adversarial attacks.
Adversarial training, in which a network is trained on adversarial examples, is one of the few defenses against adversarial attacks that withstands strong attacks. Unfortunately, the high cost of generating strong adversarial examples makes standard adversarial training impractical on large-scale problems like ImageNet…
Paper improves adversarial training using a learned optimizer.
problem Improving robustness of deep learning models against adversarial attacks.
method Empirically identified PGD attack's limitations and used a learning-to-learn framework to train an adaptive inner optimizer.
result The proposed framework consistently improves model robustness over traditional adversarial training methods.
New method shows adversarial training can be as effective as traditional training without the high cost.
problem The high cost of adversarial training limits its practical application.
method Using FGSM with random initialization and efficient training techniques.
result Adversarial training with FGSM can achieve robust accuracy comparable to PGD-based training at a lower cost.
Recent work on adversarial attack has shown that Projected Gradient Descent (PGD) Adversary is a universal first-order adversary, and the classifier adversarially trained by PGD is robust against a wide range of first-order attacks. It is worth noting that the original objective of an attack/defense model relies on a d…
Paper tackles catastrophic overfitting in single-step adversarial training.
problem Catastrophic overfitting leads to sudden drop in robust accuracy.
method Proposes a method to prevent overfitting by using all adversarial examples.
result Demonstrates prevention of catastrophic overfitting and improves robustness.
Adversarial training can lead to unfair accuracy disparities between different groups.
problem Adversarial training algorithms introduce unfair accuracy disparities between different groups of data.
method Propose a Fair-Robust-Learning (FRL) framework to mitigate unfairness in adversarial defenses.
result Empirical and theoretical validation of FRL's effectiveness in mitigating unfairness.
Improved method reduces projection calls for nonsmooth convex optimization.
problem Optimizing nonsmooth convex functions with convex constraints.
method MOPES and MOLES methods combining Moreau-Yosida smoothing and accelerated first-order schemes.
result Achieves ε-suboptimality with significantly fewer projection calls. MimicGAN improves robustness of image projections under corruption.
problem Robust projection onto image manifolds is challenging due to corruptions.
method Proposes corruption mimicking to robustly project images.
result Significantly more robust than PGD under various corruptions.
CEB enhances model resilience through simple entropy bottleneck.
problem Improving model robustness against adversarial attacks.
method Conditional Entropy Bottleneck (CEB) combined with data augmentation.
result CEB significantly boosts adversarial robustness on various benchmarks.
Paper proposes an ensemble of attacks to evaluate adversarial robustness more reliably.
problem Insufficient evaluation of adversarial defenses leads to incorrect robustness assessments.
method Developed two extensions of PGD-attack and combined them with two complementary attacks.
result Identified several broken defenses with lower robust test accuracy than reported.
Faster reconstruction of compressed signals using conditional GAN and NPGD.
problem Recovering compressed signals from measurements.
method Network-based projected gradient descent (NPGD) combined with measurement-conditional generative adversarial networks (GANs/BEGANs).
result Significant speed-up in reconstruction (up to 140-175 times faster).
PolyGraph Discrepancy improves graph generative model evaluation.
problem Inability of existing metrics to provide an absolute performance measure and comparability across different graph descriptors.
method Approximates Jensen-Shannon distance using binary classifiers trained to distinguish between real and generated graphs.
result PGD provides a more robust and insightful evaluation compared to MMD metrics.
This work examines how adversarial vulnerability changes with the dimensionality of the subspace of perturbations.
problem Understanding adversarial vulnerability in constrained input spaces.
method Investigates adversarial vulnerability in subspace V of the input space X with varying dimensions, using PGD attacks and analyzing the dependence on ε and dim(V)/dim(X). result Adversarial success of PGD attacks is a monotonically increasing function of $ε( rac{dim(V)}{dim(X)})^{ rac{1}{q}}$.
In this paper, we introduce a powerful technique based on Leave-one-out analysis to the study of low-rank matrix completion problems. Using this technique, we develop a general approach for obtaining fine-grained, entrywise bounds for iterative stochastic procedures in the presence of probabilistic dependency. We demon…
Improved neural network robustness to adversarial attacks through smoothed inference.
problem Vulnerability of deep neural networks to adversarial attacks.
method Randomized smoothing applied to adversarial training, improving both robustness and performance.
result Significant improvement in accuracy on adversarial attacks (e.g., 60.4% on CIFAR-10 with ResNet-20, outperforming previous methods by 11.7%).
A new approach to make classifiers safer by allowing them to abstain from making decisions on adversarial inputs.
problem Making machine learning systems robust against adversarial attacks, especially in safety-critical applications.
method Introducing a novel objective function and a simple baseline for adversarial robustness with abstention, followed by CARL (Combined Abstention Robustness Learning) for joint classifier and abstention region learning.
result Training with CARL results in a more accurate, robust, and efficient classifier than a simple baseline.
A Generative Adversarial Network (GAN) with generator G trained to model the prior of images has been shown to perform better than sparsity-based regularizers in ill-posed inverse problems. Here, we propose a new method of deploying a GAN-based prior to solve linear inverse problems using projected gradient descent (…