Paper tackles catastrophic overfitting in single-step adversarial training.
problem Catastrophic overfitting leads to sudden drop in robust accuracy.
method Proposes a method to prevent overfitting by using all adversarial examples.
result Demonstrates prevention of catastrophic overfitting and improves robustness.
New method defends against both single-step and iterative adversarial examples.
problem Defending against adversarial examples, especially iterative ones, is computationally expensive.
method Single-Step Adversarial Training (SST) with modifications.
result Our method outperforms state-of-the-art methods in both accuracy and training time.
A new method reduces adversarial training time without overfitting.
problem Catastrophic overfitting in single-step adversarial training.
method FGSMPR: FGSM with PGD Regularization.
result Reduces the gap to multi-step adversarial training.
Efficient regularization mitigates catastrophic overfitting in single-step adversarial training.
problem Catastrophic overfitting in single-step adversarial training.
method ELLE regularization term to enforce local linearity of the loss function.
result Our regularization term effectively mitigates catastrophic overfitting without the drawbacks of previous methods.
Adversarial examples are malicious inputs designed to fool machine learning models. They often transfer from one model to another, allowing attackers to mount black box attacks without knowledge of the target model's parameters. Adversarial training is the process of explicitly training a model on adversarial examples,…
Lazy neural networks are vulnerable to adversarial attacks.
problem Adversarial robustness issues in lazy training models.
method Extending recent work on adversarial examples to lazy training models.
result Over-parametrized neural networks that generalize well remain vulnerable to single-step gradient ascent attacks.
New method improves adversarial training efficiency and robustness.
problem High computational costs and lack of stability in adversarial training.
method Backward smoothing for randomized smoothing of random initialization.
result Our method achieves similar model robustness as state-of-the-art methods but with significantly less training time.
Adversarial examples are perturbed inputs designed to fool machine learning models. Adversarial training injects such examples into training data to increase robustness. To scale this technique to large datasets, perturbations are crafted using fast single-step methods that maximize a linear approximation of the model'…
Certified training improves robustness against adversarial attacks.
problem Certified training's gap with empirical robustness limits its practical utility.
method Combining adversarial attacks with network over-approximations.
result Certified training can prevent catastrophic overfitting and bridge the gap to multi-step baselines.
Due to the surprisingly good representation power of complex distributions, neural network (NN) classifiers are widely used in many tasks which include natural language processing, computer vision and cyber security. In recent works, people noticed the existence of adversarial examples. These adversarial examples break…
By injecting adversarial examples into training data, adversarial training is promising for improving the robustness of deep learning models. However, most existing adversarial training approaches are based on a specific type of adversarial attack. It may not provide sufficiently representative samples from the adversa…
Unified model improves sampling speed and quality.
problem Difficult to balance sampling speed and quality.
method Multistep Consistency Models combining consistency and diffusion models.
result Improved sampling quality with reduced steps.
Deep neural networks excel at learning the training data, but often provide incorrect and confident predictions when evaluated on slightly different test examples. This includes distribution shifts, outliers, and adversarial examples. To address these issues, we propose Manifold Mixup, a simple regularizer that encoura…
Single gradient step finds adversarial examples in random neural networks.
problem Finding adversarial examples in neural networks with random architectures.
method Gradient descent approach applied to random undercomplete and overcomplete two-layers neural networks.
result A single gradient step is sufficient to find adversarial examples in random neural networks.
The convolutional neural network (CNN) architecture is increasingly being applied to new domains, such as malware detection, where it is able to learn malicious behavior from raw bytes extracted from executables. These architectures reach impressive performance with no feature engineering effort involved, but their rob…
Single-step samplers generate high-quality samples efficiently.
problem Sampling from unnormalized distributions is computationally expensive.
method Developed consistent diffusion samplers that generate samples in a single step.
result Single-step samplers produce high-fidelity samples with less than 1% of traditional samplers' evaluations.
Learning with a primary objective, such as softmax cross entropy for classification and sequence generation, has been the norm for training deep neural networks for years. Although being a widely-adopted approach, using cross entropy as the primary objective exploits mostly the information from the ground-truth class f…
We present an extension of our Molecular Transformer architecture combined with a hyper-graph exploration strategy for automatic retrosynthesis route planning without human intervention. The single-step retrosynthetic model sets a new state of the art for predicting reactants as well as reagents, solvents and catalysts…
New method creates powerful, transferable adversarial attacks for deep learning networks.
problem Vulnerability of deep learning networks to adversarial attacks.
method Developed a method to generate strong, transferable adversarial attacks.
result Demonstrated that the proposed method can easily transfer strong adversarial attacks between different models.
This paper establishes a theoretical foundation for consistency training in diffusion models.
problem Lack of a comprehensive theoretical understanding of consistency training in diffusion models.
method Demonstrates the necessity of a number of steps in consistency learning exceeding d5/2/ε for generating samples within ε proximity to the target distribution. result Establishes rigorous insights into the validity and efficacy of consistency models, offering theoretical underpinnings for their utility.
Paper proposes a new method to speed up diffusion models.
problem High computational cost of sampling from diffusion models.
method Stochastic Runge-Kutta method for acceleration.
result Provable acceleration with reduced score function evaluations.
Enhances SSL with mixup and Lipschitz regularization.
problem Improving SSL performance with limited labeled data.
method Adopts mixup for smoothness, adds adversarial Lipschitz regularization.
result Combined regularization improves SSL performance and robustness.
SDM Policy accelerates inference for robotic tasks while maintaining high action quality.
problem Prolonged inference times in diffusion-based policies for high-frequency control tasks.
method Two-stage optimization: score matching and distribution matching; dual-teacher mechanism.
result 6x inference speedup with state-of-the-art action quality.
Improved chemical reaction prediction using augmented NLP models.
problem Predicting chemical reactions from text representations.
method Data augmentation and Transformer architecture for SMILES representation.
result Significantly improved accuracy in predicting chemical reactions.
CTM improves diffusion model sampling quality with efficient ODE traversal.
problem Lack of natural trade-off between sample quality and speed in consistency models.
method CTM trains a neural network to output scores and traverse ODE trajectories efficiently.
result CTM achieves state-of-the-art FIDs and improves sample quality with increased computational budget.
SPAT improves adversarial robustness by preserving semantics in adversarial training.
problem Adversarial examples often have different semantics than original data, introducing unintended biases.
method Semantics-preserving adversarial training (SPAT) that encourages pixel perturbation shared among all classes.
result SPAT improves adversarial robustness and achieves state-of-the-art results in CIFAR-10 and CIFAR-100.
The paper studies the asymptotic behavior of adversarial training under ℓ∞-perturbation.
problem Theoretical guarantees for sparsity-recovery in adversarial training.
method Investigation of the asymptotic distribution of the adversarial training estimator in generalized linear models.
result The asymptotic distribution of the adversarial training estimator under ℓ∞-perturbation could have a positive probability mass at 0 when the true parameter is 0. PaGoDA reduces diffusion model training costs by 64x.
problem Diffusion models are computationally expensive during training.
method Three-stage pipeline: downsampled training, distillation, progressive super-resolution.
result PaGoDA achieves state-of-the-art performance with reduced training costs.
Paper explores fast adversarial training to improve robustness with less computation.
problem Efficiently defending against adversarial examples.
method Integrates simple self-attacks for faster training, focusing on overfitting recovery.
result Shows superior robust accuracy with reduced training time compared to strong adversarial training.
Adversarial training, in which a network is trained on adversarial examples, is one of the few defenses against adversarial attacks that withstands strong attacks. Unfortunately, the high cost of generating strong adversarial examples makes standard adversarial training impractical on large-scale problems like ImageNet…
Adversarial training, in which a network is trained on both adversarial and clean examples, is one of the most trusted defense methods against adversarial attacks. However, there are three major practical difficulties in implementing and deploying this method - expensive in terms of extra memory and computation costs; …
Adversarial training improves audio classifier robustness.
problem Improving robustness of audio classifiers against adversarial attacks.
method Adversarial training of deep neural networks on audio data.
result Adversarially trained models can resist higher perturbations and fooling rates.
Adversarial training is a training scheme designed to counter adversarial attacks by augmenting the training dataset with adversarial examples. Surprisingly, several studies have observed that loss gradients from adversarially trained DNNs are visually more interpretable than those from standard DNNs. Although this phe…
Adversarial training is one of the strongest defenses against adversarial attacks, but it requires adversarial examples to be generated for every mini-batch during optimization. The expense of producing these examples during training often precludes adversarial training from use on complex image datasets. In this study…
Adversarial samples are perturbed inputs crafted to mislead the machine learning systems. A training mechanism, called adversarial training, which presents adversarial samples along with clean samples has been introduced to learn robust models. In order to scale adversarial training for large datasets, these perturbati…
Adversarial training achieves optimal test error for shallow networks.
problem Achieving optimal adversarial test error for general data distributions.
method Applying new Rademacher complexity bounds and properties of optimal adversarial predictors.
result Adversarial training can achieve optimal adversarial test error for general data distributions.
Free adversarial training reduces the generalization gap compared to vanilla method.
problem Improving generalization in adversarial training.
method Analysis of algorithmic stability in free adversarial training.
result Free adversarial training shows a lower generalization gap.
New method makes neural networks more resilient to location-optimized adversarial patches.
problem Neural networks' vulnerability to adversarial patches that are visible but still effective.
method Developed a practical approach to optimize patch locations and applied adversarial training.
result Significantly improved robustness against adversarial patches on CIFAR10 and GTSRB.
Adversarial training improves graph autoencoder generalization.
problem Improving graph autoencoder generalization.
method Formulated L2 and L1 adversarial training for graph autoencoders and variational graph autoencoders.
result Adversarial training boosts graph autoencoder and variational graph autoencoder generalization.
New approach improves adversarial robustness without sacrificing natural generalization.
problem Balancing adversarial robustness and natural generalization in machine learning.
method Friendly adversarial training (FAT) using early-stopped PGD to find least adversarial data.
result Early-stopped PGD achieves adversarial robustness without compromising natural generalization.
Adversarial robustness has become a central goal in deep learning, both in the theory and the practice. However, successful methods to improve the adversarial robustness (such as adversarial training) greatly hurt generalization performance on the unperturbed data. This could have a major impact on how the adversarial …
This paper improves adversarial robustness of deep learning models.
problem Vulnerability of machine learning models to adversarial perturbations.
method Analyzes adversarial training for linear regression and neural networks, incorporating L1 penalty.
result Incorporating L1 penalty leads to consistent adversarially robust estimation in high-dimensional settings.
Today's state-of-the-art image classifiers fail to correctly classify carefully manipulated adversarial images. In this work, we develop a new, localized adversarial attack that generates adversarial examples by imperceptibly altering the backgrounds of normal images. We first use this attack to highlight the unnecessa…
Study analyzes adversarial training dynamics without data distribution assumptions.
problem Understanding training dynamics of adversarial training without data distribution assumptions.
method Mean field theory approach to analyze adversarial training in random deep neural networks.
result Upper bounds of adversarial loss derived empirically and theoretically.
MAT combines meta-learning and adversarial training to defend against universal patches.
problem Defending against universal patches that fool models in various contexts.
method Meta adversarial training (MAT) integrates meta-learning with adversarial training.
result MAT increases robustness against universal patch attacks on image classification and traffic-light detection.
In recent years, deep neural networks have demonstrated outstanding performance in many machine learning tasks. However, researchers have discovered that these state-of-the-art models are vulnerable to adversarial examples: legitimate examples added by small perturbations which are unnoticeable to human eyes. Adversari…
Injecting adversarial examples during training, known as adversarial training, can improve robustness against one-step attacks, but not for unknown iterative attacks. To address this challenge, we first show iteratively generated adversarial images easily transfer between networks trained with the same strategy. Inspir…
Adversarial training makes logistic regression weight loss landscapes sharper.
problem Understanding why adversarial training sharpens the weight loss landscape in logistic regression.
method Theoretical analysis of linear logistic regression model with L2 norm constraints, and experiments on ResNet18.
result Adversarial training sharpens the weight loss landscape in linear logistic regression models.