This paper analyzes output activation functions for adversarial losses.
problem Understanding which output activation functions form a well-behaved adversarial loss.
method Variational divergence minimization and a comparative framework for adversarial losses.
result There is no single winning combination of output activation functions and regularization approaches across all settings.
The paper sets lower bounds for adversarial robustness in multiclass classification.
problem Adversarial robustness in multiclass classification with arbitrary loss functions.
method Dual and barycentric reformulations for robust risk minimization.
result Sharp lower bounds for adversarial risks are computed efficiently.
Theoretical analysis of cross-entropy loss functions and their robustness.
problem Guarantees for using cross-entropy as a surrogate loss function.
method Theoretical analysis of a broad family of loss functions, including cross-entropy.
result First H H H -consistency bounds for comp-sum losses and smooth adversarial comp-sum losses. New algorithms achieve no-regret learning even with adversarial transitions and losses.
problem No-regret learning impossible with adversarial transitions and losses.
method Developed algorithms for adversarial Markov Decision Processes with smooth regret increase.
result Achieved O ~ ( T + C e x t s f P ) \widetilde{O}(\sqrt{T} + C^{ extsf{P}}) O ( T + C e x t s f P ) regret, with C e x t s f P C^{ extsf{P}} C e x t s f P measuring adversarial transition function. Proposes new loss functions for GANs to improve estimation accuracy and robustness.
problem Improving the training of GANs to achieve more accurate and robust models.
method Introduces Hellinger-type loss functions and analyzes their statistical properties.
result Demonstrates improved estimation accuracy and robustness of the proposed loss functions.
Smoothness analysis of adversarial training reveals L ∞ L_\infty L ∞ constraints cause more non-smoothness.
problem Non-smoothness of adversarial training loss function.
method Analyzed the smoothness of adversarial training loss function using optimal attacks for model parameters.
result The L ∞ L_\infty L ∞ constraint causes more non-smoothness than L 2 L_2 L 2 constraint. This paper analyzes adversarial attacks methods and their effectiveness.
problem Understanding the effectiveness and theoretical properties of adversarial attacks.
method Comparative and formal analysis of loss functions of three adversarial attack methods.
result The Iterative Fast Gradient Sign attack is the slowest in creating adversarial examples.
New algorithm ATENT improves adversarial robustness in neural networks.
problem Improving neural network robustness against adversarial attacks.
method Proposes a new loss function with entropic regularization for training robust neural networks.
result ATENT achieves competitive robust classification accuracy on benchmark datasets.
AWP improves robustness by flattening weight loss landscape.
problem Improving robustness of deep neural networks against adversarial examples.
method Explicitly regularizes the flatness of weight loss landscape through adversarial weight perturbation.
result AWP forms a double-perturbation mechanism in adversarial training, leading to flatter weight loss landscape.
Analyzes adversarial training's impact on loss landscape, proposing PAS to improve model performance.
problem Challenges in optimizing models under adversarial training due to loss landscape properties.
method Analytical studies of adversarial loss functions, numerical analyses, PAS strategy.
result Adversarial training impairs optimization, but PAS strategy improves model performance.
Paper analyzes adversarial risk using optimal transport.
problem Poor performance of machine learning on adversarial data.
method Optimal transport perspective, optimal transport plans (couplings), convexity, smoothness assumptions.
result Fundamental limits on adversarial risk calculated for various datasets.
Generative Adversarial Privacy (GAP) learns privacy mechanisms from data.
problem Learning optimal privacy mechanisms from data.
method Formulates privacy as a constrained minimax game between privatizer and adversary.
result GAP provides privacy guarantees against strong adversaries.
Study non-asymptotic bounds for robust estimators under misspecified models.
problem Evaluate performance of robust estimators under adversarial conditions.
method Propose a general approach to adversarial risk analysis, including investigations on generalization and approximation errors.
result Establish non-asymptotic upper bounds for adversarial excess risk under Lipschitz loss functions.
Ensemble of models with uncorrelated loss functions improves adversarial robustness.
problem Adversarial attacks on deep neural networks.
method Diversity Training: training an ensemble of models with uncorrelated loss functions.
result Our method significantly improves adversarial robustness of ensembles.
This paper characterizes and designs loss functions for robust classification with abstention.
problem Ensuring robustness against adversarial attacks and knowing when to abstain from prediction.
method Proposes adversarial robust reject option loss and characterizes surrogates for calibration.
result Shifted Double Ramp Loss and Shifted Double Sigmoid Loss satisfy the calibration conditions.
We study minimax convergence rates of nonparametric density estimation under a large class of loss functions called "adversarial losses", which, besides classical L p \mathcal{L}^p L p losses, includes maximum mean discrepancy (MMD), Wasserstein distance, and total variation distance. These losses are closely related to the …
The paper proposes effective margin regularization to improve adversarial robustness in deep neural networks.
problem Adversarial vulnerability of deep neural networks (DNNs).
method Regularization of effective weight norm during training to maximize effective margins.
result Effective margin regularization (EMR) boosts adversarial robustness in both standard and adversarial training.
New training scheme reduces adversarial examples by increasing decision boundary margin.
problem Vulnerability of neural networks to adversarial examples.
method Differential training using a loss function on feature differences.
result Differential training significantly reduces adversarial examples.
This study connects Jacobian regularization to adversarial robustness and improves generalization.
problem Adversarial attacks make deep neural networks vulnerable.
method Developed a connection between Jacobian regularization and adversarial training, and established robust generalization gaps.
result Jacobian norms are related to both standard and robust generalization.
New approach combines likelihood and adversarial losses for better precipitation predictions.
problem Spatially inconsistent precipitation projections from likelihood-based models.
method Fuses likelihood-based and adversarial losses for generative models.
result Improves spatial consistency in precipitation downscaling.
New loss function reduces adversarial examples by controlling Fisher information matrix eigenvalues.
problem Defending against adversarial attacks in neural networks.
method Adding a term to the loss function representing the trace of the Fisher information matrix.
result Effective and robust defensive capability, reducing adversarial example fooling ratio.
Study on calibration and consistency of adversarial surrogate losses.
problem Designing robust classifiers with theoretical guarantees.
method Extensive analysis of H-calibration and H-consistency of adversarial surrogate losses.
result Some convex loss functions and supremum-based convex losses are not H-calibrated for important hypothesis sets.
Our work proves convergence to low robust training loss for polynomial width ReLU networks.
problem Understanding why adversarial training leads to low robust training loss in over-parameterized neural nets.
method Extending convergence theory for standard supervised training to adversarial training, using tools from online learning and showing ReLU networks can approximate the step function.
result Convergence to low robust training loss for polynomial width ReLU networks under natural assumptions.
This research finds that using mean-squared error and codeword targets improves adversarial robustness.
problem Evaluating and improving the robustness of neural networks against adversarial attacks.
method Training neural networks on mean-squared error and using codeword targets as representations.
result The modified models show up to 98.7% increase in accuracy against untargeted attacks and up to 99.8% decrease in targeted attack success rates.
Optimistic algorithm reduces regret and constraint violations in online convex optimization with adversarial constraints.
problem Online convex optimization with adversarial constraints.
method Improved algorithm using accurate predictions of loss and constraint functions.
result Improved bounds on regret and cumulative constraint violations.
New expressive losses improve adversarial robustness without sacrificing accuracy.
problem Training networks for robustness at the expense of accuracy.
method Formalizing expressivity, using convex combinations of adversarial attacks and IBP bounds.
result Trivial expressive losses yield state-of-the-art results in various settings.
A new algorithm tackles adversarial linear contextual bandits using kernelized loss functions.
problem Online learning in adversarial linear contextual bandits with flexible loss functions.
method Proposes a computationally efficient algorithm using an optimistically biased estimator for reproducing kernel Hilbert space loss functions.
result Achieves near-optimal regret guarantees under polynomial and exponential eigendecay assumptions.
New loss function improves neural network robustness.
problem Improving neural network robustness to adversarial attacks.
method Combining energy landscape techniques with a novel loss function.
result The proposed loss function leads to more robust neural networks without significant degradation in classification error.
EdgeFool generates adversarial images to mislead classifiers.
problem Misleading classifiers with adversarial images.
method Trains a fully convolutional neural network to generate perturbations that enhance image details and mislead classifiers.
result EdgeFool outperforms other adversarial methods on various classifiers and datasets.
New GAN loss functions improve image generation quality and stability.
problem Improving the performance of GANs in generating high-quality images.
method Introducing least kth-order GAN (L k k k GAN) and Rényi-centric GAN loss functions. result The proposed loss functions lead to better image quality and stability.
Proposes SOVR loss to improve adversarial robustness by increasing logit margins.
problem Adversarial training's difficulty in robustness against sophisticated attacks.
method Introduces SOVR loss function that switches from cross-entropy to one-vs-the-rest loss for important samples.
result SOVR loss increases logit margins of important samples, improving robustness against Auto-Attack.
Study improves H H H -consistency bounds for regression analysis.
problem Improving H H H -consistency bounds for regression analysis. method Generalized theorems and novel H H H -consistency bounds for various surrogate loss functions. result Derives principled surrogate losses for adversarial regression.
01 loss is robust to outliers and adversarial attacks, outperforming convex losses.
problem Improving robustness in binary classification models.
method Developed stochastic coordinate descent algorithms for 01 loss classifiers and neural networks.
result 01 loss classifiers are more robust to adversarial attacks compared to convex losses.
Framework to mitigate adversarial attacks by allowing classifiers to abstain.
problem Vulnerability of classifiers to adversarial examples.
method Adversarial training with a rejection option.
result ATRO framework improves classifier reliability against adversarial attacks.
Adversarial training makes logistic regression weight loss landscapes sharper.
problem Understanding why adversarial training sharpens the weight loss landscape in logistic regression.
method Theoretical analysis of linear logistic regression model with L2 norm constraints, and experiments on ResNet18.
result Adversarial training sharpens the weight loss landscape in linear logistic regression models.
New proof shows faster convergence rate for robust estimation with Lasso in adversarially contaminated outputs.
problem Robust estimation of parameters in the presence of adversarial output contamination.
method Extended Lasso with Huber loss function and L 1 L_1 L 1 penalty, focusing on specific properties of the Huber function. result Same convergence rate as Dalalyan and Thompson (2019), but with a different proof.
GANs can generate realistic data without minimizing a divergence, contrary to current theory.
problem Current theory suggests GANs minimize a divergence to generate realistic data.
method Discussed various loss functions for G, showing they are not divergences and do not have the same equilibrium.
result GANs can use a wide range of loss functions, not just divergences, to generate realistic data.
Adversarial consistency depends on the uniqueness of adversarial Bayes classifiers.
problem Consistency of adversarial surrogate losses is not guaranteed.
method Connected consistency of adversarial surrogate losses to the uniqueness of adversarial Bayes classifiers.
result A convex surrogate loss is statistically consistent for adversarial learning if and only if the adversarial Bayes classifier is unique.
Optimal bounds on regret and constraint violation in adversarial COCO.
problem Minimizing regret and cumulative constraint violation in adversarial COCO.
method New surrogate loss function and Follow-the-Regularized-Leader/Online Gradient Descent.
result Achieved optimal O ( T ) O(\sqrt{T}) O ( T ) bounds on both regret and cumulative constraint violation. This research analyzes the consistency of convex and nonconvex surrogate losses for adversarially robust classification.
problem Ensuring classifiers are robust to adversarial perturbations.
method Analysis of convex and nonconvex surrogate losses through the lens of calibration.
result No convex surrogate loss is calibrated with respect to the adversarial 0-1 loss for linear models, but nonconvex losses can be calibrated under certain conditions.
A new adversarial attack method using structured search and contextual bandits.
problem Black-box adversarial attacks on deep learning models.
method Structured search space and Bayesian optimization for contextual bandits.
result Achieves state-of-the-art success rates and query efficiencies.
New loss function improves adversarial robustness without sacrificing standard accuracy.
problem Adversarial robustness requires more samples than standard accuracy, and new data collection is costly.
method Proposed Max-Mahalanobis center (MMC) loss to concentrate feature points in the feature space.
result Empirical results show MMC loss significantly improves robustness under strong adaptive attacks.
We analyze adversarial learning using a minimax approach.
problem Analyzing risk bounds in adversarial learning problems.
method Reducing adversarial learning to a minimax statistical learning problem and proving a new risk bound.
result Deriving adversarial risk bounds for SVMs, deep neural networks, and PCA.
Paper proposes AoA, a new adversarial attack method that significantly increases transferability.
problem Developing universal adversarial attacks on deep neural networks.
method AoA attacks the semantic property of attention in DNNs, enhancing transferability by altering the loss function.
result AoA generates adversarial samples that defeat 13 well-trained DNNs, achieving over 85% error rate.
Improved regret bound for adversarial MDPs with linear function approximation.
problem Learning in adversarial MDPs with changing loss functions and large state spaces.
method Two algorithms: refined FTRL with log-barrier regularizer and magnitude-reduced loss estimator.
result Achieved i l d e O ( K ) ilde{\mathcal O}(\sqrt K) i l d e O ( K ) regret, improving over i l d e O ( K 2 / 3 ) ilde{\mathcal O}(K^{2/3}) i l d e O ( K 2/3 ) . In recent years, the Word2Vec model trained with the Negative Sampling loss function has shown state-of-the-art results in a number of machine learning tasks, including language modeling tasks, such as word analogy and word similarity, and in recommendation tasks, through Prod2Vec, an extension that applies to modeling…
This work simplifies adversarial attacks using neural networks, reducing computation and improving training convergence.
problem Efficiently generating and training against ideal adversarial attacks with minimal computational overhead.
method Representing ideal adversarial attacks as smooth piece-wise functions and approximating them with neural networks. Using a mathematical game between an attack network and a defense network for adversarial training.
result Obtained convergence rates of adversarial loss in terms of sample size n n n for adversarial training. 3D GAN improves MRI image quality from low-res scans.
problem Improving MRI image quality from low-resolution scans.
method Adversarial learning using 3D convolutions and least squares loss.
result 3D GAN generates high-quality 3D MRI images from low-resolution scans.