Fast Gradient Sign Method (FGSM) is a popular method to generate adversarial examples that make neural network models robust against perturbations. Despite its empirical success, its theoretical property is not well understood. This paper develops theory to explain the regularization effect of Generalized FGSM, a class…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “Fast Gradient Sign Method”
Efficiently performs robust and sparse kernel regression.
problem Robust and sparse kernel regression.
method Sign gradient descent and early stopping.
result Sign gradient descent achieves robust and sparse kernel regression efficiently.
This paper analyzes adversarial attacks methods and their effectiveness.
problem Understanding the effectiveness and theoretical properties of adversarial attacks.
method Comparative and formal analysis of loss functions of three adversarial attack methods.
result The Iterative Fast Gradient Sign attack is the slowest in creating adversarial examples.
DIP-FAT improves adversarial training by diversifying perturbations.
problem Adversarial examples fool deep neural networks, leading to overfitting and poor performance.
method DIP-FAT uses random directions to diversify perturbations in adversarial training.
result DIP-FAT reduces overfitting and improves clean data accuracy.
A new method reduces adversarial training time without overfitting.
problem Catastrophic overfitting in single-step adversarial training.
method FGSMPR: FGSM with PGD Regularization.
result Reduces the gap to multi-step adversarial training.
New method shows adversarial training can be as effective as traditional training without the high cost.
problem The high cost of adversarial training limits its practical application.
method Using FGSM with random initialization and efficient training techniques.
result Adversarial training with FGSM can achieve robust accuracy comparable to PGD-based training at a lower cost.
A new metric learning framework for signed graphs using Gershgorin disc alignment.
problem Learning Mahalanobis metrics from signed graphs efficiently.
method Proposes a fast metric learning framework using Gershgorin disc perfect alignment (GDPA) to circumvent full eigen-decomposition.
result Proves that Gershgorin disc left-ends of similarity transform are perfectly aligned at the smallest eigenvalue, enabling efficient optimization.
RFML systems are vulnerable to adversarial attacks, especially in OTA transmissions.
problem Vulnerability of RFML systems to adversarial machine learning attacks.
method Differentiated adversarial threats, developed methodology for evaluating vulnerabilities, used Fast Gradient Sign Method.
result RFML systems are vulnerable to adversarial examples, even in OTA attacks.
Machine Learning models are vulnerable to adversarial attacks that rely on perturbing the input data. This work proposes a novel strategy using Autoencoder Deep Neural Networks to defend a machine learning model against two gradient-based attacks: The Fast Gradient Sign attack and Fast Gradient attack. First we use an …
In recent years, neural networks have demonstrated outstanding effectiveness in a large amount of applications.However, recent works have shown that neural networks are susceptible to adversarial examples, indicating possible flaws intrinsic to the network structures. To address this problem and improve the robustness …
Paper tackles catastrophic overfitting in single-step adversarial training.
problem Catastrophic overfitting leads to sudden drop in robust accuracy.
method Proposes a method to prevent overfitting by using all adversarial examples.
result Demonstrates prevention of catastrophic overfitting and improves robustness.
Unified signSGD and gradient descent analysis for neural networks.
problem Performance of sign-based optimization methods in neural networks.
method Unified analysis of separable smoothness and -smoothness, isolating geometric properties affecting performance.
result Sign-based methods are preferable over gradient descent under specific Hessian properties in deep networks.
The signed volume function for polyhedra can be generalized to a mean volume function for volume elements by averaging over the triangulations of the underlying polyhedron. If we consider these up to translation and scaling, the resulting quotient space is diffeomorphic to a sphere. The mean volume function restricted …
Directional Kernel Mean Difference: A Fast Signed Statistic for Univariate Distribution Comparisonstat.ML
DKMD is a fast signed statistic for comparing univariate distributions.
problem Comparing univariate distributions, especially preserving directionality.
method DKMD integrates kernel mean embeddings against an odd weighting function.
result DKMD preserves directionality and is robust to outliers.
Unified sign-based compression for federated learning with faster convergence.
problem High communication cost in federated learning with large-scale models.
method Unified noisy perturbation scheme for sign-based compression.
result Achieves faster convergence rate than existing sign-based methods.
Deep Neural Networks are built to generalize outside of training set in mind by using techniques such as regularization, early stopping and dropout. But considerations to make them more resilient to adversarial examples are rarely taken. As deep neural networks become more prevalent in mission-critical and real-time sy…
Paper assesses adversarial robustness of MCMC and BDK methods for deep Bayesian networks.
problem Assessing adversarial robustness of deep neural networks under MCMC and BDK approximations.
method Characterizes robustness of MCMC and BDK methods to FGSM and PGD attacks.
result Full MCMC-based inference shows excellent robustness, outperforming standard point estimation.
Minimax defense improves neural network security against gradient-based attacks.
problem Gradient-based adversarial attacks on neural networks.
method Minimax optimization in a GAN framework to create a discriminator that plays a minimax game with the generator.
result Minimax defense significantly reduces adversarial attack success rates compared to standard classifiers.
New method prevents adversarial training failure in deep networks.
problem Adversarial training failure in deep networks.
method GradAlign method to prevent catastrophic overfitting.
result GradAlign prevents adversarial training failure in deep networks.
Study improves kernel quadrature for infinitely wide models with faster approximation and estimation rates.
problem Efficiently approximating and estimating expectations in infinitely wide models.
method Developed general kernel quadrature (GKQ) for parameter distributions, achieving faster rates.
result Achieved a fast approximation rate of and a fast estimation rate of .
In this paper, we analyze efficacy of the fast gradient sign method (FGSM) and the Carlini-Wagner's L2 (CW-L2) attack. We prove that, within a certain regime, the untargeted FGSM can fool any convolutional neural nets (CNNs) with ReLU activation; the targeted FGSM can mislead any CNNs with ReLU activation to classify a…
New method approximates neural network training for robustness.
problem Training robust neural networks with adversarial input perturbations.
method Stochastic convex optimization approach to adversarial training.
result Method achieves better adversarial robustness and performance.
The study examines the dynamic behavior of RMSprop and Adam algorithms.
problem Understanding the training loss curve patterns of adaptive gradient algorithms.
method Careful numerical experiments and theoretical explanations using the signGD flow.
result Adam converges smoother and faster when momentum factors are close to each other.
signADAM++ uses confidence functions to train deep neural networks faster.
problem Training deep neural networks efficiently and effectively.
method Integrates sign operation into ADAM and introduces a confidence function.
result signADAM++ generates more sparse gradients and improves performance.
Recent successes in word embedding and document embedding have motivated researchers to explore similar representations for networks and to use such representations for tasks such as edge prediction, node label prediction, and community detection. Such network embedding methods are largely focused on finding distribute…
Paper solves k-sparse parity problem with sign SGD, matching SQ lower bound.
problem Solving k-sparse parity problems efficiently.
method Sign stochastic gradient descent on neural networks.
result Matches Statistical Query lower bound for solving k-sparse parity problems.
This paper explores the use of adversarial examples in training speech recognition systems to increase robustness of deep neural network acoustic models. During training, the fast gradient sign method is used to generate adversarial examples augmenting the original training data. Different from conventional data augmen…
Develops method for learning signed graphs from smooth signals.
problem Learning signed graphs from observed data, especially in contexts with both positive and negative interactions.
method Uses net Laplacian as graph shift operator and minimizes total variation of observed signals with ADMM.
result Theoretical proofs of convergence and estimation error bound provided.
New algorithm eliminates sign function in PGD attacks, improving performance.
problem Improving robustness of neural networks against adversarial attacks.
method Proposes a new raw gradient descent (RGD) algorithm that eliminates the sign function in PGD attacks.
result The RGD algorithm outperforms PGD and other competitors in various settings.
New methods improve adversarial attacks' transferability to other models.
problem Vulnerability of deep learning models to adversarial examples.
method Nesterov Iterative Fast Gradient Sign Method (NI-FGSM) and Scale-Invariant attack Method (SIM).
result NI-FGSM and SIM generate more transferable adversarial examples.
ODE guided Neural Data Augmentation Techniques for Time Series Data and its Benefits on Robustnesscs.LG
Paper introduces techniques to enhance time series data robustness against adversarial attacks.
problem Vulnerability of deep learning time series classifiers to adversarial attacks.
method Two local gradient based and one spectral density based data augmentation techniques.
result Trained models with augmented data achieve state-of-the-art classification accuracy and robustness.
We improve the robustness of Deep Neural Net (DNN) to adversarial attacks by using an interpolating function as the output activation. This data-dependent activation remarkably improves both the generalization and robustness of DNN. In the CIFAR10 benchmark, we raise the robust accuracy of the adversarially trained Res…
The ADAM optimizer is exceedingly popular in the deep learning community. Often it works very well, sometimes it doesn't. Why? We interpret ADAM as a combination of two aspects: for each weight, the update direction is determined by the sign of stochastic gradients, whereas the update magnitude is determined by an esti…
Adversarial perturbations fool deepfake detectors with high accuracy.
problem Improving deepfake detection accuracy against adversarial attacks.
method Used adversarial perturbations and two defenses: Lipschitz regularization and Deep Image Prior (DIP).
result Deepfake detectors achieved 27% accuracy on perturbed images, compared to 95% on unperturbed.
New tests for high-dimensional data improve on existing methods.
problem Testing mean vectors in high-dimensional data.
method Generalized multivariate sign transformation, using different norm functions.
result Tests using generalized signs have higher power than existing tests.
Paper presents faster, robust adversarial training methods.
problem Increasing neural network robustness against adversarial attacks.
method Integrates FGSM with Pixelwise Noise Injection Layer (PNIL) and uniform noise.
result Achieves comparable results to PGD-based adversarial training but faster.
PPC learns binary codes from data similarities and dissimilarities.
problem Creating efficient binary codes from data similarities and dissimilarities.
method PPC learns binary codes by modeling attractive and repulsive forces in a signed graph.
result PPC achieves superior results in nearest-neighbor searches compared to spectral methods.
This paper focuses on the problem of prescribing mean curvature on the unit ball. Assume that , which is allowed to change sign, satisfies Morse index counting or certain kind of symmetry condition. By using a negative gradient flow method, we then prove that can be realized as the boundary mean curvature of som…
New method generates adversarial examples with conditional GAN.
problem Vulnerability of deep neural networks to adversarial examples.
method Conditional Generative Adversarial Network (cGAN) for generating adversarial examples.
result Our models reduce generating cost and improve robustness.
Study on SignGD optimization of two-layer transformer on noisy data.
problem Understanding how SignGD optimizes transformers and its generalization.
method Analysis of a two-layer transformer with SignGD on a linearly separable noisy dataset.
result SignGD converges fast but has poor generalization on noisy data.
Paper proposes a black-box technique to generate adversarial samples.
problem Robustness of Deep Neural Networks (DNNs) to adversarial samples.
method Black-box Momentum Iterative Fast Gradient Sign Method (BMI-FGSM) using Differential Evolution to approximate gradients.
result Achieves high success rates in generating adversarial samples and misclassification.
We extend natural-gradient methods to mixtures of exponential-family distributions, improving inference speed.
problem Complex, multimodal posterior distributions are difficult to approximate with simple exponential-family distributions.
method We use minimal conditional-EF representations and derive simple natural-gradient updates.
result Our natural-gradient method converges faster than black-box methods with reparameterization gradients.
Two algorithms improve federated learning efficiency and resilience.
problem Scalability issues in federated learning due to communication, privacy, and Byzantine attacks.
method Proposes two algorithms, Ada-StoSign and -StoSign, that compress gradients into bit vectors to reduce communication.
result Ada-StoSign converges with a rate of and outperforms existing methods.
Proposes Stochastic-Sign SGD for federated learning with theoretical guarantees.
problem Developing efficient, private, and resilient parameter estimation methods for federated learning.
method Introduces Stochastic-Sign SGD, a novel method based on SIGNSGD, which addresses convergence and communication efficiency.
result Demonstrates the effectiveness of Stochastic-Sign SGD through experiments on MNIST and CIFAR-10 datasets.
We analyze a fast incremental aggregated gradient method for optimizing nonconvex problems of the form . Specifically, we analyze the SAGA algorithm within an Incremental First-order Oracle framework, and show that it converges to a stationary point provably faster than both gradient descent and s…
EdgeFool generates adversarial images to mislead classifiers.
problem Misleading classifiers with adversarial images.
method Trains a fully convolutional neural network to generate perturbations that enhance image details and mislead classifiers.
result EdgeFool outperforms other adversarial methods on various classifiers and datasets.
Efficient method defends deep learning models from adversarial attacks.
problem Vulnerability of deep learning models to adversarial attacks.
method Training a Denoising Autoencoder (DAE) to reduce data dimensions and serve as a defense.
result The method provides a 2.43x speedup while maintaining robustness against adversarial attacks.
Stochastic gradient method converges as fast as deterministic for overparametrized models.
problem Convergence rate of stochastic gradient methods in overparametrized models.
method Proposes a regularity condition enabling fast convergence of SGD.
result Stochastic gradient method achieves the same convergence rate as deterministic gradient method.