Gradient perturbation, widely used for differentially private optimization, injects noise at every iterative update to guarantee differential privacy. Previous work first determines the noise level that can satisfy the privacy requirement and then analyzes the utility of noisy gradient updates as in the non-private cas…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “gradient perturbation”
SGD converges with perturbed forward-backward passes, explained by geometric amplification.
problem Analyzing convergence of SGD with perturbed forward-backward passes in composite optimization.
method Characterized propagation and amplification of perturbations, derived convergence guarantees for non-convex and PL objectives.
result Perturbations cascade through the computational graph, affecting convergence order under specific conditions.
Implicit Regularization in Perturbed Deep Matrix Factorization: Spectral Conditions and Stabilitymath.OC
The paper examines how gradient descent stabilizes low-rank matrix factorization in noisy conditions.
problem Stability of low-rank implicit regularization in perturbed deep matrix factorization.
method Derives spectral conditions for gradient descent to exhibit a low-rank phase in noiseless settings and analyzes perturbed dynamics.
result Gradient descent converges to a low-rank solution under perturbation, with explicit dependence on perturbation size.
The paper shows how to stabilize perturbed Kähler-Ricci solitons.
problem Stabilizing perturbed Kähler-Ricci solitons.
method Normalized Kähler-Ricci flow starting from perturbed metrics.
result The flow converges to an asymptotically conical gradient expanding Kähler-Ricci soliton.
Implicit Regularization Makes Overparameterized Asymmetric Matrix Sensing Robust to Perturbationscs.LG
Paper shows robustness of gradient descent in matrix sensing despite perturbations.
problem Understanding robustness of gradient descent in matrix sensing.
method Developed perturbed gradient flow to capture noise and improve robustness.
result Gradient descent is robust to perturbations in matrix sensing.
Most random ReLU networks are vulnerable to small, Euclidean adversarial perturbations.
problem Vulnerability of ReLU networks to adversarial attacks.
method Analysis of random ReLU networks with decreasing dimensions, using gradient flow and descent.
result Most examples can be perturbed by small Euclidean distances via gradient methods.
We present a version of the equivariant gradient degree defined for equivariant gradient perturbations of an equivariant unbounded self-adjoint operator with purely discrete spectrum in Hilbert space. Two possible applications are discussed.
In this paper we introduce a family of stochastic gradient estimation techniques based of the perturbative expansion around the mean of the sampling distribution. We characterize the bias and variance of the resulting Taylor-corrected estimators using the Lagrange error formula. Furthermore, we introduce a family of va…
Noise injection before gradient steps helps in regularization for neural networks.
problem Improving generalization in overparametrized neural networks.
method Injecting small noise perturbations before computing gradient steps, especially in layer-wise fashion.
result Small noise perturbations can explicitly regularize neural networks without variance explosion.
New method creates universal perturbations to fool neural network interpretations.
problem Vulnerability of gradient-based saliency maps to adversarial perturbations.
method Gradient-based optimization and PCA-based approach to create UPI.
result Existence and successful application of Universal Perturbation for Interpretation (UPI).
Paper proposes faster method to find local minima in nonconvex optimization.
problem Escaping saddle points and finding local minima in nonconvex optimization.
method LENA (Last stEp shriNkAge) framework for faster perturbed stochastic gradient methods.
result LENA finds -approximate local minima within evaluations.
Adversarial perturbations fool deepfake detectors with high accuracy.
problem Improving deepfake detection accuracy against adversarial attacks.
method Used adversarial perturbations and two defenses: Lipschitz regularization and Deep Image Prior (DIP).
result Deepfake detectors achieved 27% accuracy on perturbed images, compared to 95% on unperturbed.
We consider in this work a system of two stochastic differential equations named the perturbed compositional gradient flow. By introducing a separation of fast and slow scales of the two equations, we show that the limit of the slow motion is given by an averaged ordinary differential equation. We then demonstrate that…
This paper aims at achieving a "good" estimator for the gradient of a function on a high-dimensional space. Often such functions are not sensitive in all coordinates and the gradient of the function is almost sparse. We propose a method for gradient estimation that combines ideas from Spall's Simultaneous Perturbation …
Gradient-based methods introduce noise that penalizes models sensitive to weight perturbations.
problem Noise in gradient-based optimization methods.
method Analysis of Gradient Descent (GD) and Stochastic Gradient Descent (SGD) updating all parameters simultaneously.
result Noise introduced by simultaneous parameter updates penalizes models sensitive to weight perturbations.
LoRA fine-tuning explained with gradient dynamics for low-rank perturbations.
problem Understanding why gradient descent converges to useful low-rank perturbations in LoRA fine-tuning.
method Generalized student-teacher setting with i.i.d. samples and online gradient descent.
result Gradient descent converges to the teacher model in iterations under certain conditions.
Learnable token perturbations boost extrapolation in LLMs.
problem Limited flexibility of current discrete perturbations in large language models.
method Learnable continuous latent vector transformations in embedding space, unbiased estimating equations, stochastic gradient descent optimization.
result Significant gains in out-of-domain settings over state-of-the-art methods.
Adversarial training, in which a network is trained on both adversarial and clean examples, is one of the most trusted defense methods against adversarial attacks. However, there are three major practical difficulties in implementing and deploying this method - expensive in terms of extra memory and computation costs; …
Proposes a new method to improve deep model security against adversarial deformations.
problem Deep neural networks' resistance to adversarial attacks, especially location perturbations.
method Regularizes flow gradients to provide a tighter bound and improve model resistance.
result Models trained with flow gradient regularization show better resistance to adversarial deformations compared to input gradient regularization and adversarial training.
The paper studies heat kernels on modified manifolds and bounds their properties.
problem Bounding heat kernels on modified Riemannian manifolds.
method Derives upper bounds and gradient estimates for the heat kernel of .
result Establishes upper bounds and gradient estimates for the heat kernel of modified manifolds.
Although gradient descent (GD) almost always escapes saddle points asymptotically [Lee et al., 2016], this paper shows that even with fairly natural random initialization schemes and non-pathological functions, GD can be significantly slowed down by saddle points, taking exponential time to escape. On the other hand, g…
SGD generalization bounds derived from information theory.
problem Understanding generalization of SGD for non-convex functions.
method Combining information-theoretic bounds with perturbation analysis.
result Upper bounds on SGD's generalization error based on gradient variance and function smoothness.
PWGF escapes saddle points in nonconvex optimization.
problem Escaping saddle points in nonconvex optimization.
method PWGF uses noisy perturbations via Gaussian process to escape saddle points.
result PWGF achieves second-order optimality for nonconvex objectives.
Tseytlin has recently proposed that an action functional exists whose gradient generates to all orders in perturbation theory the Renormalization Group (RG) flow of the target space metric in the worldsheet sigma model. The gradient is defined with respect to a metric on the space of coupling constants which is explici…
Information geometry of perturbed gradient flow systems on hypergraphs: A perspective towards nonequilibrium physicscond-mat.stat-mech
The article reviews how gradient flow systems on hypergraphs connect to information geometry and nonequilibrium physics.
problem Understanding the geometry of perturbed gradient flow systems on hypergraphs.
method Formulating modern nonequilibrium principles within the framework of perturbed gradient flow systems on hypergraphs.
result New concepts like moduli spaces and thermodynamical area are introduced to understand speed limits.
Gradient descent with chaotic perturbations improves generalization.
problem Improving generalization of gradient descent.
method Introducing chaotic perturbations to gradient descent to achieve improved generalization.
result Gradient descent with chaotic perturbations converges to a heavy-tailed SDE, leading to improved generalization.
Machine Learning models are vulnerable to adversarial attacks that rely on perturbing the input data. This work proposes a novel strategy using Autoencoder Deep Neural Networks to defend a machine learning model against two gradient-based attacks: The Fast Gradient Sign attack and Fast Gradient attack. First we use an …
Paper introduces input perturbation for privacy in machine learning models.
problem Protecting both training data and model parameters while maintaining privacy.
method Add noise to training data and train with perturbed data for differential privacy.
result Achieves (ε,δ)-differential privacy on the final model with privacy on original data.
We study stability of non-compact gradient Kaehler-Ricci flow solitons with positive holomorphic bisectional curvature. Our main result is that any compactly supported perturbation and appropriately decaying perturbations of the Kaehler potential of the soliton will converge to the original soliton under Kaehler-Ricci …
Improved perturbation reduces matrix condition number to O(n) with minimal storage.
problem Reducing the condition number of deterministic matrices for efficient algorithmic use.
method Introduced pattern matrices and sparse perturbations with dependent entries.
result Condition number reduced to O(n) with O(n) random numbers in O(log n) precision.
DIP-FAT improves adversarial training by diversifying perturbations.
problem Adversarial examples fool deep neural networks, leading to overfitting and poor performance.
method DIP-FAT uses random directions to diversify perturbations in adversarial training.
result DIP-FAT reduces overfitting and improves clean data accuracy.
New methods help escape strict saddle points in nonsmooth optimization.
problem Escaping strict saddle points in nonsmooth optimization.
method An inexact stochastically perturbed gradient method applied to the Moreau envelope.
result A variety of algorithms for nonsmooth optimization can efficiently escape strict saddle points of the Moreau envelope.
Paper examines adversarial attacks on weather forecasting models, focusing on TC trajectory prediction.
problem Adversarial attacks can mislead downstream TC trajectory predictions in DLWF models.
method Proposes Cyc-Attack, a method using a surrogate model and skewness-aware loss function to generate adversarial TC paths.
result Cyc-Attack achieves higher true positive rates and lower false alarm rates compared to conventional methods.
Recent advances show that deep neural networks are not robust to deliberately crafted adversarial examples which many are generated by adding human imperceptible perturbation to clear input. Consider norms attacks, Project Gradient Descent (PGD) and the Carlini and Wagner (C\&W) attacks are the two main methods, …
New algorithm eliminates sign function in PGD attacks, improving performance.
problem Improving robustness of neural networks against adversarial attacks.
method Proposes a new raw gradient descent (RGD) algorithm that eliminates the sign function in PGD attacks.
result The RGD algorithm outperforms PGD and other competitors in various settings.
In this paper we consider a perturbation of the Ricci solitons equation proposed in \cite{jpb1} and studied in \cite{CaMa} and we classify noncompact gradient shrinkers with bounded nonnegative sectional curvature.
Node-perturbation learning is a type of statistical gradient descent algorithm that can be applied to problems where the objective function is not explicitly formulated, including reinforcement learning. It estimates the gradient of an objective function by using the change in the object function in response to the per…
Study on low-dimensional adversarial perturbations in classification models.
problem Understanding and quantifying the effectiveness of low-dimensional adversarial perturbations.
method Analytical lower-bounds for fooling rate, considering binary classifiers under generic regularity conditions.
result Rigorous explanation for the success of heuristic methods in generating low-dimensional adversarial perturbations.
A functional central limit theorem for kernel gradient flow and infinitesimal gradient boostingmath.PR
Functional central limit theorem for kernel gradient flow and infinitesimal gradient boosting
problem Fluctuations of boosting processes around their deterministic limit
method Stochastic perturbation analysis of ODEs in Banach spaces
result Rescaled deviations converge to a Gaussian process
Saliency methods can make deep neural network predictions more interpretable by identifying a set of critical features in an input sample, such as pixels that contribute most strongly to a prediction made by an image classifier. Unfortunately, recent evidence suggests that many saliency methods poorly perform, especial…
Deep neural networks are easily fooled high confidence predictions for adversarial samples
A key problem in research on adversarial examples is that vulnerability to adversarial examples is usually measured by running attack algorithms. Because the attack algorithms are not optimal, the attack algorithms are prone to overestimating the size of perturbation needed to fool the target model. In other words, the…
STRATA generates code adversarial examples efficiently without gradients.
problem Generating adversarial examples for code that retains functional meaning.
method Uses token frequency statistics to construct gradient-free adversarial examples.
result Empirically outperforms gradient-based methods with less information and effort.
We present CROSSGRAD, a method to use multi-domain training data to learn a classifier that generalizes to new domains. CROSSGRAD does not need an adaptation phase via labeled or unlabeled data, or domain features in the new domain. Most existing domain adaptation methods attempt to erase domain signals using technique…
Anti-correlated noise improves machine learning model generalization.
problem Improving machine learning model generalization.
method Injecting anticorrelated noise into gradient descent steps.
result Anti-correlated noise leads to better model generalization than uncorrelated noise.
Unified framework for gradient estimation in combinatorial spaces.
problem Scaling relaxed gradient estimators to large combinatorial distributions.
method Introducing stochastic softmax tricks within the perturbation model framework.
result Stochastic softmax tricks improve model performance and discover more latent structure.
In this study, we propose the leveraging of interpretability for tasks beyond purely the purpose of explainability. In particular, this study puts forward a novel strategy for leveraging gradient-based interpretability in the realm of adversarial examples, where we use insights gained to aid adversarial learning. More …
Geometry-informed irreversible perturbations for accelerated convergence of Langevin dynamicsstat.ME
Novel geometry-informed irreversible perturbation accelerates Langevin dynamics convergence.
problem Accelerating convergence of Langevin dynamics for Bayesian computation.
method Geometry-informed irreversible perturbation of Riemannian manifold Langevin dynamics.
result Improves estimation performance over irreversible perturbations that ignore geometry.