Perceptor Gradients learns symbolic representations from raw data.
problem Learning transferable symbolic representations from raw data.
method Decomposes policy into perceptor network and task encoding program.
result Efficiently learns symbolic representations for control tasks.
Study shows gradient variance increases during deep learning training, contrary to common belief.
problem Understanding and minimizing gradient variance in deep learning models.
method Gradient Clustering method using stratified sampling to minimize gradient variance.
result Gradient variance increases during training, and smaller learning rates coincide with higher variance.
New input gradient regularization improves adversarial robustness efficiently.
problem Improving adversarial robustness in machine learning models.
method Derive robustness bounds, implement scaleable input gradient regularization, avoid double backpropagation.
result Input gradient regularization is competitive with adversarial training and avoids gradient obfuscation.
Researchers compare different gradient methods for ridge regression, finding conjugate gradients have similar performance.
problem Comparing statistical properties of different gradient methods in ridge regression.
method Explicit non-standard error decomposition to bound prediction error of conjugate gradient iterates.
result Conjugate gradient iterates share optimality properties with gradient flow and ridge regression up to a constant factor.
Gradient descent implicitly regularizes neural networks by penalizing large loss gradients.
problem How to optimize deep neural networks without explicit regularization.
method Backward error analysis to calculate implicit gradient regularization and demonstrate its effectiveness empirically.
result Implicit gradient regularization biases gradient descent toward flat minima, improving model robustness and test errors.
Policy gradient aims to maximize expected return using gradient ascent.
problem Finding a policy that maximizes expected return in a given class of policies.
method Gradient ascent applied to a differentiable model of the policy, estimating the gradient of expected return.
result Policy gradient methods require on-policy data for gradient estimation, limiting sample efficiency.
Improved sampling method using regularized Stein Variational Gradient Flow.
problem Improving the accuracy of sampling methods in machine learning.
method Proposed Regularized Stein Variational Gradient Flow to interpolate between SVGD and Wasserstein Gradient Flow.
result Established theoretical properties and provided preliminary numerical evidence of improved performance.
A new method reduces communication in distributed learning by skipping less informative gradient updates.
problem Efficient communication in distributed machine learning.
method Quantizes and skips less informative gradients to reduce communication overhead.
result Proves linear convergence rate similar to gradient descent with significant communication savings.
Deep Leakage from Gradients can reveal private training data from shared gradients.
problem The safety of gradients in multi-node machine learning systems is questionable.
method Empirical validation of Deep Leakage from Gradient attack on computer vision and natural language processing tasks.
result The attack is more effective than previous methods, achieving pixel-wise accuracy for images and token-wise matching for texts.
A new gradient descent method speeds up in flat regions and slows in steep directions.
problem Improving the speed and stability of gradient descent algorithms.
method Introducing a 'power gradient' where each gradient component is replaced by its H-th power, with 0<H<1. result The new gradient descent methods achieve significantly better performances, especially for Nesterov accelerated gradient and AMSGrad.
FGD reduces noisy gradient variance in SGD for neural networks.
problem Noisy and unreliable gradient estimation in SGD for deep learning.
method Solves an adaptive filtering problem to consistently estimate the local gradient.
result Significantly reduces gradient variance and accelerates convergence.
Proposes a new adaptive gradient method based on gradient differences.
problem Manual tuning of stepsize in vanilla gradient methods.
method Adaptation driven by cumulative squared norms of gradient differences.
result More robust than AdaGrad in various settings.
New methods using natural gradient for structured optimization.
problem Structured optimization problems.
method Structured second-order methods via natural gradient descent.
result Efficiency demonstrated on non-convex and deep learning problems.
CProp scales learning rate based on past gradient direction.
problem Adaptive learning rate scheduling during training.
method Gradient scaling method based on past gradient conformity.
result Significant gain in training speed on SGD and adaptive methods.
Paper examines the structure of stochastic gradients in deep learning.
problem Exploring the structure and heavy tails of stochastic gradients in deep learning.
method Conducted formal statistical tests on stochastic gradients and gradient noise.
result Stochastic gradients and gradient noise do not exhibit power-law heavy tails, but their covariance spectra do.
Faster policy learning via continuous-time gradients.
problem Efficiently estimating policy gradients for continuous-time systems.
method Approximating continuous-time gradients directly, using adaptive discretization.
result More efficient policy gradient estimator leads to faster learning.
Optimization with noisy gradients has become ubiquitous in statistics and machine learning. Reparameterization gradients, or gradient estimates computed via the "reparameterization trick," represent a class of noisy gradients often used in Monte Carlo variational inference (MCVI). However, when these gradient estimator…
Gradient flossing stabilizes RNN training by controlling Lyapunov exponents.
problem Gradient instability in RNNs leading to exploding and vanishing gradients.
method Regularizing Lyapunov exponents through backpropagation using differentiable linear algebra.
result Gradient flossing improves RNN training success rate and convergence speed.
Off-policy stochastic actor-critic methods rely on approximating the stochastic policy gradient in order to derive an optimal policy. One may also derive the optimal policy by approximating the action-value gradient. The use of action-value gradients is desirable as policy improvement occurs along the direction of stee…
Channel normalization prevents vanishing gradients in convolutional neural networks.
problem Vanishing gradients in convolutional neural networks during optimization.
method Channel normalization, which centers and normalizes each channel individually.
result Channel normalization avoids vanishing gradients, enabling efficient optimization.
Adaptive methods improve gradient descent and proximal gradient for convex optimization.
problem Improving efficiency of gradient descent and proximal gradient methods.
method Adaptive versions of GD and ProxGD using local curvature information.
result Proved convergence with local Lipschitz gradient assumptions.
Paper finds gradient estimates for warped product gradient almost Ricci solitons.
problem Nonexistence of certain gradient almost Ricci solitons.
method Modified Li-Yau technique to estimate warping function gradients.
result Nonexistence theorem for specific gradient almost Ricci solitons.
Stein variational gradient decent (SVGD) has been shown to be a powerful approximate inference algorithm for complex distributions. However, the standard SVGD requires calculating the gradient of the target density and cannot be applied when the gradient is unavailable. In this work, we develop a gradient-free variant …
We present ErasureHead, a new approach for distributed gradient descent (GD) that mitigates system delays by employing approximate gradient coding. Gradient coded distributed GD uses redundancy to exactly recover the gradient at each iteration from a subset of compute nodes. ErasureHead instead uses approximate gradien…
In this note, we observe the behavior of gradient flow and discrete and noisy gradient descent in some simple settings. It is commonly noted that addition of noise to gradient descent can affect the trajectory of gradient descent. Here, we run some computer experiments for gradient descent on some simple functions, and…
SGLRW improves robustness of stochastic gradient MCMC methods.
problem Sensitivity to minibatch size and gradient noise in stochastic-gradient MCMC methods.
method Proposes Stochastic Gradient Lattice Random Walk (SGLRW) with lattice-based discretization.
result SGLRW remains stable in regimes where SGLD fails, including heavy-tailed gradient noise.
The reparameterization gradient has become a widely used method to obtain Monte Carlo gradients to optimize the variational objective. However, this technique does not easily apply to commonly used distributions such as beta or gamma without further approximations, and most practical applications of the reparameterizat…
New method for natural policy gradients converges linearly.
problem Improving natural policy gradient methods for better convergence.
method Fisher-Rao gradient flow applied to state-action distributions.
result Linear convergence rate with geometry-dependent factor.
Paper improves reinforcement learning efficiency with deterministic value gradients.
problem High sample complexity in model-free DDPG algorithms for continuous control tasks.
method Proposes DVG and DVPG algorithms with infinite horizon value gradients to improve sample efficiency.
result DVPG algorithm substantially outperforms state-of-the-art methods on continuous control benchmarks.
Blind Descent avoids gradient issues, using a different learning approach.
problem Gradient issues like exploding and vanishing gradients.
method Does not use gradients to guide learning; instead, it is a more fundamental learning process.
result Gradient descent is a specific case of Blind Descent.
This paper bridges variational inference and Wasserstein gradient flows.
problem Combining variational inference and Wasserstein gradient flows for more efficient approximations.
method Recasting Bures-Wasserstein gradient flow as a Euclidean gradient flow and using path-derivative gradient estimator.
result A new gradient estimator for f-divergences that can be implemented using machine learning libraries. Paper proposes new gradient codes for robust distributed machine learning.
problem Robustness against slow machines (stragglers) in distributed machine learning.
method Constructs gradient codes based on probabilistic and Kronecker product methods.
result Gradient codes exist for a wide range of system parameters with comparable error performance.
A new gradient tree boosting framework reduces variance and accelerates performance.
problem High variance in stochastic gradient boosting.
method Combining gradient tree boosting with importance sampling and a regularizer.
result Achieves a linear convergence rate on logistic loss and 2.5x--18x acceleration on LogitBoost and LambdaMART.
FSGLD improves federated data sampling by correcting noisy gradients.
problem Noisy gradients and delayed communication in federated data.
method Conductive gradients to correct noisy gradients in distributed SGLD.
result FSGLD converges to true posterior even with delayed communication.
Proposes VSGD optimizer combining probabilistic and gradient-based methods.
problem Uncertainty modeling in deep neural networks.
method Combines probabilistic and gradient-based approaches using SVI.
result VSGD outperforms Adam and SGD on image classification tasks.
Adaptive batch sizes improve local gradient methods in distributed training.
problem Communication bottlenecks in distributed deep learning.
method Adaptive batch size strategies for local gradient methods.
result Adaptive batch sizes reduce minibatch gradient variance and improve training efficiency.
Proposes log density gradient to improve reinforcement learning sample complexity.
problem Residual error in gradient estimation in policy gradient methods.
method Log density gradient method to correct residual error, using state-action discounted distributional formulation.
result Min-max optimization method to approximate log density gradient with on-policy samples, achieving sample complexity of m−1/2. We propose gradient adversarial training, an auxiliary deep learning framework applicable to different machine learning problems. In gradient adversarial training, we leverage a prior belief that in many contexts, simultaneous gradient updates should be statistically indistinguishable from each other. We enforce this c…
Interpreting gradient methods as fixed-point iterations, we provide a detailed analysis of those methods for minimizing convex objective functions. Due to their conceptual and algorithmic simplicity, gradient methods are widely used in machine learning for massive data sets (big data). In particular, stochastic gradien…
COMP-AMS optimizes distributed training with compressed gradients, achieving similar accuracy with less communication.
problem Efficiently training large-scale models in distributed environments with reduced communication costs.
method Distributed optimization framework using gradient averaging and adaptive AMSGrad, with gradient compression and error feedback.
result COMP-AMS achieves the same convergence rate and linear speedup as standard AMSGrad with less communication.
Paper introduces a new multi-kernel algorithm for better gradient approximation.
problem Improving gradient approximation in high-dimensional problems.
method Develops a multi-kernel passive stochastic gradient algorithm with variance reduction.
result The multi-kernel algorithm performs better in high-dimensional problems.
DBQPG improves policy gradient estimation with fewer samples.
problem Accurate policy gradient estimation with limited samples.
method Deep Bayesian Quadrature Policy Gradient (DBQPG).
result DBQPG provides more accurate and less variable gradient estimates.
Gradient sparsification enhances privacy-preserving machine learning models.
problem Improving performance of differentially-private machine learning models under privacy constraints.
method Gradient sparsification combined with compressed sensing and additive Laplace noise.
result Gradient sparsification can improve performance of differentially-private machine learning models for small privacy budgets.
This paper presents a new class of gradient methods for distributed machine learning that adaptively skip the gradient calculations to learn with reduced communication and computation. Simple rules are designed to detect slowly-varying gradients and, therefore, trigger the reuse of outdated gradients. The resultant gra…
Characterizes corridors in loss surfaces for gradient-based optimization.
problem Understanding and mitigating training instabilities in gradient-based optimization.
method Characterizes corridors as regions where gradient descent and gradient flow trajectories are linearly related.
result Corridors indicate regions without implicit regularization effects, leading to better learning rate adaptation schemes.
Optimal defenses protect FL models from gradient reconstruction attacks.
problem Gradient reconstruction attacks compromise FL models by recovering original data from shared gradients.
method Derive a theoretical lower bound of reconstruction error, customize noise and pruning defenses, and achieve optimal trade-off between leakage and utility.
result Our methods outperform Gradient Noise and Pruning in protecting training data and maintaining model utility.
A new method computes gradients without backpropagation.
problem Optimization of machine learning models.
method Forward mode automatic differentiation to compute gradients.
result Forward gradient is an unbiased estimate of the gradient, eliminating the need for backpropagation.
Gradient penalty improves GAN performance by inducing a large-margin classifier.
problem Improving GAN performance and addressing vanishing gradients.
method A unifying framework of expected margin maximization, showing gradient penalties induce large-margin classifiers.
result Gradient penalties reduce vanishing gradients and produce better generated outputs.