Guarantees for tuning step size using meta-gradient descent.
problem Choosing optimal parameters for optimization algorithms.
method Learning-to-learn approach with meta-gradient descent.
result Meta-gradient can explode/vanish, requiring careful meta-objective design.
Meta-gradient RL learns to learn from experience.
problem Learning efficiency and adaptability in reinforcement learning.
method Meta-gradient descent to discover its own objective function.
result Meta-gradient RL adapts to learn more efficiently over time.
Representations are fundamental to artificial intelligence. The performance of a learning system depends on the type of representation used for representing the data. Typically, these representations are hand-engineered using domain knowledge. More recently, the trend is to learn these representations through stochasti…
Meta-gradient D4PG optimizes performance and constraint adherence in RL.
problem Balancing performance and adherence to complex constraints in RL.
method Uses meta-gradients to find a balance between expected return and minimizing constraint violations.
result Meta-gradient D4PG consistently outperforms baselines across MuJoCo domains.
This paper investigates different vector step-size adaptation approaches for non-stationary online, continual prediction problems. Vanilla stochastic gradient descent can be considerably improved by scaling the update with a vector of appropriately chosen step-sizes. Many methods, including AdaGrad, RMSProp, and AMSGra…
Evolutionary Strategies optimize hyper-parameters for off-policy learning.
problem Hyper-parameter sensitivity in off-policy learning.
method Application of Evolutionary Strategies for online hyper-parameter tuning.
result Our method outperforms state-of-the-art baselines.
Reinforcement learning (RL) has had many successes in both "deep" and "shallow" settings. In both cases, significant hyperparameter tuning is often required to achieve good performance. Furthermore, when nonlinear function approximation is used, non-stationarity in the state representation can lead to learning instabil…
Proposes a meta-learning algorithm to improve semi-supervised learning.
problem Overfitting and limited model generalization in consistency-based semi-supervised learning.
method Introduces a learn-to-generalize regularization term and meta-gradient optimization.
result Demonstrates improved performance on various datasets compared to state-of-the-art methods.
Learning an efficient update rule from data that promotes rapid learning of new tasks from the same distribution remains an open problem in meta-learning. Typically, previous works have approached this issue either by attempting to train a neural network that directly produces updates or by attempting to learn better i…
Synthetic data helps prevent forgetting when learning sequentially.
problem Catastrophic forgetting in neural networks.
method Generate synthetic data via two-step optimisation process using meta-gradients.
result Training on synthetic data prevents forgetting when learning sequentially.
Accelerates policy optimization in RL with optimistic and adaptive updates.
problem Improving policy optimization methods in reinforcement learning.
method Integrates foresight into policy improvement step via optimistic and adaptive updates.
result Designs an optimistic policy gradient algorithm, adaptive via meta-gradient learning.
MSLG generates soft labels to improve DNN performance on noisy datasets.
problem Significant performance degradation of DNNs due to noisy labels.
method Meta-learning techniques to estimate optimal label distribution and iteratively update soft labels.
result MSLG outperforms state-of-the-art methods by a large margin on various datasets.
Deep neural networks have been shown to be very powerful modeling tools for many supervised learning tasks involving complex input patterns. However, they can also easily overfit to training set biases and label noises. In addition to various regularizers, example reweighting algorithms are popular solutions to these p…
AdvImmune improves certifiable robustness of GNNs against adversarial attacks.
problem Vulnerability of graph neural networks to adversarial attacks.
method Proposes AdvImmune, an algorithm that optimizes with meta-gradient to improve certifiable robustness.
result Remarkably improves the ratio of robust nodes by 12%, 42%, 65% with an affordable immune budget of only 5% edges.
A core capability of intelligent systems is the ability to quickly learn new tasks by drawing on prior experience. Gradient (or optimization) based meta-learning has recently emerged as an effective approach for few-shot learning. In this formulation, meta-parameters are learned in the outer loop, while task-specific m…
Self-Tuning Actor-Critic improves reinforcement learning performance.
problem Manual hyperparameter tuning is time-consuming and domain-specific.
method Uses metagradients for online hyperparameter adaptation.
result Improves performance across various domains and tasks.
Deep learning models for graphs have advanced the state of the art on many tasks. Despite their recent success, little is known about their robustness. We investigate training time attacks on graph neural networks for node classification that perturb the discrete graph structure. Our core principle is to use meta-gradi…
In few-shot learning, typically, the loss function which is applied at test time is the one we are ultimately interested in minimising, such as the mean-squared-error loss for a regression problem. However, given that we have few samples at test time, we argue that the loss function that we are interested in minimising…
Improves meta-learning efficiency with mixed-mode differentiation.
problem Efficiently calculating complex derivatives in meta-learning.
method Mixed-Flow Meta-Gradients (MixFlow-MG) for scalable differentiation.
result Significant memory and time improvements in meta-learning tasks.
DataRater learns which data points are most valuable for training models.
problem Training model efficiency depends on high-quality training data.
method Meta-learning to estimate the value of data points for training.
result Meta-learning improves compute efficiency by filtering data effectively.
The goal of reinforcement learning algorithms is to estimate and/or optimise the value function. However, unlike supervised learning, no teacher or oracle is available to provide the true value function. Instead, the majority of reinforcement learning algorithms estimate and/or optimise a proxy for the value function. …
MetAL improves graph classification models with fewer labeled data.
problem Efficiently selecting unlabeled graph instances for training.
method Formulates AL as bilevel optimization, uses meta-learning to approximate model performance.
result MetAL outperforms existing AL algorithms on multiple graph datasets.
New algorithm improves dataset distillation with 108% improvement on ImageNet.
problem Improving dataset distillation performance.
method Reparameterization and convexification of implicit gradients (RCIG).
result Establishes new state-of-the-art on various dataset distillation tasks.
New method optimises learning via surrogate PAC-Bayes bounds.
problem Computational intractability of optimising generalisation bounds.
method Iteratively optimising surrogate training objectives derived from PAC-Bayes bounds.
result Iteratively optimising surrogates implies optimising original generalisation bounds.
New framework guarantees convergence of multi-step MAML.
problem Convergence of multi-step MAML in nonconvex settings.
method Developed a theoretical framework for two types of MAML objective functions.
result Guaranteed convergence rate and computational complexity for multi-step MAML.
Improved learning to reweight using deep interactions between student and teacher models.
problem Limitation of existing learning to reweight methods in utilizing student model's internal states.
method Proposes an algorithm that uses the student model's internal states to the teacher model, which returns adaptive weights to enhance student model training.
result Significant improvement over previous methods in image classification and neural machine translation experiments.
Blind Descent avoids gradient issues, using a different learning approach.
problem Gradient issues like exploding and vanishing gradients.
method Does not use gradients to guide learning; instead, it is a more fundamental learning process.
result Gradient descent is a specific case of Blind Descent.
G-Meta learns graph meta-learning from local subgraphs.
problem Learning from scarce data in graph tasks.
method Uses local subgraphs to transfer subgraph-specific information and learn transferable knowledge.
result G-Meta outperforms existing methods by up to 16.3% on seven datasets.
Reparameterizes mirror descent as gradient descent for efficient sparse learning.
problem Efficiently training small sparse networks with mirror descent.
method Develops a framework to convert mirror descent updates into gradient descent updates on different parameters.
result Mirror descent can be reparameterized as gradient descent on modified parameters, facilitating standard backpropagation.
Derives Mirror Descent from gradient flow on a Riemannian manifold.
problem No specific problem stated; focuses on derivation.
method Derives Mirror Descent from gradient flow on a Riemannian manifold with a natural discretization.
result Generalizes Mirror Descent to non-Hessian metrics.
Accelerates coordinate descent methods for machine learning problems.
problem Slowness of coordinate descent methods in machine learning.
method Extrapolation-based accelerated coordinate descent.
result Significant speed-up in practice compared to existing methods.
Stochastic gradient descent on manifolds improves low-rank approximation.
problem Efficiently approximate large matrices with lower rank.
method Stochastic gradient descent on a manifold.
result Algorithm outperforms Euclidean space methods on Netflix Prize data.
In this note, we observe the behavior of gradient flow and discrete and noisy gradient descent in some simple settings. It is commonly noted that addition of noise to gradient descent can affect the trajectory of gradient descent. Here, we run some computer experiments for gradient descent on some simple functions, and…
New analysis shows GMD can converge linearly under PL-like conditions.
problem Establishing linear convergence for generalized mirror descent.
method PL-based analysis for time-dependent mirrors, Taylor-series approach for stochastic GMD.
result Linear convergence of stochastic GMD under PL-like conditions.
A new method improves stochastic gradient descent for faster and more efficient estimation.
problem Efficient and fast parametric estimation methods.
method Projected stochastic gradient descent corrected by Fisher scoring.
result The method is faster and more efficient than traditional methods.
Double descent phenomenon explained in simple terms.
problem Understanding the surprising drop in test error in overparameterized models.
method Informal explanation using linear algebra and probability, visual intuition with polynomial regression, mathematical analysis with ordinary linear regression.
result Three factors create double descent: data undersampling, model size, and parameter count. Ablating any one of these factors prevents double descent.
New adaptive step-size method for convex optimization without tuning.
problem Optimizing convex functions efficiently with stochastic gradients.
method Adapted Adaptive Gradient Descent Without Descent to stochastic setting.
result Stochastic gradient descent converges under various assumptions.
New insights into double descent phenomenon in neural networks.
problem Understanding the double descent behavior in deep learning models.
method Linear teacher-student setup and tools from statistical physics.
result Distinct features are learned at different scales, leading to epoch-wise double descent.
Gradient descent dynamics in nonconvex models explained with universality.
problem Understanding long-time behavior of nonconvex gradient descent.
method Developed a state evolution system for tracking gradient descent iterates.
result Gradient descent iterates are approximately independent of data and strongly incoherent with feature vectors.
Gradient descent variants improve phase retrieval accuracy.
problem Phase retrieval problem in high-dimensional spaces.
method Gradient descent, stochastic gradient descent, Langevin algorithm, dynamical mean-field theory.
result Stochastic variants of gradient descent achieve better generalization in phase retrieval.
The paper connects tempering and entropic mirror descent for sampling.
problem Sampling from a target distribution with known unnormalized density.
method Establishes the connection between tempering SMC and entropic mirror descent, deriving convergence rates and geometric insights.
result Tempering SMC iterates correspond to entropic mirror descent on the reverse KL divergence, providing new optimization perspectives.
We consider the behavior of gradient flow and of discrete and noisy gradient descent. It is commonly noted that the addition of noise to the process of discrete gradient descent can affect the trajectory of gradient descent. In previous work, we observed such effects. There, we considered the case where the minima had …
Develops DP-SCD for stochastic coordinate descent, making it differentially private.
problem Privacy leak in auxiliary information during stochastic coordinate descent training.
method Develops DP-SCD, leveraging independent noise addition and decoupling/parallelizing coordinate updates.
result Demonstrates competitive performance against DP-SGD with less tuning.
Information geometry applies concepts in differential geometry to probability and statistics and is especially useful for parameter estimation in exponential families where parameters are known to lie on a Riemannian manifold. Connections between the geometric properties of the induced manifold and statistical properti…
Paper uses Mirror Descent for efficient risk budgeting portfolios.
problem Computing optimal risk budgeting weights for various risk measures.
method Employed Mirror Descent algorithms in deterministic and stochastic settings.
result Established convergence and quantitative rate for averaged Mirror Descent algorithm.
Any gradient descent optimization requires to choose a learning rate. With deeper and deeper models, tuning that learning rate can easily become tedious and does not necessarily lead to an ideal convergence. We propose a variation of the gradient descent algorithm in the which the learning rate is not fixed. Instead, w…
In this paper we elaborate a general homotopy-theoretic framework in which to study problems of descent and completion and of their duals, codescent and cocompletion. Our approach to homotopic (co)descent and to derived (co)completion can be viewed as ∞-category-theoretic, as our framework is constructed in the …
This paper explains why double descent sometimes occurs weakly or not at all from an optimization perspective.
problem Understanding the role of optimization in the phenomenon of double descent.
method Investigates model-wise double descent from an optimization perspective, proposing a unified explanation for its occurrence.
result Model-wise double descent is observed if and only if the optimizer can find a sufficiently low-loss minimum.