MaMiC proposes a dual curriculum for robot manipulation tasks with sparse rewards.
problem Overcoming exploratory constraints in robot manipulation tasks with sparse rewards.
method Includes a macro curriculum scheme and a micro curriculum scheme to guide learning.
result Combining macro and micro curriculum strategies improves performance in robot manipulation tasks.
StarCraft II poses a grand challenge for reinforcement learning. The main difficulties of it include huge state and action space and a long-time horizon. In this paper, we investigate a hierarchical reinforcement learning approach for StarCraft II. The hierarchy involves two levels of abstraction. One is the macro-acti…
Paper introduces a new curriculum generation method for reinforcement learning.
problem Improving reinforcement learning performance and speed through curriculum learning.
method The paper proposes a novel curriculum generation paradigm based on progression and mapping functions.
result Empirical results show the new approach outperforms state-of-the-art algorithms.
Curriculum learning shows marginal benefits over random ordering on standard datasets.
problem Investigating the benefits of curriculum learning over random ordering in training neural networks.
method Experiments with curriculum, anti-curriculum, and random-curriculum orderings on various datasets and conditions.
result Curriculum learning only marginally improves performance on standard datasets and is outperformed by random ordering.
Analyzes when and why curricula help neural networks learn.
problem Why and when does curriculum learning help neural networks?
method Statistical physics methods applied to a neural network model.
result Curricula can modestly speed up learning but do not provide generalization benefit.
Automatically generates a deep RL curriculum for faster and more stable learning.
problem How to automatically generate a curriculum for deep RL agents.
method Interprets curriculum generation as an inference problem, learning task distributions progressively.
result Curricula significantly improve learning performance across various environments and deep RL algorithms.
Curriculum learning in reinforcement learning is a training methodology that seeks to speed up learning of a difficult target task, by first training on a series of simpler tasks and transferring the knowledge acquired to the target task. Automatically choosing a sequence of such tasks (i.e. a curriculum) is an open pr…
Unified framework for multi-objective curriculum learning in robotics.
problem Improving sample efficiency and final performance in robotic policy learning.
method Unified automatic curriculum learning framework with multi-task hyper-net and flexible memory mechanism.
result Superior performance compared to state-of-the-art methods in robotic manipulation tasks.
Curriculum learning speeds up agent learning in Minecraft, a complex visual domain.
problem Training agents to learn multiple tasks in a complex, visual domain.
method Learning-progress based curriculum and dynamic exploration bonuses.
result Curriculum learning improves agent performance in a complex reinforcement learning problem.
New algorithm combines curriculum learning with HER for complex object manipulation tasks.
problem Learning complex sequential object manipulation tasks from scratch is challenging.
method Curriculum learning with Hindsight Experience Replay (HER) for recurrent object manipulation tasks.
result Significant improvement in learning sequential object manipulation tasks compared to vanilla-HER.
Curriculum learning and imitation learning improve control over financial time-series data.
problem Improving control performance over complex financial time-series data.
method Data augmentation for curriculum learning and policy distillation for imitation learning.
result Curriculum learning shows significant improvement over time-series control tasks.
Curriculum learning in reinforcement learning is used to shape exploration by presenting the agent with increasingly complex tasks. The idea of curriculum learning has been largely applied in both animal training and pedagogy. In reinforcement learning, all previous task sequencing methods have shaped exploration with …
This paper introduces curriculum loss to improve robustness and generalization in deep neural networks against label corruption.
problem Improving robustness and generalization in deep neural networks against label corruption.
method Proposes curriculum loss, a tighter upper bound of the 0-1 loss, which optimizes robust properties while adaptively selecting samples.
result Curriculum loss enhances robustness and generalization in deep neural networks compared to conventional losses.
Curriculum learning improves deep generative models for noisy data.
problem Training deep generative models with noisy data is challenging.
method Proposes a curriculum learning algorithm based on data point centrality.
result The algorithm learns optimal generative models for noisy data.
The paper studies the benefits of curriculum learning in linear regression tasks.
problem Theoretical understanding of curriculum learning's benefits in machine learning.
method Theoretical analysis of curriculum learning in structured and unstructured multitask linear regression problems.
result Adaptive learning in the unstructured setting is fundamentally harder than oracle learning, but not in the structured setting.
In this paper, we investigate a new form of automated curriculum learning based on adaptive selection of accuracy requirements, called accuracy-based curriculum learning. Using a reinforcement learning agent based on the Deep Deterministic Policy Gradient algorithm and addressing the Reacher environment, we first show …
Q-CurL optimizes quantum learning with a curriculum design.
problem Efficiently training quantum models with limited resources.
method Quantum curriculum learning framework.
result Q-CurL enhances training convergence and generalization.
Survey on curriculum learning for reinforcement learning.
problem Efficiently learning complex tasks in reinforcement learning.
method Framework for curriculum learning in reinforcement learning.
result Survey and classification of existing CL methods.
Curriculum learning strategies improve GAN training speed and quality.
problem Training GANs is notoriously difficult and time-consuming.
method Proposes three curriculum learning strategies based on difficulty scores.
result Curriculum learning strategies lead to faster convergence and superior results.
EPC curriculum improves MARL performance as agent population grows.
problem Challenges in learning good policies for large multi-agent systems.
method Evolutionary Population Curriculum (EPC) for scaling MARL.
result EPC consistently outperforms baselines as agent population increases.
Paper proposes a curriculum for RL agents to learn from expert demonstrations.
problem Training RL agents efficiently from expert demonstrations.
method Develops a training curriculum that starts from the goal and works backwards.
result Improves sample efficiency for some tasks compared to PPO.
In this paper we introduce Curriculum GANs, a curriculum learning strategy for training Generative Adversarial Networks that increases the strength of the discriminator over the course of training, thereby making the learning task progressively more difficult for the generator. We demonstrate that this strategy is key …
This work improves NAT translation accuracy through fine-tuning with curriculum learning.
problem Improving NAT translation accuracy while maintaining inference speed.
method Curriculum learning applied to fine-tuning of a pre-trained AT model to a NAT model.
result Significant improvement in translation accuracy (over 1 BLEU score) and speedup in inference.
Proposes a curriculum learning algorithm to maximize cumulative return in reinforcement learning.
problem Maximizing cumulative return in reinforcement learning tasks.
method Task sequencing algorithm maximizing cumulative return, using curriculum learning to minimize suboptimal actions.
result Significantly better performance on cumulative return maximization compared to metaheuristic algorithms.
This paper introduces CENIE to quantify environment novelty for better UED.
problem Challenges in measuring environment novelty for effective UED.
method CENIE framework using state-action space coverage and Gaussian Mixture Models.
result CENIE improves UED performance across multiple benchmarks.
Bi-directional Curriculum Learning improves graph anomaly detection by considering both homogeneity and heterogeneity.
problem Existing graph anomaly detection methods often ignore the different contributions of nodes to training.
method Introduces Bi-directional Curriculum Learning (BCL) to optimize GAD methods by considering both homogeneity and heterogeneity of nodes.
result Extensive experiments show that BCL significantly improves the performance of GAD anomaly detection models.
Humans tend to learn complex abstract concepts faster if examples are presented in a structured manner. For instance, when learning how to play a board game, usually one of the first concepts learned is how the game ends, i.e. the actions that lead to a terminal state (win, lose or draw). The advantage of learning end-…
Paper optimizes Bayesian optimization for complex functions with macro-actions.
problem Optimizing complex, highly uncertain functions efficiently.
method Generalized GP upper confidence bound with macro-actions for scalable lookahead.
result Asymptotically optimal anytime variant of epsilon-Macro-GPO policy.
New method learns optimal environment and goal difficulty for reinforcement learning.
problem Lack of robust transfer in reinforcement learning.
method Self-Supervised Active Domain Randomization (SS-ADR).
result Optimal domain randomization strategy improves transfer in goal-directed tasks.
CP-DRL improves curriculum reinforcement learning by leveraging causal relationships.
problem Designing effective task sequences for reinforcement learning.
method Causal-Paced Deep Reinforcement Learning (CP-DRL) that approximates SCM differences based on interaction data.
result CP-DRL outperforms existing methods on benchmarks, achieving faster convergence and higher returns.
We prove that each coarsely homogenous separable metric space X is coarsely equivalent to one of the spaces: the sigleton, the Cantor macro-cube or the Baire macro-space. This classification is derived from coarse characterizations of the Cantor macro-cube and of the Baire macro-space given in this paper. Namely, we …
ACL improves DRL by adapting task difficulty to agent's capabilities.
problem Improving sample efficiency and performance in DRL.
method Adapting task difficulty based on agent's capacity.
result Enhanced sample efficiency and asymptotic performance.
Curriculum learning helps neural networks learn parities more efficiently.
problem Improving learning efficiency for neural networks on parity targets.
method Using a curriculum learning approach with a mixture of sparse and dense inputs.
result A 2-layer ReLU neural network can learn parities more efficiently than a fully connected network.
Recent variants improve knowledge distillation performance.
problem Improving the performance of knowledge distillation.
method Introducing additional components or changing the learning process.
result These variants have shown promising results.
One problem in the application of reinforcement learning to real-world problems is the curse of dimensionality on the action space. Macro actions, a sequence of primitive actions, have been studied to diminish the dimensionality of the action space with regard to the time axis. However, previous studies relied on human…
Extracting a curriculum from a teacher network improves distillation efficiency.
problem Efficiently training a small network using a large teacher network's output.
method Random projection of teacher network's hidden representations to progressively train the student network.
result Extracted curriculum significantly outperforms one-shot distillation and achieves similar performance to progressive distillation.
SAMPLR optimizes for ground truth in aleatoric parameters to avoid curriculum-induced covariate shift.
problem Curriculum learning shifts training distribution, leading to suboptimal policies in aleatoric settings.
method SAMPLR optimizes ground-truth utility function, avoiding curriculum-induced covariate shift.
result SAMPLR preserves optimality under ground-truth distribution, promoting robustness across various environments.
New approach improves black-box planning efficiency by discovering focused macros.
problem Difficulty of deterministic planning increases exponentially with depth.
method Discovering macro-actions with focused effects to improve goal-count heuristics.
result Focused macros dramatically improve black-box planning efficiency.
Training neural networks is traditionally done by providing a sequence of random mini-batches sampled uniformly from the entire training data. In this work, we analyze the effect of curriculum learning, which involves the non-uniform sampling of mini-batches, on the training of deep networks, and specifically CNNs trai…
We introduce Mix&Match (M&M) - a training framework designed to facilitate rapid and effective learning in RL agents, especially those that would be too slow or too challenging to train otherwise. The key innovation is a procedure that allows us to automatically form a curriculum over agents. Through such a curriculum …
Proposes a new curriculum learning approach to improve deep RL agent generalization.
problem Challenges in training deep RL agents to generalize over unseen situations.
method Two-stage curriculum learning approach: teacher learns high-exploration curriculum, distills priors to generate expert curriculum.
result 50% improvement in average performance over state-of-the-art methods.
RLVR training dynamics reveal an implicit curriculum that shapes learning progression.
problem Understanding how RLVR overcomes the long-horizon barrier.
method Developed a theory of training dynamics for RLVR on transformers, using Fourier analysis on finite groups.
result Mixed-difficulty training naturally follows an implicit curriculum, shaping the learning progression from easy to hard.
RLVR learning dynamics naturally create an implicit curriculum for transformers.
problem How rewards based on final outcomes help overcome the long-horizon barrier in reasoning models.
method Developed a theory of training dynamics for RLVR on transformers, using Fourier analysis on finite groups.
result Mixed-difficulty training induces an implicit curriculum that shapes the learning progression from easy to hard.
Two different formulas for macro F1 lead to significant differences in classification evaluation.
problem Evaluation discrepancies in binary, multi-class, and multi-label classification problems.
method Comparison of two formulas for macro F1 metric.
result The two formulas can result in up to a 0.5 difference and different classifier rankings.
New algorithm outperforms existing ones by focusing on mastering rate.
problem Low sample efficiency in learning progress-based algorithms.
method Proposes a new algorithm based on mastering rate.
result Significantly outperforms learning progress-based algorithms.
Proposes a curriculum-based dropout discriminator for domain adaptation.
problem Improving domain adaptation using deep learning networks trained on large labeled datasets.
method Introduces a curriculum-based dropout discriminator that gradually increases sample variance and uses reverse gradients to align source and target feature representations.
result The proposed model outperforms state-of-the-art results in domain adaptation tasks.
Aggregated variables can mask causal effects, turning unconfounded into confounded relations.
problem Aggregated variables can mask causal effects, leading to paradoxical confounding.
method Analysis of how aggregated variables can change the definition of causality and the feasibility of causal relations.
result Macro causal relations are defined by micro states, not just aggregated variables.
New method uses label-weighted conformal prediction for macro-coverage guarantees in classification.
problem Finding a balance between class-conditional and marginal coverage in long-tailed datasets.
method Label-weighted conformal prediction for macro-coverage guarantees.
result Validated prediction sets with macro-coverage guarantees on large-scale image datasets.