Improved sample efficiency in reinforcement learning with deep Gaussian processes.
problem Efficiently learn to control actions with limited interaction data.
method Deep Gaussian processes that simulate dynamics with depth and prior knowledge.
result Significantly improved early sample efficiency across various tasks, including half-cheetah control.
This work improves sample efficiency in meta-learning for nonlinear tasks.
problem Learning complex tasks efficiently with limited data.
method Subspace-based representations for nonlinear tasks.
result Subspace-based representations can be learned efficiently and improve future task performance.
MEMEC improves sample efficiency in reinforcement learning.
problem Lack of sample efficiency in reinforcement learning.
method Proposes MEMEC, a Boltzmann policy with state-dependent temperature for more principled exploration.
result MEMEC outperforms other methods on classic RL environments and Atari games.
This work shows synthetic gradients can outperform backpropagation in sample efficiency.
problem The efficiency of backpropagation in training neural networks.
method Unified vectorized feedback framework for loss-based and reward-based learning, introducing synthetic gradients.
result Synthetic gradients can achieve lower gradient-estimation mean squared error than backpropagation under certain conditions.
Enhanced feedback model improves sample-efficiency in POMDPs.
problem Exponential hardness of learning in POMDPs.
method Multiple observations in hindsight feedback model.
result Sample-efficient learning possible for new subclasses of POMDPs.
Improved sample efficiency with normalized RBF kernels in neural networks.
problem Learning more with less data in deep learning models.
method Two-phase method to train neural networks with normalized RBF kernels as output layer.
result Normalized RBF kernel networks achieve higher sample efficiency, compactness, and separability.
Improves sample efficiency in RL by matching model-based gradients.
problem Lack of sample efficiency in model-free RL.
method Gradient matching between model-based and model-free RL components.
result Improves sample efficiency without increasing asymptotic bias.
IMPACT improves RL training speed without sacrificing sample efficiency.
problem Limited sample efficiency in scalable RL architectures.
method Proposes IMPACT, extending IMPALA with target networks, circular buffers, and truncated importance sampling.
result IMPACT achieves higher rewards and significantly reduces training time compared to IMPALA.
MineRL Competition reduced reinforcement learning sample needs.
problem Sample inefficiency in reinforcement learning.
method Human demonstrations and imitation learning integrated into reinforcement learning algorithms.
result Top solutions used deep reinforcement learning and imitation learning.
HySRL improves RL sample efficiency with shifted-dynamics data.
problem Leveraging historical data with shifted dynamics to improve sample efficiency in RL.
method HySRL, a hybrid transfer RL algorithm that uses prior information on dynamics shift to achieve better sample complexity.
result HySRL achieves problem-dependent sample complexity and outperforms pure online RL.
Combines BC and GAIL for efficient imitation learning.
problem Efficient imitation learning without reward signals.
method Integrates Behavior Cloning and Generative Adversarial Imitation Learning.
result Combination leads to stable and sample-efficient learning.
Improves sample efficiency in reinforcement learning with input representation.
problem Poor sample efficiency in reinforcement learning.
method Attention-based method to project inputs into an invariant representation space.
result Representation space is m! smaller for inputs of m objects, improving sample efficiency. Improved agnostic boosting with better sample efficiency.
problem Agnostic boosting's poor sample efficiency compared to Empirical Risk Minimization.
method Leverages sample reuse across rounds, guarantees better generalization.
result Substantially more sample-efficient agnostic boosting algorithm.
Automates RL with sample-efficient hyperparameter optimization.
problem Challenges in applying deep RL due to hyperparameter sensitivity and inefficiency.
method Population-based AutoRL framework for meta-optimizing RL algorithms and architectures.
result Reduces the number of environment interactions needed for meta-optimization by up to an order of magnitude.
Improves sample efficiency and generalization in vision-based RL by enhancing exploration.
problem Low sample efficiency in vision-based RL using images as observations.
method Integrates state representation learning to enhance exploration and sample diversity.
result Significant improvement in sample efficiency across various environments.
Deep neuroevolution, that is evolutionary policy search methods based on deep neural networks, have recently emerged as a competitor to deep reinforcement learning algorithms due to their better parallelization capabilities. However, these methods still suffer from a far worse sample efficiency. In this paper we invest…
Paper tackles sample-efficient offline RL, proposing data diversity and unified algorithms.
problem Sample-efficient learning from historical data for sequential decision-making.
method Proposes data diversity and unifies three offline RL algorithm classes: VS, RO, and PS.
result Comparable sample efficiency for VS, RO, and PS algorithms under standard assumptions.
Paper improves sample efficiency of transfer learning in diffusion models.
problem Diffusion models need too much data to train from scratch.
method Assumes shared low-dimensional representation across tasks for improved sample efficiency.
result Sample complexity of target tasks can be reduced with a well-learned representation.
Catalyst.RL accelerates RL research with efficient training.
problem Efficient reinforcement learning training in complex environments.
method Open-source PyTorch framework with distributed training and RL algorithms.
result Catalyst.RL achieved 2nd place in a computationally expensive RL challenge.
Sample efficiency is a crucial problem in deep reinforcement learning. Recent algorithms, such as REDQ and DroQ, found a way to improve the sample efficiency by increasing the update-to-data (UTD) ratio to 20 gradient update steps on the critic per environment sample. However, this comes at the expense of a greatly inc…
Improved reinforcement learning in Minecraft with human demonstrations.
problem Sample inefficiency in reinforcement learning.
method Training policy networks on human demonstrations first, then fine-tuning with reinforcement learning.
result Best agent achieved a mean score of 48 in Minecraft.
Procgen Benchmark uses procedurally generated games to test reinforcement learning.
problem Lack of diverse and high-quality training environments for reinforcement learning.
method Developed 16 procedurally generated game-like environments and used them to benchmark reinforcement learning.
result Procedurally generated environments are essential for training and evaluating reinforcement learning agents.
New RL approach tackles non-linear MDPs without linear assumptions.
problem Sample efficiency in RL for complex, nonlinear MDPs with continuous states.
method Introduces EPW condition to relax linear structure requirements; provides sample-efficient RL algorithm.
result EPW condition allows solving MDPs without linear assumptions, including Atari games.
Improves sample efficiency in reinforcement learning by controlling task distribution.
problem Learning and generalization of behaviors across related tasks in intelligent robots.
method Introduces a novel relative entropy reinforcement learning algorithm that allows the agent to control the intermediate task distribution.
result The proposed curriculum learning scheme drastically improves sample efficiency and enables learning in challenging scenarios.
This paper advances sample-efficient learning for partially observable RL by introducing B-stability and new algorithms.
problem Hard sample complexity for learning near-optimal policies in partially observable RL.
method Proposes B-stability as a unified structural condition and develops new algorithms for sample-efficient learning.
result Any B-stable PSR can be learned with polynomial samples, improving over current best complexities.
Increasing input dimensionality improves deep RL performance and sample efficiency.
problem Real-world reinforcement learning applications often lack sufficient training data.
method Proposed an online feature extractor network (OFENet) to improve deep RL performance and sample efficiency.
result RL agents learn more efficiently with high-dimensional input representations than with lower-dimensional state observations.
Latent MoS learns multiple symmetries for efficient dynamic learning.
problem Efficiently learning dynamics from limited system measurements.
method Latent Mixture of Symmetries (Latent MoS) with hierarchical architecture.
result Latent MoS outperforms baselines in interpolation and extrapolation tasks.
A new method uses a frozen language model to improve sample efficiency in reinforcement learning.
problem Improving sample efficiency in reinforcement learning with partially observable environments.
method FROZEN Hopfield network and HELM (History Embedding Language Model) method.
result HELM achieves new state-of-the-art results on Minigrid and Procgen environments.
Paper tackles sample-efficient RL for linearly realizable MDPs with limited revisiting.
problem Sample-efficient reinforcement learning for linearly realizable MDPs with limited revisiting.
method Develops a new sampling protocol that allows for backtracking and revisiting states in a controlled manner.
result Achieves polynomial sample complexity scaling with feature dimension, horizon, and inverse sub-optimality gap.
Novelty search in low-dimensional space improves sample efficiency in exploration tasks.
problem Efficient exploration in complex environments with sparse rewards.
method Combines model-based and model-free objectives to learn a low-dimensional representation. Uses intrinsic novelty rewards based on nearest neighbor distances in this space.
result Our approach achieves more sample-efficient exploration compared to strong baselines on various tasks.
New framework improves sample efficiency and robustness in RL with smooth policies.
problem Sample inefficiency and lack of robustness in deep reinforcement learning.
method SR^2L framework, smoothness-inducing regularization.
result Improved sample efficiency and robustness in both on-policy and off-policy RL algorithms.
New findings show good representations alone are insufficient for efficient reinforcement learning.
problem Understanding when good representations are enough for efficient reinforcement learning.
method Statistical analysis of reinforcement learning methods, focusing on value-based, model-based, and policy-based learning.
result Hard thresholds for reinforcement learning methods show good representations alone are insufficient, unless they meet certain quality criteria.
Reinforcement Learning (RL) algorithms can suffer from poor sample efficiency when rewards are delayed and sparse. We introduce a solution that enables agents to learn temporally extended actions at multiple levels of abstraction in a sample efficient and automated fashion. Our approach combines universal value functio…
Improved sample-efficient learning for non-coherent digital jamming.
problem Learning optimal jamming strategies in non-coherent digital modulation schemes without prior knowledge.
method Introduced a linear bandit algorithm that accounts for action similarities and integrates context features.
result Significantly improved convergence behavior compared to prior art.
DAC enhances exploration in reinforcement learning with entropy regularization.
problem Improving exploration efficiency in reinforcement learning.
method Sample-aware entropy regularization using replay buffer action distributions.
result DAC significantly outperforms existing algorithms in reinforcement learning tasks.
A new correction term improves sample efficiency in deep reinforcement learning.
problem Momentum accumulation in TD learning leads to doubly stale gradients.
method Proposed a correction term to address the issue of doubly stale gradients.
result Improves sample efficiency in policy evaluation.
CoT improves transformer sample efficiency by reducing input token dependencies and attention sparsity.
problem Transformer sample inefficiency in simple tasks.
method Demonstrated through parity-learning setup, showing CoT reduces required samples from exponential to polynomial.
result Transformer learns function within polynomial samples with CoT, requiring exponential samples without CoT.
This work explores feature learning tradeoffs in neural networks.
problem Resource tradeoffs in neural feature learning.
method Theoretical and experimental investigation of offline sparse parity learning.
result Width improves sample efficiency in sparse feature learning.
PCHID improves sample efficiency in reinforcement learning tasks.
problem Sparse rewards make learning policies difficult in reinforcement learning.
method Hindsight Inverse Dynamics with Hindsight Experience Replay and Policy Continuation.
result PCHID significantly improves sample efficiency and final performance on multi-goal tasks.
CCLF improves sample efficiency in RL by prioritizing informative samples.
problem Learning from high-dimensional observations is challenging and unstable.
method Contrastive-Curiosity-Driven Learning Framework (CCLF) that prioritizes informative samples.
result CCLF improves sample efficiency and learning performance in RL.
Improving sample efficiency has been a longstanding goal in reinforcement learning. This paper proposes VRMPO algorithm: a sample efficient policy gradient method with stochastic mirror descent. In VRMPO, a novel variance-reduced policy gradient estimator is presented to improve sample efficiency.…
New BE dimension measure reveals rich RL problems with sample-efficient algorithms.
problem Finding sample-efficient algorithms for complex RL problems.
method Introducing Bellman Eluder (BE) dimension and designing GOLF and OLIVE algorithms.
result GOLF and OLIVE algorithms learn near-optimal policies for low BE dimension problems with polynomial samples.
This paper tackles sample-efficient reinforcement learning for partially observable Markov games.
problem Learning in partially observable Markov games with incomplete information.
method A simple algorithm combining optimism and Maximum Likelihood Estimation (MLE) for self-play, and a variant of optimistic MLE for adversarial opponents.
result The proposed algorithms achieve approximate Nash, correlated, and coarse correlated equilibria in polynomial samples for weakly revealing POMGs.
Active testing reduces label costs for efficient model evaluation.
problem Real-world applications require expensive test labels, disconnecting from existing model evaluation methods.
method Derives acquisition strategies to select test points efficiently, addressing label bias and variance.
result Active testing improves model evaluation efficiency without sacrificing accuracy.
Compact neural networks improve reinforcement learning performance with less data.
problem Sample inefficiency in deep reinforcement learning.
method Tensor factorization and wavelet scattering for compact representation.
result Achieved 2-10 times fewer total coefficients with comparable performance.
PixL2R maps natural language to pixel-based rewards for RL, improving sample efficiency.
problem Sparse reward settings in RL limit applicability to complex problems.
method Directly maps natural language descriptions to pixel-based rewards for guiding RL.
result Language-based rewards significantly improve sample efficiency in policy learning.
Q-chunking improves RL for long tasks by chunking actions.
problem Improving sample efficiency and exploration in offline-to-online RL.
method Action chunking in TD-based RL methods.
result Q-chunking outperforms prior methods on long-horizon tasks.
Maximize to Explore integrates RL components for efficient policy discovery.
problem Balancing exploration and exploitation in online RL with general function approximators.
method Integrates estimation, planning, and exploration into a single objective function.
result Achieves sublinear regret for MDPs and MGs with general function approximations.