Improved sample efficiency in reinforcement learning with deep Gaussian processes.
problem Efficiently learn to control actions with limited interaction data.
method Deep Gaussian processes that simulate dynamics with depth and prior knowledge.
result Significantly improved early sample efficiency across various tasks, including half-cheetah control.
This work shows synthetic gradients can outperform backpropagation in sample efficiency.
problem The efficiency of backpropagation in training neural networks.
method Unified vectorized feedback framework for loss-based and reward-based learning, introducing synthetic gradients.
result Synthetic gradients can achieve lower gradient-estimation mean squared error than backpropagation under certain conditions.
Improved sample efficiency with normalized RBF kernels in neural networks.
problem Learning more with less data in deep learning models.
method Two-phase method to train neural networks with normalized RBF kernels as output layer.
result Normalized RBF kernel networks achieve higher sample efficiency, compactness, and separability.
This work improves sample efficiency in meta-learning for nonlinear tasks.
problem Learning complex tasks efficiently with limited data.
method Subspace-based representations for nonlinear tasks.
result Subspace-based representations can be learned efficiently and improve future task performance.
Enhanced feedback model improves sample-efficiency in POMDPs.
problem Exponential hardness of learning in POMDPs.
method Multiple observations in hindsight feedback model.
result Sample-efficient learning possible for new subclasses of POMDPs.
Novelty search in low-dimensional space improves sample efficiency in exploration tasks.
problem Efficient exploration in complex environments with sparse rewards.
method Combines model-based and model-free objectives to learn a low-dimensional representation. Uses intrinsic novelty rewards based on nearest neighbor distances in this space.
result Our approach achieves more sample-efficient exploration compared to strong baselines on various tasks.
Improves sample efficiency and generalization in vision-based RL by enhancing exploration.
problem Low sample efficiency in vision-based RL using images as observations.
method Integrates state representation learning to enhance exploration and sample diversity.
result Significant improvement in sample efficiency across various environments.
Deep neuroevolution, that is evolutionary policy search methods based on deep neural networks, have recently emerged as a competitor to deep reinforcement learning algorithms due to their better parallelization capabilities. However, these methods still suffer from a far worse sample efficiency. In this paper we invest…
Improved agnostic boosting with better sample efficiency.
problem Agnostic boosting's poor sample efficiency compared to Empirical Risk Minimization.
method Leverages sample reuse across rounds, guarantees better generalization.
result Substantially more sample-efficient agnostic boosting algorithm.
Automates RL with sample-efficient hyperparameter optimization.
problem Challenges in applying deep RL due to hyperparameter sensitivity and inefficiency.
method Population-based AutoRL framework for meta-optimizing RL algorithms and architectures.
result Reduces the number of environment interactions needed for meta-optimization by up to an order of magnitude.
Sample efficiency is a crucial problem in deep reinforcement learning. Recent algorithms, such as REDQ and DroQ, found a way to improve the sample efficiency by increasing the update-to-data (UTD) ratio to 20 gradient update steps on the critic per environment sample. However, this comes at the expense of a greatly inc…
Improves sample efficiency in reinforcement learning with input representation.
problem Poor sample efficiency in reinforcement learning.
method Attention-based method to project inputs into an invariant representation space.
result Representation space is m! smaller for inputs of m objects, improving sample efficiency. Paper tackles sample-efficient offline RL, proposing data diversity and unified algorithms.
problem Sample-efficient learning from historical data for sequential decision-making.
method Proposes data diversity and unifies three offline RL algorithm classes: VS, RO, and PS.
result Comparable sample efficiency for VS, RO, and PS algorithms under standard assumptions.
Improves sample efficiency in RL by matching model-based gradients.
problem Lack of sample efficiency in model-free RL.
method Gradient matching between model-based and model-free RL components.
result Improves sample efficiency without increasing asymptotic bias.
New method improves sample-efficiency in neural posterior estimation using simulator gradients.
problem High-fidelity posterior estimation with complex physical simulations is time-consuming.
method Neural Posterior Estimation (NPE) with differentiable simulators and gradient information.
result Improves sample-efficiency in posterior density estimation.
DR-MCTS improves decision quality and sample efficiency in complex environments.
problem Improving decision quality and sample efficiency in complex environments.
method Integrates Doubly Robust off-policy estimation into Monte Carlo Tree Search (MCTS).
result DR-MCTS achieves superior performance in Tic-Tac-Toe and VirtualHome tasks.
HySRL improves RL sample efficiency with shifted-dynamics data.
problem Leveraging historical data with shifted dynamics to improve sample efficiency in RL.
method HySRL, a hybrid transfer RL algorithm that uses prior information on dynamics shift to achieve better sample complexity.
result HySRL achieves problem-dependent sample complexity and outperforms pure online RL.
Deep neuroevolution and deep reinforcement learning (deep RL) algorithms are two popular approaches to policy search. The former is widely applicable and rather stable, but suffers from low sample efficiency. By contrast, the latter is more sample efficient, but the most sample efficient variants are also rather unstab…
Improving sample efficiency has been a longstanding goal in reinforcement learning. This paper proposes VRMPO algorithm: a sample efficient policy gradient method with stochastic mirror descent. In VRMPO, a novel variance-reduced policy gradient estimator is presented to improve sample efficiency.…
New RL approach tackles non-linear MDPs without linear assumptions.
problem Sample efficiency in RL for complex, nonlinear MDPs with continuous states.
method Introduces EPW condition to relax linear structure requirements; provides sample-efficient RL algorithm.
result EPW condition allows solving MDPs without linear assumptions, including Atari games.
Paper tackles sample-efficient RL for linearly realizable MDPs with limited revisiting.
problem Sample-efficient reinforcement learning for linearly realizable MDPs with limited revisiting.
method Develops a new sampling protocol that allows for backtracking and revisiting states in a controlled manner.
result Achieves polynomial sample complexity scaling with feature dimension, horizon, and inverse sub-optimality gap.
Abstractor enhances Transformers for relational reasoning, improving sample efficiency and performance.
problem Improving sample efficiency and performance in relational tasks.
method Introduces Abstractor module with relational cross-attention to enable explicit relational reasoning.
result Dramatic improvements in sample efficiency and performance on various relational tasks.
Parallelized Bayesian quadrature improves sample efficiency and inference.
problem Efficient Bayesian inference and model evidence calculation.
method Batch Bayesian quadrature with kernel recombination for parallel sampling.
result Empirically, outperforms state-of-the-art methods in various datasets.
A new diffusion model uses efficient conditional estimators for discrete data.
problem Efficient estimation of conditional probabilities for discrete data.
method Discrete denoising diffusion framework with sample-efficient NeurISE conditional estimation.
result The method outperforms existing approaches in various metrics on binary and scientific data.
Catalyst.RL accelerates RL research with efficient training.
problem Efficient reinforcement learning training in complex environments.
method Open-source PyTorch framework with distributed training and RL algorithms.
result Catalyst.RL achieved 2nd place in a computationally expensive RL challenge.
Imitation learning is the problem of recovering an expert policy without access to a reward signal. Behavior cloning and GAIL are two widely used methods for performing imitation learning. Behavior cloning converges in a few iterations but doesn't achieve peak performance due to its inherent iid assumption about the st…
Enhances BO in high dimensions with Newton methods.
problem Challenges in scaling BO to high-dimensional spaces.
method Construct multiple local quadratic models using gradients and Hessians from a global GP, and select new sample points by solving bound-constrained quadratic programs.
result Outperforms existing high-dimensional BO techniques on synthetic and real-world applications.
Paper improves sample efficiency of transfer learning in diffusion models.
problem Diffusion models need too much data to train from scratch.
method Assumes shared low-dimensional representation across tasks for improved sample efficiency.
result Sample complexity of target tasks can be reduced with a well-learned representation.
A new method speeds up sampling in diffusion models.
problem Slow sample generation in diffusion models.
method Proposed Splitting Integrators for fast stochastic sampling.
result Achieved FID score of 2.36 in 100 NFE, significantly faster than baselines.
This paper advances sample-efficient learning for partially observable RL by introducing B-stability and new algorithms.
problem Hard sample complexity for learning near-optimal policies in partially observable RL.
method Proposes B-stability as a unified structural condition and develops new algorithms for sample-efficient learning.
result Any B-stable PSR can be learned with polynomial samples, improving over current best complexities.
CoT improves transformer sample efficiency by reducing input token dependencies and attention sparsity.
problem Transformer sample inefficiency in simple tasks.
method Demonstrated through parity-learning setup, showing CoT reduces required samples from exponential to polynomial.
result Transformer learns function within polynomial samples with CoT, requiring exponential samples without CoT.
We propose a semantic segmentation model that exploits rotation and reflection symmetries. We demonstrate significant gains in sample efficiency due to increased weight sharing, as well as improvements in robustness to symmetry transformations. The group equivariant CNN framework is extended for segmentation by introdu…
A new method uses a frozen language model to improve sample efficiency in reinforcement learning.
problem Improving sample efficiency in reinforcement learning with partially observable environments.
method FROZEN Hopfield network and HELM (History Embedding Language Model) method.
result HELM achieves new state-of-the-art results on Minigrid and Procgen environments.
Maximize to Explore integrates RL components for efficient policy discovery.
problem Balancing exploration and exploitation in online RL with general function approximators.
method Integrates estimation, planning, and exploration into a single objective function.
result Achieves sublinear regret for MDPs and MGs with general function approximations.
DAC enhances exploration in reinforcement learning with entropy regularization.
problem Improving exploration efficiency in reinforcement learning.
method Sample-aware entropy regularization using replay buffer action distributions.
result DAC significantly outperforms existing algorithms in reinforcement learning tasks.
New framework improves sample efficiency and robustness in RL with smooth policies.
problem Sample inefficiency and lack of robustness in deep reinforcement learning.
method SR^2L framework, smoothness-inducing regularization.
result Improved sample efficiency and robustness in both on-policy and off-policy RL algorithms.
Improved sample-efficient learning for non-coherent digital jamming.
problem Learning optimal jamming strategies in non-coherent digital modulation schemes without prior knowledge.
method Introduced a linear bandit algorithm that accounts for action similarities and integrates context features.
result Significantly improved convergence behavior compared to prior art.
Improved reinforcement learning in Minecraft with human demonstrations.
problem Sample inefficiency in reinforcement learning.
method Training policy networks on human demonstrations first, then fine-tuning with reinforcement learning.
result Best agent achieved a mean score of 48 in Minecraft.
Deep networks have enabled reinforcement learning to scale to more complex and challenging domains, but these methods typically require large quantities of training data. An alternative is to use sample-efficient episodic control methods: neuro-inspired algorithms which use non-/semi-parametric models that predict valu…
To facilitate research in the direction of sample efficient reinforcement learning, we held the MineRL Competition on Sample Efficient Reinforcement Learning Using Human Priors at the Thirty-third Conference on Neural Information Processing Systems (NeurIPS 2019). The primary goal of this competition was to promote the…
Active testing reduces label costs for efficient model evaluation.
problem Real-world applications require expensive test labels, disconnecting from existing model evaluation methods.
method Derives acquisition strategies to select test points efficiently, addressing label bias and variance.
result Active testing improves model evaluation efficiency without sacrificing accuracy.
Latent MoS learns multiple symmetries for efficient dynamic learning.
problem Efficiently learning dynamics from limited system measurements.
method Latent Mixture of Symmetries (Latent MoS) with hierarchical architecture.
result Latent MoS outperforms baselines in interpolation and extrapolation tasks.
New BE dimension measure reveals rich RL problems with sample-efficient algorithms.
problem Finding sample-efficient algorithms for complex RL problems.
method Introducing Bellman Eluder (BE) dimension and designing GOLF and OLIVE algorithms.
result GOLF and OLIVE algorithms learn near-optimal policies for low BE dimension problems with polynomial samples.
New method breaks symmetry in neural networks, improving sample efficiency.
problem Symmetry in neural networks limits their ability to learn unique features.
method Introduces 'relaxed equivariance' to overcome symmetry limitations.
result Equivariant multilayer perceptrons (E-MLPs) can now break symmetry at the sample level.
New coverage conditions improve sample efficiency in online reinforcement learning.
problem Improving sample efficiency in online reinforcement learning with function approximation.
method Identifying and studying new coverage conditions for online reinforcement learning.
result Improved regret bounds achieved with new coverage conditions.
A new correction term improves sample efficiency in deep reinforcement learning.
problem Momentum accumulation in TD learning leads to doubly stale gradients.
method Proposed a correction term to address the issue of doubly stale gradients.
result Improves sample efficiency in policy evaluation.
WMPG reduces policy gradient variance using world models.
problem Reducing variance in policy gradient estimates.
method Trains a world model online to estimate policy gradients and uses imagined trajectories as a baseline.
result WMPG achieves better sample efficiency compared to AC and MAC.
Reinforcement Learning (RL) algorithms can suffer from poor sample efficiency when rewards are delayed and sparse. We introduce a solution that enables agents to learn temporally extended actions at multiple levels of abstraction in a sample efficient and automated fashion. Our approach combines universal value functio…