Multi-task learning improves robotic control in continuous action spaces.
problem Robotic control in continuous action spaces lacks effective multi-task learning methods.
method Applied multi-task learning methods to continuous action spaces and compared performance with baselines.
result Multi-task learning outperforms baselines and alternative methods in continuous control tasks.
A new policy gradient estimator reduces variance for clipped actions in continuous control tasks.
problem Policy gradient methods struggle with bounded action spaces.
method Proposes a new policy gradient estimator that accounts for clipped actions.
result The new estimator achieves lower variance and outperforms conventional methods.
Alpha Zero adapts to continuous action spaces for real-world tasks.
problem Real-world reinforcement learning domains often have continuous action spaces.
method Interleaves tree search and deep learning, extending Alpha Zero for continuous action spaces.
result Preliminary experiments on the Pendulum task show feasibility of the approach.
DiGrad improves multi-task reinforcement learning in robotic systems.
problem Efficient multi-task reinforcement learning in complex robotic systems with shared actions.
method Differential Policy Gradient (DiGrad) for simultaneous training of multiple tasks in a single actor-critic network.
result DiGrad outperforms related methods in continuous action spaces, supporting efficient multi-task learning.
Hybrid SAC improves RL for video games with discrete, continuous actions.
problem Improving RL performance in video games with practical constraints.
method Extension of Soft Actor-Critic (SAC) for handling discrete, continuous, and parameterized actions.
result Hybrid SAC successfully solves a high-speed driving task and is competitive on parameterized actions benchmarks.
New STDP rule for spiking neurons solves discrete action reinforcement learning tasks.
problem Applying standard STDP to discrete action reinforcement learning tasks.
method Feedback-modulated TD-STDP learning rule for spiking neuron networks.
result Feedback modulation improves credit assignment in reinforcement learning.
AQL uses amortized inference to handle high-dimensional action spaces in Q-learning.
problem Difficulty in maximizing over large action spaces in Q-learning.
method Replace expensive maximization over all actions with a maximization over a small subset sampled from a learned proposal distribution.
result AQL outperforms existing methods on continuous control tasks with up to 21 dimensional actions.
Adapts Deep Q-Learning for continuous control tasks.
problem Continuous control in reinforcement learning.
method Actor-critic, model-free algorithm based on policy gradient.
result Robustly solves over 20 simulated physics tasks.
New methods analyze continuous strategic games, overcoming typical assumptions.
problem Analyzing rich, naturalistic behavioral data.
method Inverse reinforcement learning, machine learning models.
result Successfully generated rich, varied trajectories without mode collapse.
New method predicts continuous actions using sequence models.
problem High-dimensional continuous control problems.
method Sequence-to-sequence models for discretized action spaces.
result Achieves state-of-the-art performance in continuous control tasks.
Enhances SAC for better sample efficiency in continuous-action tasks.
problem Improving sample efficiency in soft actor-critic algorithms.
method Integrating Emphasizing Recent Experience (ERE) with Soft Actor-Critic (SAC) and Priority Experience Replay (PER).
result ERE significantly improves sample efficiency compared to vanilla SAC, especially for continuous-action tasks.
MO2 learns useful behaviours from past experience for new tasks.
problem Discovering useful behaviours from past experience and transferring them to new tasks.
method Model-Based Offline Options (MO2) framework supporting sample-efficient bottleneck option discovery over continuous state-action spaces.
result MO2 outperforms recent option learning methods on complex long-horizon continuous control tasks.
New method improves policy gradient performance in continuous control tasks.
problem Improving policy gradient methods for continuous control tasks.
method Numerical integration approach to all-action policy gradient.
result Improved performance and sample efficiency in continuous control tasks.
This paper tackles reinforcement learning in large action spaces, presenting a method that embeds actions in a continuous space and uses approximate nearest-neighbor methods.
problem Current reinforcement learning methods struggle with environments having large numbers of discrete actions, making them inapplicable to many real-world tasks.
method The approach embeds actions in a continuous space and uses approximate nearest-neighbor methods for efficient training.
result The proposed method enables reinforcement learning to be applied to large-scale learning problems previously intractable with current methods.
Robot learns multiple tasks hierarchically by transferring knowledge.
problem Learning multiple complex tasks in open-ended environments.
method Task-oriented procedures, goal-babbling, imitation learning, active learning, intrinsic motivation.
result Robots can learn complex tasks more efficiently by transferring knowledge from simpler ones.
POSA optimizes policy gradients by reducing variance with RB and CV.
problem High variance in policy gradient estimators for high-dimensional control tasks.
method POSA uses ASDG estimator combining RB and CV, learning action space factorization.
result POSA improves performance on high-dimensional control tasks.
Action chunking and data exploration improve behavior cloning in robotics.
problem Exponential errors in learning from demonstrations for continuous control tasks.
method Action chunking and exploratory data collection.
result Control-theoretic stability is key to improving imitation learning.
Robot learns new actions and plans complex tasks efficiently.
problem Efficiently plan and execute complex tasks with limited sensorimotor primitives.
method Active model learning and diverse action sampling using Gaussian process methods and adaptive sampling.
result Improved planning and execution of long-horizon tasks.
New algorithms tackle multi-agent problems with hybrid action spaces.
problem Applying deep reinforcement learning to multi-agent problems with discrete-continuous hybrid action spaces.
method Proposed two novel algorithms: Deep MAPQN and Deep MAHHQN, using centralized training and decentralized execution.
result Empirical results show both algorithms significantly outperform existing methods.
A new method for RL with continuous actions improves stability and scalability.
problem Stability and scalability issues in existing RL methods.
method Soft policy gradient with entropy regularization, combined with double sampling for soft Bellman equation.
result Outperforms off-policy prior methods in continuous action RL tasks.
MAGE optimizes policies using action gradients from model-based learning.
problem Lack of direct gradient information from critics in actor-critic methods.
method Model-based actor-critic algorithm that learns action-value gradient.
result MAGE outperforms model-free and model-based baselines on continuous control tasks.
Proposes a value-based method for continuous control without an actor.
problem Computational infeasibility of evaluating Q-values in continuous action spaces.
method Structurally maximizable Q-functions, actor-free approach.
result Performance and sample efficiency comparable to actor-critic methods.
Enhances reinforcement learning safety through risk-averse exploration.
problem Safety concerns in reinforcement learning due to sub-optimal actions.
method Distributionally robust policy iteration scheme with lower bound guarantees.
result Efficient algorithm that prevents poor decisions and converges to optimal policy.
VIME improves RL exploration in high-dimensional spaces.
problem Scalable and effective exploration in high-dimensional deep RL.
method Variational Information Maximizing Exploration (VIME) using variational inference in Bayesian neural networks.
result Significantly better performance compared to heuristic methods across various tasks and algorithms.
Optimistic Actor-Critic improves exploration efficiency in reinforcement learning.
problem Poor sample efficiency in existing actor-critic methods.
method Introduces Optimistic Actor-Critic, approximating upper and lower bounds on state-action value function.
result Achieves state-of-the-art sample efficiency in challenging continuous control tasks.
SDPG algorithm improves sample efficiency and reward in DRL for continuous action spaces.
problem Improving sample efficiency and reward in distributional reinforcement learning for continuous action spaces.
method SDPG algorithm models return distribution using samples via reparameterization technique.
result SDPG shows better sample efficiency and higher reward in OpenAI Gym environments.
Paper introduces SALE for better state-action learning in RL.
problem Challenges in representation learning for low-level states in RL.
method Introduces SALE, a novel approach for learning embeddings of state-action interactions.
result TD7 algorithm significantly outperforms existing continuous control algorithms.
Agent uses message passing to optimize robot navigation, balancing exploration and exploitation.
problem Optimizing robot navigation in continuous-valued spaces with uncertainty.
method Autoregressive active inference agent using message passing on a factor graph.
result Agent modulates action based on predictive uncertainty, leading to better model of dynamics.
Unified family of estimators for bounded action spaces reduces variance in policy gradients.
problem High variance in policy gradients for bounded action spaces.
method Marginal policy gradients family of estimators for directional control.
result APG estimator offers substantial improvement over standard policy gradient.
Paper introduces a meta-critic for accelerating off-policy actor-critic learning.
problem Improving sample efficiency in continuous control tasks.
method Meta-critic that meta-learns an additional loss for the actor.
result Online meta-critic learning leads to improved performance in various continuous control environments.
CLIP dataset helps extract action items from hospital discharge notes.
problem Lost action items in long discharge notes hinder information sharing.
method Created CLIP dataset of annotated discharge notes, used multi-aspect extractive summarization, trained models on pre-trained language models and context.
result Best models improved by incorporating context and pre-trained language models.
Self-organized action hierarchy and compositionality learned by RNNs.
problem Improving RNN architectures for reinforcement learning.
method Multiple-timescale, stochastic RNN for RL.
result Network autonomously learns sub-goals and develops an action hierarchy.
USAC balances pessimism and optimism in actor-critic training for better exploration and performance.
problem Excessive pessimism limits exploration, while excessive optimism leads to high-risk behaviors.
method Utility Soft Actor-Critic (USAC) dynamically adapts exploration based on critic uncertainty.
result USAC consistently outperforms state-of-the-art algorithms in continuous control tasks.
Unified visual environment for rapid task learning and switching.
problem Rapid and flexible learning of new tasks in a variety of environments.
method Unified visual interaction environment, reward map prediction, modular architecture, dynamic neural voting scheme.
result Modules with specific design principles outperform standard neural networks, requiring fewer training examples and neurons.
HiDe learns hierarchical control for complex tasks by separating planning and control.
problem Solving long horizon control tasks with generalization to unseen scenarios.
method Functional decomposition of state-action spaces, RL-based planner, modular transfer of policy layers.
result Generalizes across unseen test environments and scales to longer horizons.
Paper tackles continual reinforcement learning by forgetting, proposing a planning method with online world models.
problem Catastrophic forgetting in reinforcement learning when learning new tasks.
method Planning with an online world model using model predictive control.
result The proposed FTL Online Agent (OA) learns new tasks without forgetting old skills.
RL agent learns to smoothly change lanes in a dynamic driving environment.
problem Challenging lane change control with safety and comfort.
method Formulated continuous action for lane change in DDPG algorithm, defined reward function for learning.
result Successfully changed lanes with 100% success rate in diverse driving situations.
ARPs improve exploration and sample efficiency in continuous control tasks.
problem Limited exploration in continuous control tasks leading to low sample efficiency.
method Introduce autoregressive policies (ARPs) with temporally coherent standard normal distributions.
result ARPs enhance exploration and sample efficiency in both simulated and real-world domains.
New algorithm learns optimal actions in complex decision problems.
problem Optimal policy learning in continuous Markov decision problems.
method Nonparametric stochastic compositional gradient descent in RKHS.
result Algorithm converges to optimal policies with low Bellman error.
Weakly-supervised RL identifies meaningful tasks, improving performance in complex environments.
problem Learning to efficiently explore and distinguish between meaningful and irrelevant tasks.
method Weak supervision to automatically disentangle meaningful tasks from a large space of nonsensical tasks.
result The learned subspace of meaningful tasks leads to substantial performance gains, especially in complex environments.
Deep network predicts action sequences for complex tasks from a scene image.
problem Scalable task and motion planning from initial scene images.
method Deep convolutional recurrent neural network that predicts action sequences.
result Predicts promising action sequences, reducing motion planning problems.
A new framework handles hybrid action spaces in reinforcement learning.
problem Handling reinforcement learning with both discrete and continuous actions.
method Parametrized Deep Q-Networks (P-DQN) framework integrating DQN and DDPG.
result Empirical validation of efficiency and effectiveness in simulated RoboCup soccer and game King of Glory.
Develops DPG methods for continuous-time RL with deterministic policies.
problem High variance and slow convergence in stochastic policy RL methods.
method Derives continuous-time policy gradient formula and proposes CT-DDPG algorithm.
result CT-DDPG achieves superior stability and faster convergence in continuous-time RL.
QAM uses adjoint matching to optimize continuous-action RL policies efficiently.
problem Efficient optimization of expressive diffusion or flow-matching policies with respect to a Q-function.
method QAM leverages adjoint matching to bypass the numerical instability of backpropagation through multi-step denoising processes.
result QAM consistently outperforms prior approaches on hard, sparse reward tasks in offline and offline-to-online RL.
New algorithm for context bandits with continuous actions.
problem Efficient decision-making with unknown action structures.
method Reduction-style algorithm combining supervised learning.
result Proven to work in general and validated with experiments.
Safe actions learned in finite trials, without infinite exploration.
problem Learning safe actions in unknown environments efficiently.
method Defining a handicap metric and using sequential probability ratio test for discarding unsafe actions.
result Achieves constant handicap, discarding unsafe machines with probability one in finite rounds.
OpenAI Five defeated Dota 2 champions using deep reinforcement learning.
problem Complex, continuous state-action spaces in Dota 2.
method Deep reinforcement learning, distributed training, self-play.
result Superhuman performance in Dota 2 achieved through reinforcement learning.
New method improves data efficiency in reinforcement learning by composing skills.
problem Improving data efficiency in reinforcement learning by composing previously mastered skills.
method Extending policy improvement to maximum entropy framework, introducing successor features, and explicitly learning divergence between base policies.
result Proposes a novel approach that outperforms or matches existing methods in various tasks.