New method improves smoothness of robot learning.
problem Jerky motion patterns on real robots from Deep RL exploration.
method Adapting state-dependent exploration to Deep RL algorithms with gSDE.
result Improved exploration and performance on real robots.
RL controls small soccer robots in a real league, beating human-designed policies.
problem Training robots to play complex, real-world sports.
method Sim-to-Real RL approach, training in simulated environment, applying to real-world robots.
result Robots learned policies to compete effectively, beating human-designed strategies.
Novel approach for sim-to-real transfer using MPC and task representations.
problem Difficulty of sim-to-real transfer systems producing generalizable policies.
method Model-predictive control (MPC) and task representation learning.
result Direct transfer of multi-skill policy to real robot for unseen tasks.
Prototype real-world RL environment for robotics training.
problem Lack of common real-world RL benchmark.
method Developed OffWorld Gym with open access to real-world robotics environments.
result Baseline results in navigation task on real and simulated terrain.
Learning to control robots directly based on images is a primary challenge in robotics. However, many existing reinforcement learning approaches require iteratively obtaining millions of robot samples to learn a policy, which can take significant time. In this paper, we focus on learning a realistic world model capturi…
This work tackles force control for contact-rich manipulation tasks with rigid robots using RL.
problem Challenges in working with real robotic hardware, especially position-controlled robots.
method Combines RL with traditional force control techniques, implementing parallel position/force control and admittance control.
result Validated methods on both simulation and real robot (UR3 e-series) for force control.
Study benchmarks RL algorithms on real robots, revealing their performance and hyper-parameter sensitivity.
problem Lack of benchmark tasks and source code for reinforcement learning on physical robots.
method Introduced benchmark tasks with multiple robots, tested 4 RL algorithms, analyzed hyper-parameter sensitivity.
result Some RL implementations can be applied to physical robots with proper setup, but hyper-parameters need re-tuning.
Survey examines challenges and solutions in sim-to-real transfer for robotics.
problem Challenges in transferring robotic systems from simulation to real-world environments.
method Leveraging techniques like domain randomization, real-to-sim transfer, state and action abstractions, and sim-real co-training.
result Promising results in closing the reality gap across various robotic domains.
A new framework for robot block-stacking tasks using causal probabilistic models.
problem Robots fail outside controlled environments due to uncertainty and lack of explicit design for all scenarios.
method Causal probabilistic framework combining causal models and probabilistic representations of noise.
result Robots can perceive, reason about, and explain their environment for block-stacking tasks.
Robots detect and recognize objects in real-time for better manipulation.
problem Real-time object detection and recognition for humanoid robots.
method Modified YOLOv3 algorithm, quantization, and re-arrangement of layers for low-compute NAO robots.
result Robots can perform real-time detection, recognition, and localization of objects.
Reinforcement learning is a promising approach to developing hard-to-engineer adaptive solutions for complex and diverse robotic tasks. However, learning with real-world robots is often unreliable and difficult, which resulted in their low adoption in reinforcement learning research. This difficulty is worsened by the …
Deep model predicts robot trajectories in real time.
problem Real-time robot trajectory prediction with low latency.
method Deep conditional generative model trained with SGDVB.
result More accurate long-term predictions with lower latency.
Robot learns agile leg movements from simulations to real robots.
problem Training legged robots for dynamic maneuvers is challenging and expensive.
method Simulation-based reinforcement learning for real legged systems.
result ANYmal robot can follow high-level commands, run faster, and recover from falls.
Robustified controllers reduce fine-tuning time for sim-to-real transfer learning.
problem Reducing fine-tuning time for sim-to-real transfer learning of complex robotic tasks.
method Learn robustified controllers in simulation by changing parameters for successive episodes.
result Fine-tuning time is substantially reduced for robustified controllers.
This work tackles real-world robotic reinforcement learning challenges.
problem Limited success of reinforcement learning in real-world robotics.
method Proposes a system for autonomous real-world learning without instrumentation.
result Demonstrates a complete system that learns without human intervention.
Method trains vision and control policies on real robots quickly.
problem Training vision-based control policies on real robots efficiently.
method Multi-task Reinforcement Learning with auxiliary tasks.
result Significant learning speed-ups and task learning from-scratch.
Deep RL learns robot walking gaits in real-world environments.
problem Difficulty in applying deep RL to real-world robotic tasks due to poor sample complexity and hyperparameter sensitivity.
method Sample-efficient deep RL algorithm based on maximum entropy RL, requiring minimal per-task tuning and modest trials.
result Acquired stable walking gaits on a real-world Minitaur robot in about two hours.
A new optimizer d-AmsGrad improves deep learning for robot learning in non-stationary problems.
problem Noise and outliers in real-world data make deep learning challenging for robot learning.
method Proposed an improved version of AmsGrad optimizer that slowly decays the maximum second momentum to adapt to non-stationary problems.
result The new optimizer outperformed baseline optimizers in robotics problems.
Method learns diverse robot skills for complex tasks.
problem Simulation-to-real transfer for robot skills.
method Decomposes skills, learns parameterized embeddings, and composes them.
result Transferable high-level policies using low-level skills.
This work evaluates task-agnostic exploration methods for fixed-batch learning.
problem Expensive real-world experience for robotics tasks.
method Fixed datasets for arbitrary task learning.
result Improved offline learning for robotics tasks.
Central to robot exploration and mapping is the task of persistent localization in environmental fields characterized by spatially correlated measurements. This paper presents a Gaussian process localization (GP-Localize) algorithm that, in contrast to existing works, can exploit the spatially correlated field measurem…
Detects anomalies in autonomous mobile robots using vision.
problem Anomaly detection for autonomous mobile robots.
method Unsupervised deep learning methods and a novel dataset.
result State-of-the-art approach tested on a new dataset.
This research introduces an autonomous robot navigation method using reinforcement learning.
problem Improving robot navigation in complex environments.
method Deep Q Network (DQN) and Proximal Policy Optimization (PPO) models for path planning and decision-making.
result The models enhance robot navigation ability and adaptive learning in unknown environments.
A robot learns environmental fields using physics-based models and Bayesian methods.
problem Accurately learning complex environmental fields from limited robot measurements.
method Bayesian framework with Gaussian processes to select and update physics-based models in real-time.
result The robot's learned flow field approximates real flow better than prior solutions and data-driven methods.
Optimizes synthetic image augmentation for sim2real policy transfer in robotics.
problem Difficulty in transferring learned policies from simulated to real environments.
method Optimizes random transformations to augment synthetic images, enabling policy learning without real data.
result Significant improvement in policy accuracy on real robots for three manipulation tasks.
Automates decision-making for human operators managing multiple robots.
problem Limited human operator attention when controlling multiple robots.
method Learned model of user preferences from easy settings to automatically identify the most critical robot.
result Automated decision-making can assist human operators in managing more robots than their attention allows.
Robot learns to juggle two balls from 56 minutes of experience.
problem Learning high-acceleration tasks in the real-world with binary rewards.
method Designs a learning system that incorporates safety and sample efficiency.
result High-speed manipulator learns to juggle for up to 33 minutes.
This work analyzes how multi-agent reinforcement learning can bridge the gap to reality in distributed multi-robot systems.
problem Collaborative learning in distributed multi-robot systems with varying sensors and actuators.
method Simulation-based analysis using PPO and Bullet physics engine, considering different types of perturbations.
result PPO's robustness is affected by the presence of different types of perturbations and the number of agents experiencing them.
Robotic learning without reward engineering from images.
problem Manual reward engineering for reinforcement learning in robotics.
method Learning from examples and active solicitation of labels.
result Efficient learning of robotic skills from images without manual rewards.
Robots navigate wilderness trails using virtual-to-real-world transfer learning.
problem Autonomous navigation of outdoor trails is challenging due to lack of annotated training data.
method Virtual-to-real-world transfer learning with deep learning models trained on synthetic data.
result Classification accuracies of up to 95% on synthetic data and feasibility in real-world trails.
ROBEL platform accelerates reinforcement learning with low-cost robots.
problem Accelerating reinforcement learning research in robotics.
method Open-source platform of cost-effective robots for real-world reinforcement learning.
result Robots D'Claw and D'Kitty facilitate learning dexterous manipulation and agile locomotion tasks.
For a safe, natural and effective human-robot social interaction, it is essential to develop a system that allows a robot to demonstrate the perceivable responsive behaviors to complex human behaviors. We introduce the Multimodal Deep Attention Recurrent Q-Network using which the robot exhibits human-like social intera…
Robot learns new tasks sequentially without forgetting past ones.
problem Teaching a robot to solve tasks in a continual learning scenario.
method Policy distillation and sim2real transfer.
result Robot can solve all encountered tasks without forgetting past ones.
Neural circuit model re-purposed for robotic control tasks.
problem Learning simple robotic control tasks.
method Re-purposing a biological neural circuit model to control robotic tasks using a search-based optimization algorithm.
result Neuronal Circuit Policies (NCPs) perform on par and in some cases surpass contemporary deep learning models with fewer parameters and interpretable dynamics.
Robots learn to navigate rough terrain using reinforcement learning.
problem Generalizing robot behavior to new, unseen rough terrains.
method PPMC RL Training Algorithm
result Robots achieve 100% success rate in learning new rough terrain maps.
New framework learns robot tasks quickly from simplified simulations.
problem Long training times and variable-length inputs in RL.
method Combines deep sets encoding with modular RL.
result Effective policies learned in minutes from simplified simulations.
Robots learn conservatively from human corrections, avoiding unintended changes to their objectives.
problem Robots learn from human corrections but may not align with the intended objectives due to misspecified hypothesis spaces.
method Robots reason in real-time about the relevance of human corrections to their hypothesis space, learning more conservatively.
result Robots can avoid unintended learning from human corrections, improving alignment with intended objectives.
Paper presents a method for efficient robot adaptation using fine-tuning.
problem Continuous adaptation of robot learning systems in real-world scenarios.
method Fine-tuning previously learned policies using off-policy reinforcement learning.
result Fine-tuning leads to substantial performance gains and adaptation to new conditions.
Robot solves Rubik's cube using simulation-trained models.
problem Solving Rubik's cube with a robot hand.
method Automatic domain randomization (ADR) and a custom robot platform.
result Control policies and vision state estimators trained with ADR exhibit improved sim2real transfer.
Robot learns from human demonstrations to work autonomously.
problem Leveraging human and robotic strengths in human-robot systems.
method Bayesian inference for detecting and classifying human heterogeneity.
result Bayesian approach outperforms conventional methods by up to 12.8. Model-free deep reinforcement learning has been shown to exhibit good performance in domains ranging from video games to simulated robotic manipulation and locomotion. However, model-free methods are known to perform poorly when the interaction time with the environment is limited, as is the case for most real-world ro…
Robot learns to control itself without rewards.
problem Autonomous reinforcement learning without access to rewards.
method Distributional planning networks optimizing for an embedding space.
result Learned goal metrics enable autonomous reinforcement learning.
Framework enables robots to learn tasks from unlabeled human instructions.
problem Robotic task learning with unlabeled human instructions.
method TICS architecture combining reward function, human feedback, and unlabeled instructions.
result Framework accelerates task-learning and reduces teaching signals.
A new model for simulating cloth manipulation in robots, accurate to within 1cm.
problem Accurately simulating cloth manipulation in robots, especially in moderate stress environments.
method A continuous, isometric strain model for textiles, treating them as inextensible surfaces with only isometric motions. Aerodynamic effects are incorporated through virtual uncoupling of mass.
result Simulations are accurate to within 1cm compared to real-world manipulation, even with coarse meshes.
The deep reinforcement learning method usually requires a large number of training images and executing actions to obtain sufficient results. When it is extended a real-task in the real environment with an actual robot, the method will be required more training images due to complexities or noises of the input images, …
Hybrid RL combines simulated and real data for robust autonomous flight.
problem Challenges in training deep RL models for real-world robotic tasks.
method Combines real-world and simulated data to improve generalization.
result Quadrotor avoids collisions using only a monocular camera.
Robotic arm learns to manipulate a ball by choosing goals from learned experience.
problem Efficient discovery of skills for long-living autonomous agents without supervision.
method Intrinsically motivated goal exploration using learned goal spaces from deep representation learning.
result Recent results show applicability of learned goal spaces on real-world robotic tasks.
Efficient attacks on DRL models without model access and low computation.
problem Vulnerabilities of DRL models to adversarial attacks.
method Adapting black-box attacks, introducing efficient online sequential attacks, exploring perturbations in environment dynamics, and generating robust physical perturbations.
result Demonstrated the effectiveness of proposed attacks on real-world robots.