A new Q-learning controller improves line follower robot control.
problem Challenges in controlling line follower robots due to unknown mechanical characteristics and uncertainties.
method Simulated annealing based Q learning method to address controller performance issues.
result The proposed controller outperforms conventional P controllers in line follower robots.
Learning robot tasks or controllers using deep reinforcement learning has been proven effective in simulations. Learning in simulation has several advantages. For example, one can fully control the simulated environment, including halting motions while performing computations. Another advantage when robots are involved…
Simulation-to-real transfer is an important strategy for making reinforcement learning practical with real robots. Successful sim-to-real transfer systems have difficulty producing policies which generalize across tasks, despite training for thousands of hours equivalent real robot time. To address this shortcoming, we…
A new model for simulating cloth manipulation in robots, accurate to within 1cm.
problem Accurately simulating cloth manipulation in robots, especially in moderate stress environments.
method A continuous, isometric strain model for textiles, treating them as inextensible surfaces with only isometric motions. Aerodynamic effects are incorporated through virtual uncoupling of mass.
result Simulations are accurate to within 1cm compared to real-world manipulation, even with coarse meshes.
New method improves smoothness of robot learning.
problem Jerky motion patterns on real robots from Deep RL exploration.
method Adapting state-dependent exploration to Deep RL algorithms with gSDE.
result Improved exploration and performance on real robots.
Automates decision-making for human operators managing multiple robots.
problem Limited human operator attention when controlling multiple robots.
method Learned model of user preferences from easy settings to automatically identify the most critical robot.
result Automated decision-making can assist human operators in managing more robots than their attention allows.
Survey examines challenges and solutions in sim-to-real transfer for robotics.
problem Challenges in transferring robotic systems from simulation to real-world environments.
method Leveraging techniques like domain randomization, real-to-sim transfer, state and action abstractions, and sim-real co-training.
result Promising results in closing the reality gap across various robotic domains.
Robot solves Rubik's cube using simulation-trained models.
problem Solving Rubik's cube with a robot hand.
method Automatic domain randomization (ADR) and a custom robot platform.
result Control policies and vision state estimators trained with ADR exhibit improved sim2real transfer.
Efficiently uses imitation, adaptation, and simulation to improve robot learning.
problem Improving efficiency in data collection and curation for autonomous systems.
method Utilizes imitation learning, domain adaptation, and transfer from simulation.
result Increased efficiency in data-driven approaches for robot learning.
Legged robots pose one of the greatest challenges in robotics. Dynamic and agile maneuvers of animals cannot be imitated by existing methods that are crafted by humans. A compelling alternative is reinforcement learning, which requires minimal craftsmanship and promotes the natural evolution of a control policy. Howeve…
A new framework for robot block-stacking tasks using causal probabilistic models.
problem Robots fail outside controlled environments due to uncertainty and lack of explicit design for all scenarios.
method Causal probabilistic framework combining causal models and probabilistic representations of noise.
result Robots can perceive, reason about, and explain their environment for block-stacking tasks.
A novel RL approach learns robotic manipulation without human demonstrations.
problem Learning robotic manipulation policies efficiently and effectively.
method Introducing simulated locomotion demonstration rewards (SLDRs) to enable RL learning.
result The approach achieves higher success rates and faster learning compared to alternatives.
TACTO simulates high-resolution touch sensing for robotics.
problem Accurate simulation of touch sensing in robotics.
method Fast, flexible, open-source simulator for vision-based tactile sensors.
result Demonstrated TACTO's effectiveness in grasping stability prediction and marble manipulation control.
We present a novel solution to the problem of simulation-to-real transfer, which builds on recent advances in robot skill decomposition. Rather than focusing on minimizing the simulation-reality gap, we learn a set of diverse policies that are parameterized in a way that makes them easily reusable. This diversity and p…
Neuro-robotics model shows gestures improve counting ability.
problem Improving counting ability in robots and children.
method Introduces a neuro-robotics model trained with pointing data, tests various model and training conditions.
result Model's performance aligns with human children's learning to count.
This work analyzes how multi-agent reinforcement learning can bridge the gap to reality in distributed multi-robot systems.
problem Collaborative learning in distributed multi-robot systems with varying sensors and actuators.
method Simulation-based analysis using PPO and Bullet physics engine, considering different types of perturbations.
result PPO's robustness is affected by the presence of different types of perturbations and the number of agents experiencing them.
Paper tackles non-uniform coverage planning for robots.
problem Non-uniform coverage planning for robots that need to visit some points more frequently.
method Proposes a novel reinforcement learning approach in a Semi-Markov Decision Process.
result Significant improvement over existing greedy approach in simulations.
Through many recent successes in simulation, model-free reinforcement learning has emerged as a promising approach to solving continuous control robotic tasks. The research community is now able to reproduce, analyze and build quickly on these results due to open source implementations of learning algorithms and simula…
Semi-parametric models improve robot dynamics modeling accuracy.
problem Improving inverse dynamics model accuracy in robotics.
method Comparison of semi-parametric Gaussian process regression and a novel model-based neural network.
result Semi-parametric Gaussian process regression yields the most accurate models.
Optimizes synthetic image augmentation for sim2real policy transfer in robotics.
problem Difficulty in transferring learned policies from simulated to real environments.
method Optimizes random transformations to augment synthetic images, enabling policy learning without real data.
result Significant improvement in policy accuracy on real robots for three manipulation tasks.
Deep reinforcement learning (deep RL) holds the promise of automating the acquisition of complex controllers that can map sensory inputs directly to low-level actions. In the domain of robotic locomotion, deep RL could enable learning locomotion skills with minimal engineering and without an explicit model of the robot…
Robotics: Rolling robots on a moving platform can be controlled.
problem Controlling the motion of rolling robots atop a moving platform.
method Developed a mathematical model and demonstrated simulations.
result Platform acceleration can control robot's heading and motion.
The design of gaits for robot locomotion can be a daunting process which requires significant expert knowledge and engineering. This process is even more challenging for robots that do not have an accurate physical model, such as compliant or micro-scale robots. Data-driven gait optimization provides an automated alter…
Robotic arm learns to manipulate a ball by choosing goals from learned experience.
problem Efficient discovery of skills for long-living autonomous agents without supervision.
method Intrinsically motivated goal exploration using learned goal spaces from deep representation learning.
result Recent results show applicability of learned goal spaces on real-world robotic tasks.
We present a method for fast training of vision based control policies on real robots. The key idea behind our method is to perform multi-task Reinforcement Learning with auxiliary tasks that differ not only in the reward to be optimized but also in the state-space in which they operate. In particular, we allow auxilia…
ORRB enables fast, customizable rendering of robotics environments.
problem Fast and customizable rendering of robotics environments.
method Based on Unity3d and MuJoCo, optimized for cloud deployment.
result Visual domain randomization for improved simulation.
RL controls small soccer robots in a real league, beating human-designed policies.
problem Training robots to play complex, real-world sports.
method Sim-to-Real RL approach, training in simulated environment, applying to real-world robots.
result Robots learned policies to compete effectively, beating human-designed strategies.
Deep RL trains a robust humanoid push-recovery policy.
problem Training robust humanoid push-recovery policies.
method Model-free Deep Reinforcement Learning.
result Policy learns robust behaviors across the entire body.
Central to robot exploration and mapping is the task of persistent localization in environmental fields characterized by spatially correlated measurements. This paper presents a Gaussian process localization (GP-Localize) algorithm that, in contrast to existing works, can exploit the spatially correlated field measurem…
This work tackles force control for contact-rich manipulation tasks with rigid robots using RL.
problem Challenges in working with real robotic hardware, especially position-controlled robots.
method Combines RL with traditional force control techniques, implementing parallel position/force control and admittance control.
result Validated methods on both simulation and real robot (UR3 e-series) for force control.
Differentiable Algorithm Networks (DAN) enable composable robot learning.
problem Training robots to learn from limited data and imperfect models.
method Composable architecture of neural network modules, each encoding a differentiable robot algorithm and model, trained end-to-end from data.
result DAN modules adapt to one another and compensate for imperfect models and algorithms, achieving best overall system performance.
This work shows how to use simulators to learn efficient exploration in real-world RL.
problem Sample complexity of real-world reinforcement learning.
method Coupling exploratory policies learned in simulators with practical approaches.
result Polynomial sample complexity in real world, exponential improvement over direct sim2real transfer.
The paper proposes a method to learn from both simulation and real-world data.
problem Training autonomous systems in simulation and applying them to real-world environments.
method Balancing samples from simulation and real-world data using a replay buffer.
result The method achieves better performance in real-world tasks compared to training only in simulation.
Deep learning is an established framework for learning hierarchical data representations. While compute power is in abundance, one of the main challenges in applying this framework to robotic grasping has been obtaining the amount of data needed to learn these representations, and structuring the data to the task at ha…
In this paper we present a geometric control law for position and line-of-sight stabilization of the nonholonomic spherical robot actuated by three independent actuators. A simple configuration error function with an appropriately defined transport map is proposed to extract feedforward and proportional-derivative cont…
ORIL learns a reward function from unlabeled data to improve robot learning.
problem Leveraging unlabeled data for robot learning.
method ORIL learns a reward function from demonstrator and unlabeled trajectories, annotates data, and trains an agent via offline reinforcement learning.
result ORIL consistently outperforms BC agents on various robotic tasks.
A new reinforcement learning method for robots thinking and moving simultaneously.
problem Concurrent control in robotic systems where actions must be decided while the system is still evolving.
method Continuous-time Bellman equations, discretization aware of system delays, and architectural extension to deep reinforcement learning.
result The method successfully handles tasks requiring simultaneous decision-making and action execution.
In this paper, we propose a framework that enables a human teacher to shape a robot behaviour by interactively providing it with unlabeled instructions. We ground the meaning of instruction signals in the task-learning process, and use them simultaneously for guiding the latter. We implement our framework as a modular …
Deep reinforcement learning provides a promising approach for vision-based control of real-world robots. However, the generalization of such models depends critically on the quantity and variety of data available for training. This data can be difficult to obtain for some types of robotic systems, such as fragile, smal…
Applications of safety, security, and rescue in robotics, such as multi-robot target tracking, involve the execution of information acquisition tasks by teams of mobile robots. However, in failure-prone or adversarial environments, robots get attacked, their communication channels get jammed, and their sensors may fail…
This research introduces an autonomous robot navigation method using reinforcement learning.
problem Improving robot navigation in complex environments.
method Deep Q Network (DQN) and Proximal Policy Optimization (PPO) models for path planning and decision-making.
result The models enhance robot navigation ability and adaptive learning in unknown environments.
In this paper, we explore deep reinforcement learning algorithms for vision-based robotic grasping. Model-free deep reinforcement learning (RL) has been successfully applied to a range of challenging environments, but the proliferation of algorithms makes it difficult to discern which particular approach would be best …
New metric solves correspondence problem for robotic arm imitation learning.
problem Establishing corresponding states and actions between different robotic arms.
method Introducing a distance measure between dissimilar robotic arms and using it as a loss function.
result The distance measure effectively learns imitation policies by minimizing distance between robotic arms.
Robo-PlaNet learns robot tasks faster than PlaNet.
problem Training robot tasks with limited or noisy sensor data.
method Asynchronous version of PlaNet that interleaves dynamics model training and data collection.
result Robo-PlaNet achieves higher performance in less time.
Safe reinforcement learning for robots using model predictive shielding.
problem Ensuring safety of learned policies in robotics tasks.
method Model Predictive Shielding (MPS) that switches between learned and backup policies.
result Guaranteed safety of learned policies in challenging robotics tasks.
Prototype real-world RL environment for robotics training.
problem Lack of common real-world RL benchmark.
method Developed OffWorld Gym with open access to real-world robotics environments.
result Baseline results in navigation task on real and simulated terrain.
Robotic navigation improves with RL and ultrasound images.
problem Improving accuracy in robotic navigation for ultrasound-guided procedures.
method Combines deep Q-networks with ultrasound images and binary classifier.
result Significantly improved navigation accuracy compared to pure RL and SL.
Proposes KStar Diffuser for kinematics-aware bimanual robotic manipulation.
problem Challenges in applying imitation learning to bimanual robotic tasks.
method Integrates physical robot structure into action prediction using a dynamic spatial-temporal graph and differentiable kinematics.
result Effective generation of kinematics-aware actions in both simulation and real-world environments.