Deep RL trains a robust humanoid push-recovery policy.
problem Training robust humanoid push-recovery policies.
method Model-free Deep Reinforcement Learning.
result Policy learns robust behaviors across the entire body.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Humanoid Control”
RFC enhances humanoid control to imitate complex human motions.
problem Dynamics mismatch between humanoid models and real humans.
method Residual Force Control (RFC) augments control policies with external forces.
result RFC outperforms state-of-the-art methods in convergence speed and motion quality.
Study aims to develop a humanoid robot dialogue system.
problem Current dialogue systems lack attention to non-verbal cues.
method Participated in a competition to develop a system with facial expressions and gaze control.
result Developed a humanoid robot dialogue system.
Paper learns versatile balancing and recovery motions for humanoid robots.
problem Training humanoid robots to handle unexpected perturbations.
method Hierarchical Deep Reinforcement Learning in a physics simulator.
result Learned skills comparable to preprogrammed controllers but more adaptable.
CNN identifies nonlinear human posture control models efficiently.
problem Identifying nonlinear human posture control models.
method Convolutional Neural Networks (CNN) for model identification.
result Efficiently identifies nonlinear human posture control models.
Simple model-based reinforcement learning outperforms model-free methods in complex tasks.
problem Lagging performance of model-based reinforcement learning agents in non-trivial environments.
method Combining soft value estimates with stochastic value gradients.
result Simple model-based agents achieve state-of-the-art results in a high-dimensional humanoid control task.
Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Criticscs.LG
Paper tackles overestimation bias in continuous control, improving performance by 25%.
problem Overestimation bias in off-policy learning.
method Truncated Quantile Critics (TQC) combines distributional representation, truncation, and ensembling of critics.
result TQC outperforms state-of-the-art methods by 25% on the Humanoid environment.
Defense strategy improves controller robustness against adversarial attacks.
problem Adversarial attacks on learning-enabled controllers in CPS.
method Two-stage defense strategy treating controller and environment as black-boxes with unknown dynamics.
result Defense strategy effectively improves controller robustness in realistic control domains.
We demonstrate that models trained only in simulation can be used to solve a manipulation problem of unprecedented complexity on a real robot. This is made possible by two key components: a novel algorithm, which we call automatic domain randomization (ADR) and a robot platform built for machine learning. ADR automatic…
We propose a plan online and learn offline (POLO) framework for the setting where an agent, with an internal model, needs to continually act and learn in the world. Our work builds on the synergistic relationship between local model-based control, global value function learning, and exploration. We study how local traj…
One of the open challenges in designing robots that operate successfully in the unpredictable human environment is how to make them able to predict what actions they can perform on objects, and what their effects will be, i.e., the ability to perceive object affordances. Since modeling all the possible world interactio…
This paper proposes a framework which is able to generate a sequence of three-dimensional human dance poses for a given music. The proposed framework consists of three components: a music feature encoder, a pose generator, and a music genre classifier. We focus on integrating these components for generating a realistic…
Robots detect and recognize objects in real-time for better manipulation.
problem Real-time object detection and recognition for humanoid robots.
method Modified YOLOv3 algorithm, quantization, and re-arrangement of layers for low-compute NAO robots.
result Robots can perform real-time detection, recognition, and localization of objects.
Deep reinforcement learning algorithms have been successfully applied to a range of challenging control tasks. However, these methods typically struggle with achieving effective exploration and are extremely sensitive to the choice of hyperparameters. One reason is that most approaches use a noisy version of their oper…
Humans are able to perform a myriad of sophisticated tasks by drawing upon skills acquired through prior experience. For autonomous agents to have this capability, they must be able to extract reusable skills from past experience that can be recombined in new ways for subsequent tasks. Furthermore, when controlling com…
Agents that can learn to imitate given video observation -- \emph{without direct access to state or action information} are more applicable to learning in the natural world. However, formulating a reinforcement learning (RL) agent that facilitates this goal remains a significant challenge. We approach this challenge us…
With the emergence of onboard vision processing for areas such as the internet of things (IoT), edge computing and autonomous robots, there is increasing demand for computationally efficient convolutional neural network (CNN) models to perform real-time object detection on resource constraints hardware devices. Tiny-YO…
A novel approach learns goal-conditioned policies for locomotion using batch RL.
problem Training goal-conditioned policies for rotation invariant locomotion.
method Data augmentation and Siamese framework for invariance.
result Our approach outperforms existing RL algorithms on 3D locomotion agents.
Every living organism struggles against disruptive environmental forces to carve out and maintain an orderly niche. We propose that such a struggle to achieve and preserve order might offer a principle for the emergence of useful behaviors in artificial agents. We formalize this idea into an unsupervised reinforcement …
Meta-SAC automatically tunes SAC's entropy temperature for better exploration.
problem Exploration-exploitation dilemma in reinforcement learning.
method Meta-SAC uses metagradient and a novel meta objective to automatically adjust SAC's entropy temperature.
result Meta-SAC outperforms SAC-v2 by 10% on the humanoid-v2 task.
This paper presents a novel approach for incremental semiparametric inverse dynamics learning. In particular, we consider the mixture of two approaches: Parametric modeling based on rigid body dynamics equations and nonparametric modeling based on incremental kernel methods, with no prior information on the mechanical …
Most policy search algorithms require thousands of training episodes to find an effective policy, which is often infeasible with a physical robot. This survey article focuses on the extreme other end of the spectrum: how can a robot adapt with only a handful of trials (a dozen) and a few minutes? By analogy with the wo…
This paper presents a semi-parametric algorithm for online learning of a robot inverse dynamics model. It combines the strength of the parametric and non-parametric modeling. The former exploits the rigid body dynamics equa- tion, while the latter exploits a suitable kernel function. We provide an extensive comparison …
This paper addresses the problem of evaluating learning systems in safety critical domains such as autonomous driving, where failures can have catastrophic consequences. We focus on two problems: searching for scenarios when learned agents fail and assessing their probability of failure. The standard method for agent e…
Deep learning has revolutionised many fields, but it is still challenging to transfer its success to small mobile robots with minimal hardware. Specifically, some work has been done to this effect in the RoboCup humanoid football domain, but results that are performant and efficient and still generally applicable outsi…
Deep neural networks have been widely adopted in recent years, exhibiting impressive performances in several application domains. It has however been shown that they can be fooled by adversarial examples, i.e., images altered by a barely-perceivable adversarial noise, carefully crafted to mislead classification. In thi…
We explore the use of Evolution Strategies (ES), a class of black box optimization algorithms, as an alternative to popular MDP-based RL techniques such as Q-learning and Policy Gradients. Experiments on MuJoCo and Atari show that ES is a viable solution strategy that scales extremely well with the number of CPUs avail…
Deep reinforcement learning (RL) policies are known to be vulnerable to adversarial perturbations to their observations, similar to adversarial examples for classifiers. However, an attacker is not usually able to directly modify another agent's observations. This might lead one to wonder: is it possible to attack an R…
Maximum entropy deep reinforcement learning (RL) methods have been demonstrated on a range of challenging continuous tasks. However, existing methods either suffer from severe instability when training on large off-policy data or cannot scale to tasks with very high state and action dimensionality such as 3D humanoid l…
New optimization algorithms on orthogonal group for machine learning.
problem Efficient optimization on the orthogonal group for machine learning tasks.
method Stochastic geometric algorithms on Lie groups.
result Strong performance on diverse machine learning tasks.
Regression aims at estimating the conditional mean of output given input. However, regression is not informative enough if the conditional density is multimodal, heteroscedastic, and asymmetric. In such a case, estimating the conditional density itself is preferable, but conditional density estimation (CDE) is challeng…
Safe-EF improves federated learning for non-smooth, constrained optimization.
problem Federated learning's communication bottlenecks with high-dimensional model updates.
method Error feedback (EF) for non-smooth convex optimization with safety constraints.
result Safe-EF matches lower complexity bounds and ensures safety constraints.
Most reinforcement learning algorithms are inefficient for learning multiple tasks in complex robotic systems, where different tasks share a set of actions. In such environments a compound policy may be learnt with shared neural network parameters, which performs multiple tasks concurrently. However such compound polic…
DreamerV2 learns Atari game behaviors from a world model, achieving human-level performance.
problem Learning complex behaviors in Atari games from limited data.
method DreamerV2 uses a world model with discrete representations to predict behaviors in a compact latent space.
result Achieves human-level performance on 55 Atari tasks.
As the size and richness of available datasets grow larger, the opportunities for solving increasingly challenging problems with algorithms learning directly from data grow at the same pace. Consequently, the capability of learning algorithms to work with large amounts of data has become a crucial scientific and techno…
In this paper, we propose an active perception method for recognizing object categories based on the multimodal hierarchical Dirichlet process (MHDP). The MHDP enables a robot to form object categories using multimodal information, e.g., visual, auditory, and haptic information, which can be observed by performing acti…
Modern automation systems rely on closed loop control, wherein a controller interacts with a controlled process, based on observations. These systems are increasingly complex, yet most controllers are linear Proportional-Integral-Derivative (PID) controllers. PID controllers perform well on linear and near-linear syste…
Derives optimal control conditions using calculus of variations.
problem Optimizing Markov control in stochastic control problems.
method Calculus of variations approach to derive necessary conditions.
result Solves the Merton portfolio optimization problem.
NewtonianVAE: Proportional Control and Goal Identification from Pixels via Physical Latent Spacescs.LG
Framework simplifies vision-based control and goal discovery.
problem Learning proportional control from visual data.
method Introduces NewtonianVAE for proportional control and goal discovery.
result Dramatic simplification and acceleration of vision-based controllers.
Paper studies constrained control games with a novel approximation method.
problem Games with constrained control directions.
method Approximation procedure based on -stability estimates and almost sure convergence.
result Existence of game's value and optimal strategy for the stopper.
Learning Force Control for Contact-rich Manipulation Tasks with Rigid Position-controlled Robotscs.LG
This work tackles force control for contact-rich manipulation tasks with rigid robots using RL.
problem Challenges in working with real robotic hardware, especially position-controlled robots.
method Combines RL with traditional force control techniques, implementing parallel position/force control and admittance control.
result Validated methods on both simulation and real robot (UR3 e-series) for force control.
RL applied to TCLs for power consumption control.
problem Optimizing power consumption using TCLs with RL.
method Modelica-based reinforcement learning (Q-learning) for stochastic TCLs.
result Q-learning parameters affect controller performance.
A framework integrates machine learning with robust control for safer, more reliable systems.
problem Combining machine learning with robust control for systems with stringent safety and reliability requirements.
method Integrates Gaussian Process Regression and state-of-the-art robust controller synthesis within a framework that provides rigorous guarantees.
result Demonstrated improved performance with more data while maintaining rigorous guarantees.
Paper uses deep reinforcement learning for better control of rocket engines during start-up phases.
problem Lack of optimal control during transient phases of liquid rocket engines.
method Deep reinforcement learning approach for optimal control of a gas-generator engine's continuous start-up phase.
result Deep reinforcement learning controller achieves highest performance and minimal computational effort.
Neural ODEs control graph dynamics with low energy feedback.
problem Controlling complex dynamical systems on graphs.
method Neural Ordinary Differential Equation Control (NODEC) framework.
result NODEC learns low-energy control signals for graph dynamical systems.
Just as an explicit parameterisation of system dynamics by state, i.e., a choice of coordinates, can impede the identification of general structure, so it is too with an explicit parameterisation of system dynamics by control. However, such explicit and fixed parameterisation by control is commonplace in control theory…
Guaranteeing Safety of Learned Perception Modules via Measurement-Robust Control Barrier Functionseess.SY
Unified control theory and machine learning for safety in uncertain systems.
problem Safety guarantees for systems with measurement model uncertainty.
method Measurement-Robust Control Barrier Functions (MR-CBFs) for control synthesis.
result MR-CBFs ensure safety in perception systems with measurement model uncertainty.
Paper studies optimal control for a specific geometric problem.
problem Optimal control problem associated with the Paneitz obstacle problem.
method Existence and regularity results for optimal controls.
result Existence of optimal controls and their properties.