arXiv research

Paper solves pendulum swing-up problem using RL.

An energy based approach for stabilizing a mechanical system has offered a simple yet powerful control scheme. However, since it does not impose such strong constraints on parameter space of the controller, finding appropriate parameter values for an optimal controller is known to be hard. This paper intends to generat…

Method learns to map dynamics of different systems.

Paper proposes adaptive control for unknown systems using reinforcement learning.

We propose directed time series regression, a new approach to estimating parameters of time-series models for use in certainty equivalent model predictive control. The approach combines merits of least squares regression and empirical optimization. Through a computational study involving a stochastic version of a well …

Reinforcement learning is a powerful paradigm for learning optimal policies from experimental data. However, to find optimal policies, most reinforcement learning algorithms explore all possible actions, which may be harmful for real-world systems. As a consequence, learning algorithms are rarely applied on safety-crit…

Controller seeks informative system observations to predict nonlinear dynamics.

In this paper we show that there are applications that transform the movement of a pendulum into movements in $\mathbb{R}^3$. This can be done using Euler top system of differential equations. On the constant level surfaces, Euler top system reduces to the equation of a pendulum. Those properties are also considered in…

This paper studies the topology of the constant energy surfaces of the double spherical pendulum.

The present paper extends the classical second-order variational problem of Herglotz type to the more general context of the Euclidean sphere S^n following variational and optimal control approaches. The relation between the Hamiltonian equations and the generalized Euler-Lagrange equations is established. This problem…

PID control architectures are widely used in industrial applications. Despite their low number of open parameters, tuning multiple, coupled PID controllers can become tedious in practice. In this paper, we extend PILCO, a model-based policy search framework, to automatically tune multivariate PID controllers purely bas…

Random features enhance control of complex systems.

Developing mathematical models of dynamic systems is central to many disciplines of engineering and science. Models facilitate simulations, analysis of the system's behavior, decision making and design of automatic control algorithms. Even inherently model-free control techniques such as reinforcement learning (RL) hav…

Despite the numerous advances, reinforcement learning remains away from widespread acceptance for autonomous controller design as compared to classical methods due to lack of ability to effectively tackle the reality gap. The reliance on absolute or deterministic reward as a metric for optimization process renders rein…

We present foundations for using Model Predictive Control (MPC) as a differentiable policy class for reinforcement learning in continuous state and action spaces. This provides one way of leveraging and combining the advantages of model-free and model-based approaches. Specifically, we differentiate through MPC by usin…

Learning to make decisions from observed data in dynamic environments remains a problem of fundamental importance in a number of fields, from artificial intelligence and robotics, to medicine and finance. This paper concerns the problem of learning control policies for unknown linear dynamical systems so as to maximize…

Physics-informed learning framework for pH systems and EB-PBC control.

Reinforcement learning is a promising approach to learning robotics controllers. It has recently been shown that algorithms based on finite-difference estimates of the policy gradient are competitive with algorithms based on the policy gradient theorem. We propose a theoretical framework for understanding this phenomen…

This paper compares model-based and model-free control methods using neural networks.

Reinforcement Learning (RL) algorithms have found limited success beyond simulated applications, and one main reason is the absence of safety guarantees during the learning process. Real world systems would realistically fail or break before an optimal controller can be learned. To address this issue, we propose a cont…

Study of sub-Riemannian problem on specific Lie groups, revealing symmetries and bounds.

Paper presents neural network controllers for offset-free setpoint tracking.

PenduMAV is a 6-input omnidirectional MAV without internal forces.

A core novelty of Alpha Zero is the interleaving of tree search and deep learning, which has proven very successful in board games like Chess, Shogi and Go. These games have a discrete action space. However, many real-world reinforcement learning domains have continuous action spaces, for example in robotic control, na…

This study presents the results of a series of simulation experiments that evaluate and compare four different manifold alignment methods under the influence of noise. The data was created by simulating the dynamics of two slightly different double pendulums in three-dimensional space. The method of semi-supervised fea…

Method learns low-dim. state vars from noisy high-dim. data.

This paper classifies Legendre singularities of sub-Riemannian geodesics on surfaces.

We present an approach to identify concise equations from data using a shallow neural network approach. In contrast to ordinary black-box regression, this approach allows understanding functional relations and generalizing them from observed data to unseen parts of the parameter space. We show how to extend the class o…

Data-driven control of robotic systems using Koopman operators with error bounds.

This paper detects Markov violations in RL with noise, improving policy development.

Reinforcement learning algorithms can solve dynamic decision-making and optimal control problems. With continuous-valued state and input variables, reinforcement learning algorithms must rely on function approximators to represent the value function and policy mappings. Commonly used numerical approximators, such as ne…

Expert augmentation improves hybrid model generalization.

Proposes DLGPD model to learn dynamics from images for planning.

In this work we seek for an approach to integrate safety in the learning process that relies on a partly known state-space model of the system and regards the unknown dynamics as an additive bounded disturbance. We introduce a framework for safely learning a control strategy for a given system with an additive disturba…

First principles modeling of physical systems has led to significant technological advances across all branches of science. For nonlinear systems, however, small modeling errors can lead to significant deviations from the true, measured behavior. Even in mechanical systems, where the equations are assumed to be well-kn…

Bayesian inference models failure distributions in autonomous systems.

Estimates system parameters from a single observation using kernel-based score.

Study on elastic curves with variable stiffness, derived from bending energy.

Guaranteed reachable set for unknown nonlinear systems on manifolds.

CW-EDMD improves prediction accuracy by learning local Koopman models for different state-space regions.

Safe exploration method for RL under disturbance ensures safety with probabilistic guarantees.

Paper constructs new minimal submanifolds in spheres by spinning given ones.

Discover conservation laws from trajectories using a neural network.

Paper explores foundation models for dynamical systems using synthetic data.

In this study we introduce a new technique for symbolic regression that guarantees global optimality. This is achieved by formulating a mixed integer non-linear program (MINLP) whose solution is a symbolic mathematical expression of minimum complexity that explains the observations. We demonstrate our approach by redis…

We present a self-contained proof of the Gauss-Bonnet theorem for two-dimensional surfaces embedded in $R^3$ using just classical vector calculus. The exposition should be accessible to advanced undergraduate and non-expert graduate students. It may be viewed as an illustration and exercise in multivariate calculus and…

One of the challenges in model-based control of stochastic dynamical systems is that the state transition dynamics are involved, and it is not easy or efficient to make good-quality predictions of the states. Moreover, there are not many representational models for the majority of autonomous systems, as it is not easy …

In this paper, we point out a fundamental property of the objective in reinforcement learning, with which we can reformulate the policy gradient objective into a perceptron-like loss function, removing the need to distinguish between on and off policy training. Namely, we posit that it is sufficient to only update a po…