arXiv research

The paper studies optimal transport in linear quadratic systems and derives interpolation inequalities.

LqgOpt learns optimal control in unknown LQG systems with minimal regret.

Policy gradient converges to globally optimal policy in nearly linear-quadratic systems.

Algorithm reduces regret in partially observable systems by learning dynamics and using optimistic control.

Study shows certainty equivalent policy minimizes regret in continuous-time systems.

New algorithms achieve logarithmic regret in learning linear quadratic control systems.

Develops a control framework for systemic risk under uncertainty.

We present the first computationally-efficient algorithm with $\widetilde O(\sqrt{T})$ regret for learning in Linear Quadratic Control systems with unknown dynamics. By that, we resolve an open question of Abbasi-Yadkori and Szepesvári (2011) and Dean, Mania, Matni, Recht, and Tu (2018).

Study optimizes resource allocation in noisy systems for better control.

We study the constrained linear quadratic regulator with unknown dynamics, addressing the tension between safety and exploration in data-driven control techniques. We present a framework which allows for system identification through persistent excitation, while maintaining safety by guaranteeing the satisfaction of st…

Paper uses Koopman operator and Nyström method for efficient nonlinear control.

We study the performance of the certainty equivalent controller on Linear Quadratic (LQ) control problems with unknown transition dynamics. We show that for both the fully and partially observed settings, the sub-optimality gap between the cost incurred by playing the certainty equivalent controller on the true system …

New model-free algorithm achieves similar LQR regret guarantees.

Policy gradient methods find Nash equilibrium in noisy games.

New algorithm learns LQR with $O(\sqrt{T})$ regret using Langevin dynamics and excitation.

This paper studies how gradient descent in control systems can perform well on unseen data.

We consider a general time-inconsistent stochastic linear-quadratic differential game. The time-inconsistency arises from the presence of quadratic terms of the expected state as well as state-dependent term in the objective functionals. We define an equilibrium strategy, which is different from the classical one, and …

New bounds for adaptive control in high dimensions without fixed state space.

Paper solves time-inconsistent control problems with BSDEs.

We consider adaptive control of the Linear Quadratic Regulator (LQR), where an unknown linear system is controlled subject to quadratic costs. Leveraging recent developments in the estimation of linear systems and in robust controller synthesis, we present the first provably polynomial time algorithm that provides high…

TSAC achieves optimal frequentist regret in adaptive control of LQRs.

The main challenge for adaptive regulation of linear-quadratic systems is the trade-off between identification and control. An adaptive policy needs to address both the estimation of unknown dynamics parameters (exploration), as well as the regulation of the underlying system (exploitation). To this end, optimism-based…

We study derivative-free methods for policy optimization over the class of linear policies. We focus on characterizing the convergence rate of these methods when applied to linear-quadratic systems, and study various settings of driving noise and reward feedback. We show that these methods provably converge to within a…

Policy gradient methods converge for LQR problems with noisy state dynamics.

Efficiently solves exploration-exploitation in LQR using Lagrangian relaxation.

Safe RL-based vibration control using LQR guidance.

This paper studies a class of continuous-time scalar-state stochastic Linear-Quadratic (LQ) optimal control problem with the linear control constraints. Applying the state separation theorem induced from its special structure, we develop the explicit solution for this class of problem. The revealed optimal control poli…

This paper addresses the optimal control problem known as the Linear Quadratic Regulator in the case when the dynamics are unknown. We propose a multi-stage procedure, called Coarse-ID control, that estimates a model from a few experimental trials, estimates the error in that model with respect to the truth, and then d…

Model-free approaches for reinforcement learning (RL) and continuous control find policies based only on past states and rewards, without fitting a model of the system dynamics. They are appealing as they are general purpose and easy to implement; however, they also come with fewer theoretical guarantees than model-bas…

New algorithm achieves logarithmic regret for adversarial online control.

We study the problem of adaptive control of a high dimensional linear quadratic (LQ) system. Previous work established the asymptotic convergence to an optimal controller for various adaptive control schemes. More recently, for the average cost LQ problem, a regret bound of ${O}(\sqrt{T})$ was shown, apart form logarit…

Multi-agent reinforcement learning has been successfully applied to a number of challenging problems. Despite these empirical successes, theoretical understanding of different algorithms is lacking, primarily due to the curse of dimensionality caused by the exponential growth of the state-action space with the number o…

Regularized least-squares approaches have been successfully applied to linear system identification. Recent approaches use quadratic penalty terms on the unknown impulse response defined by stable spline kernels, which control model space complexity by leveraging regularity and bounded-input bounded-output stability. T…

Study policy gradient for large-agent mean-field control and game in continuous time.

New algorithm reduces learning regret in multi-agent systems with unknown dynamics.

Solves steering problem with continuous time, Hilbert-Schmidt cost, and matrix ODEs.

We propose a simple model of the banking system incorporating a game feature where the evolution of monetary reserve is modeled as a system of coupled Feller diffusions. The Markov Nash equilibrium generated through minimizing the linear quadratic cost subject to Cox-Ingersoll-Ross type processes creates liquidity and …

We study the problem of controlling linear time-invariant systems with known noisy dynamics and adversarially chosen quadratic losses. We present the first efficient online learning algorithms in this setting that guarantee $O(\sqrt{T})$ regret under mild assumptions, where $T$ is the time horizon. Our algorithms rely …

Study task-guided exploration in linear dynamical systems, improving sample complexity.

Novel approach to Nash equilibrium in mean-field stochastic games with operator resolvents.

Optimal control strategy uses random noise to adaptively control systems with unknown parameters.

Survey explores geometric aspects of policy optimization in control systems.

RL solves discrete LQ control with Gaussian optimal policy.

This paper concerns the problem of learning control policies for an unknown linear dynamical system to minimize a quadratic cost function. We present a method, based on convex optimization, that accomplishes this task robustly: i.e., we minimize the worst-case cost, accounting for system uncertainty given the observed …

Logarithmic regret achieved in continuous-time linear-quadratic reinforcement learning.

Study learns state representations from observations for control, proving guarantees.

New algorithm minimizes regret in unknown LQ control systems.

Direct policy gradient methods for reinforcement learning and continuous control problems are a popular approach for a variety of reasons: 1) they are easy to implement without explicit knowledge of the underlying model 2) they are an "end-to-end" approach, directly optimizing the performance metric of interest 3) they…