arXiv research

LqgOpt learns optimal control in unknown LQG systems with minimal regret.

RL solves discrete LQ control with Gaussian optimal policy.

We study the problem of regret minimization in partially observable linear quadratic control systems when the model dynamics are unknown a priori. We propose ExpCommit, an explore-then-commit algorithm that learns the model Markov parameters and then follows the principle of optimism in the face of uncertainty to desig…

We study the performance of the certainty equivalent controller on Linear Quadratic (LQ) control problems with unknown transition dynamics. We show that for both the fully and partially observed settings, the sub-optimality gap between the cost incurred by playing the certainty equivalent controller on the true system …

New algorithms achieve logarithmic regret in learning linear quadratic control systems.

Logarithmic regret achieved in continuous-time linear-quadratic reinforcement learning.

Develops a control framework for systemic risk under uncertainty.

New theory extends LQ control to non-exponential discount scenarios.

Paper uses Koopman operator and Nyström method for efficient nonlinear control.

We study the constrained linear quadratic regulator with unknown dynamics, addressing the tension between safety and exploration in data-driven control techniques. We present a framework which allows for system identification through persistent excitation, while maintaining safety by guaranteeing the satisfaction of st…

Study optimizes resource allocation in noisy systems for better control.

Paper solves time-inconsistent control problems with BSDEs.

Study shows certainty equivalent policy minimizes regret in continuous-time systems.

New algorithm achieves logarithmic regret for adversarial online control.

We present the first computationally-efficient algorithm with $\widetilde O(\sqrt{T})$ regret for learning in Linear Quadratic Control systems with unknown dynamics. By that, we resolve an open question of Abbasi-Yadkori and Szepesvári (2011) and Dean, Mania, Matni, Recht, and Tu (2018).

This paper studies a class of continuous-time scalar-state stochastic Linear-Quadratic (LQ) optimal control problem with the linear control constraints. Applying the state separation theorem induced from its special structure, we develop the explicit solution for this class of problem. The revealed optimal control poli…

This paper establishes the existence of a unique nonnegative continuous viscosity solution to the HJB equation associated with a Markovian linear-quadratic control problems with singular terminal state constraint and possibly unbounded cost coefficients. The existence result is based on a novel comparison principle for…

Study solves HJB equations for time-inconsistent control problems.

Study optimizes trading in multiple assets with cross-effects.

The paper studies optimal transport in linear quadratic systems and derives interpolation inequalities.

Study cost-driven state representation learning for control from partial observations.

Policy gradient converges to globally optimal policy in nearly linear-quadratic systems.

Study on PG learning for LQ MFC problems with common noise, proving convergence and sample complexity.

TSAC achieves optimal frequentist regret in adaptive control of LQRs.

This paper studies how gradient descent in control systems can perform well on unseen data.

The paper solves TIC LQ control problems using stochastic differential games.

Study policy gradient for large-agent mean-field control and game in continuous time.

We consider the exploration-exploitation tradeoff in linear quadratic (LQ) control problems, where the state dynamics is linear and the cost function is quadratic in states and controls. We analyze the regret of Thompson sampling (TS) (a.k.a. posterior-sampling for reinforcement learning) in the frequentist setting, i.…

We consider adaptive control of the Linear Quadratic Regulator (LQR), where an unknown linear system is controlled subject to quadratic costs. Leveraging recent developments in the estimation of linear systems and in robust controller synthesis, we present the first provably polynomial time algorithm that provides high…

Study learns state representations from observations for control, proving guarantees.

In this paper, we continue our study on a general time-inconsistent stochastic linear--quadratic (LQ) control problem originally formulated in [6]. We derive a necessary and sufficient condition for equilibrium controls via a flow of forward--backward stochastic differential equations. When the state is one dimensional…

Safe RL-based vibration control using LQR guidance.

In this paper, we formulate a general time-inconsistent stochastic linear--quadratic (LQ) control problem. The time-inconsistency arises from the presence of a quadratic term of the expected state as well as a state-dependent term in the objective functional. We define an equilibrium, instead of optimal, solution withi…

New bounds for adaptive control in high dimensions without fixed state space.

This paper addresses the optimal control problem known as the Linear Quadratic Regulator in the case when the dynamics are unknown. We propose a multi-stage procedure, called Coarse-ID control, that estimates a model from a few experimental trials, estimates the error in that model with respect to the truth, and then d…

Solves optimal control with state constraints using probabilistic methods.

This work establishes safe reinforcement learning for LQR with nonlinear baselines.

We study in this paper a class of constrained linear-quadratic (LQ) optimal control problem formulations for the scalar-state stochastic system with multiplicative noise, which has various applications, especially in the financial risk management. The linear constraint on both the control and state variables considered…

Survey of theoretical foundations for policy optimization in control.

Efficiently solves exploration-exploitation in LQR using Lagrangian relaxation.

New algorithm minimizes regret in unknown LQ control systems.

New model-free algorithm achieves similar LQR regret guarantees.

Model-free approaches for reinforcement learning (RL) and continuous control find policies based only on past states and rewards, without fitting a model of the system dynamics. They are appealing as they are general purpose and easy to implement; however, they also come with fewer theoretical guarantees than model-bas…

Survey explores geometric aspects of policy optimization in control systems.

Direct policy gradient methods for reinforcement learning and continuous control problems are a popular approach for a variety of reasons: 1) they are easy to implement without explicit knowledge of the underlying model 2) they are an "end-to-end" approach, directly optimizing the performance metric of interest 3) they…

We study the problem of adaptive control of a high dimensional linear quadratic (LQ) system. Previous work established the asymptotic convergence to an optimal controller for various adaptive control schemes. More recently, for the average cost LQ problem, a regret bound of ${O}(\sqrt{T})$ was shown, apart form logarit…

Study optimizes investment strategies in markets with contagious price jumps.

The effectiveness of model-based versus model-free methods is a long-standing question in reinforcement learning (RL). Motivated by recent empirical success of RL on continuous control tasks, we study the sample complexity of popular model-based and model-free algorithms on the Linear Quadratic Regulator (LQR). We show…