arXiv research

LqgOpt learns optimal control in unknown LQG systems with minimal regret.

We study the performance of the certainty equivalent controller on Linear Quadratic (LQ) control problems with unknown transition dynamics. We show that for both the fully and partially observed settings, the sub-optimality gap between the cost incurred by playing the certainty equivalent controller on the true system …

RL solves discrete LQ control with Gaussian optimal policy.

We study the problem of regret minimization in partially observable linear quadratic control systems when the model dynamics are unknown a priori. We propose ExpCommit, an explore-then-commit algorithm that learns the model Markov parameters and then follows the principle of optimism in the face of uncertainty to desig…

We consider adaptive control of the Linear Quadratic Regulator (LQR), where an unknown linear system is controlled subject to quadratic costs. Leveraging recent developments in the estimation of linear systems and in robust controller synthesis, we present the first provably polynomial time algorithm that provides high…

We consider the exploration-exploitation tradeoff in linear quadratic (LQ) control problems, where the state dynamics is linear and the cost function is quadratic in states and controls. We analyze the regret of Thompson sampling (TS) (a.k.a. posterior-sampling for reinforcement learning) in the frequentist setting, i.…

Study shows certainty equivalent policy minimizes regret in continuous-time systems.

New theory extends LQ control to non-exponential discount scenarios.

New algorithm achieves logarithmic regret for adversarial online control.

New algorithms achieve logarithmic regret in learning linear quadratic control systems.

Develops a control framework for systemic risk under uncertainty.

Logarithmic regret achieved in continuous-time linear-quadratic reinforcement learning.

Study policy gradient for large-agent mean-field control and game in continuous time.

The study sets limits on how well systems can be controlled adaptively.

Paper uses Koopman operator and Nyström method for efficient nonlinear control.

This paper studies a class of continuous-time scalar-state stochastic Linear-Quadratic (LQ) optimal control problem with the linear control constraints. Applying the state separation theorem induced from its special structure, we develop the explicit solution for this class of problem. The revealed optimal control poli…

We study the constrained linear quadratic regulator with unknown dynamics, addressing the tension between safety and exploration in data-driven control techniques. We present a framework which allows for system identification through persistent excitation, while maintaining safety by guaranteeing the satisfaction of st…

In this work, we propose a robust approach to design distributed controllers for unknown-but-sparse linear and time-invariant systems. By leveraging modern techniques in distributed controller synthesis and structured linear inverse problems as applied to system identification, we show that near-optimal distributed con…

Study cost-driven state representation learning for control from partial observations.

Study optimizes resource allocation in noisy systems for better control.

Paper solves time-inconsistent control problems with BSDEs.

Non-bilinear observations make optimal control harder, showing non-convex costs and non-affine optimal controllers.

In this paper, we formulate a general time-inconsistent stochastic linear--quadratic (LQ) control problem. The time-inconsistency arises from the presence of a quadratic term of the expected state as well as a state-dependent term in the objective functional. We define an equilibrium, instead of optimal, solution withi…

Solves optimal control with state constraints using probabilistic methods.

New model-free algorithm achieves similar LQR regret guarantees.

The paper solves TIC LQ control problems using stochastic differential games.

Study optimizes trading in multiple assets with cross-effects.

Study solves HJB equations for time-inconsistent control problems.

We present the first computationally-efficient algorithm with $\widetilde O(\sqrt{T})$ regret for learning in Linear Quadratic Control systems with unknown dynamics. By that, we resolve an open question of Abbasi-Yadkori and Szepesvári (2011) and Dean, Mania, Matni, Recht, and Tu (2018).

This paper establishes the existence of a unique nonnegative continuous viscosity solution to the HJB equation associated with a Markovian linear-quadratic control problems with singular terminal state constraint and possibly unbounded cost coefficients. The existence result is based on a novel comparison principle for…

This paper studies how gradient descent in control systems can perform well on unseen data.

Policy gradient converges to globally optimal policy in nearly linear-quadratic systems.

Study on PG learning for LQ MFC problems with common noise, proving convergence and sample complexity.

This work establishes safe reinforcement learning for LQR with nonlinear baselines.

Study learns state representations from observations for control, proving guarantees.

We introduce a Bernstein-type inequality which serves to uniformly control quadratic forms of gaussian variables. The latter can for example be used to derive sharp model selection criteria for linear estimation in linear regression and linear inverse problems via penalization, and we do not exclude that its scope of a…

The paper studies optimal transport in linear quadratic systems and derives interpolation inequalities.

TSAC achieves optimal frequentist regret in adaptive control of LQRs.

We study the problem of controlling linear time-invariant systems with known noisy dynamics and adversarially chosen quadratic losses. We present the first efficient online learning algorithms in this setting that guarantee $O(\sqrt{T})$ regret under mild assumptions, where $T$ is the time horizon. Our algorithms rely …

New bounds for adaptive control in high dimensions without fixed state space.

We explore a new method for discrete-time control problems using randomization and entropy.

We study in this paper a class of constrained linear-quadratic (LQ) optimal control problem formulations for the scalar-state stochastic system with multiplicative noise, which has various applications, especially in the financial risk management. The linear constraint on both the control and state variables considered…

Finding optimal feedback controllers for nonlinear dynamic systems from data is hard. Recently, Bayesian optimization (BO) has been proposed as a powerful framework for direct controller tuning from experimental trials. For selecting the next query point and finding the global optimum, BO relies on a probabilistic desc…

We consider the optimal control problem for a linear conditional McKean-Vlasov equation with quadratic cost functional. The coefficients of the system and the weigh-ting matrices in the cost functional are allowed to be adapted processes with respect to the common noise filtration. Semi closed-loop strategies are intro…

Safe RL-based vibration control using LQR guidance.

New method handles robust and adaptive control of linear systems with non-convex costs.

In this paper, we continue our study on a general time-inconsistent stochastic linear--quadratic (LQ) control problem originally formulated in [6]. We derive a necessary and sufficient condition for equilibrium controls via a flow of forward--backward stochastic differential equations. When the state is one dimensional…

This paper addresses the optimal control problem known as the Linear Quadratic Regulator in the case when the dynamics are unknown. We propose a multi-stage procedure, called Coarse-ID control, that estimates a model from a few experimental trials, estimates the error in that model with respect to the truth, and then d…