arXiv research

The paper studies optimal transport in linear quadratic systems and derives interpolation inequalities.

RL solves discrete LQ control with Gaussian optimal policy.

We study the problem of regret minimization in partially observable linear quadratic control systems when the model dynamics are unknown a priori. We propose ExpCommit, an explore-then-commit algorithm that learns the model Markov parameters and then follows the principle of optimism in the face of uncertainty to desig…

LqgOpt learns optimal control in unknown LQG systems with minimal regret.

New algorithms achieve logarithmic regret in learning linear quadratic control systems.

New algorithms improve blind source separation for linear-quadratic mixtures.

Policy gradient methods find Nash equilibrium in noisy games.

Develops a control framework for systemic risk under uncertainty.

Optimizes dividends with stability for risky businesses.

We present the first computationally-efficient algorithm with $\widetilde O(\sqrt{T})$ regret for learning in Linear Quadratic Control systems with unknown dynamics. By that, we resolve an open question of Abbasi-Yadkori and Szepesvári (2011) and Dean, Mania, Matni, Recht, and Tu (2018).

Study shows certainty equivalent policy minimizes regret in continuous-time systems.

Logarithmic regret achieved in continuous-time linear-quadratic reinforcement learning.

Study on PG learning for LQ MFC problems with common noise, proving convergence and sample complexity.

We study the performance of the certainty equivalent controller on Linear Quadratic (LQ) control problems with unknown transition dynamics. We show that for both the fully and partially observed settings, the sub-optimality gap between the cost incurred by playing the certainty equivalent controller on the true system …

Policy gradient converges to globally optimal policy in nearly linear-quadratic systems.

New model-free algorithm achieves similar LQR regret guarantees.

New algorithm learns LQR with $O(\sqrt{T})$ regret using Langevin dynamics and excitation.

This paper establishes the existence of a unique nonnegative continuous viscosity solution to the HJB equation associated with a Markovian linear-quadratic control problems with singular terminal state constraint and possibly unbounded cost coefficients. The existence result is based on a novel comparison principle for…

Paper uses Koopman operator and Nyström method for efficient nonlinear control.

Study policy gradient for large-agent mean-field control and game in continuous time.

Policy gradient methods converge for LQR problems with noisy state dynamics.

Study cost-driven state representation learning for control from partial observations.

This paper studies a class of continuous-time scalar-state stochastic Linear-Quadratic (LQ) optimal control problem with the linear control constraints. Applying the state separation theorem induced from its special structure, we develop the explicit solution for this class of problem. The revealed optimal control poli…

We study the constrained linear quadratic regulator with unknown dynamics, addressing the tension between safety and exploration in data-driven control techniques. We present a framework which allows for system identification through persistent excitation, while maintaining safety by guaranteeing the satisfaction of st…

We consider a general time-inconsistent stochastic linear-quadratic differential game. The time-inconsistency arises from the presence of quadratic terms of the expected state as well as state-dependent term in the objective functionals. We define an equilibrium strategy, which is different from the classical one, and …

Study optimizes trading in multiple assets with cross-effects.

New theory extends LQ control to non-exponential discount scenarios.

Study optimizes resource allocation in noisy systems for better control.

Efficiently solves exploration-exploitation in LQR using Lagrangian relaxation.

Paper solves time-inconsistent control problems with BSDEs.

Study learns state representations from observations for control, proving guarantees.

Study optimizes investment strategies in markets with contagious price jumps.

In this paper, we continue our study on a general time-inconsistent stochastic linear--quadratic (LQ) control problem originally formulated in [6]. We derive a necessary and sufficient condition for equilibrium controls via a flow of forward--backward stochastic differential equations. When the state is one dimensional…

We study the global convergence of generative adversarial imitation learning for linear quadratic regulators, which is posed as minimax optimization. To address the challenges arising from non-convex-concave geometry, we analyze the alternating gradient algorithm and establish its Q-linear rate of convergence to a uniq…

Study solves HJB equations for time-inconsistent control problems.

This paper studies how gradient descent in control systems can perform well on unseen data.

Direct policy gradient methods for reinforcement learning and continuous control problems are a popular approach for a variety of reasons: 1) they are easy to implement without explicit knowledge of the underlying model 2) they are an "end-to-end" approach, directly optimizing the performance metric of interest 3) they…

Safe RL-based vibration control using LQR guidance.

In this paper, we formulate a general time-inconsistent stochastic linear--quadratic (LQ) control problem. The time-inconsistency arises from the presence of a quadratic term of the expected state as well as a state-dependent term in the objective functional. We define an equilibrium, instead of optimal, solution withi…

Study capacity constraints in continual learning with a simple model.

Model-free approaches for reinforcement learning (RL) and continuous control find policies based only on past states and rewards, without fitting a model of the system dynamics. They are appealing as they are general purpose and easy to implement; however, they also come with fewer theoretical guarantees than model-bas…

We study in this paper a class of constrained linear-quadratic (LQ) optimal control problem formulations for the scalar-state stochastic system with multiplicative noise, which has various applications, especially in the financial risk management. The linear constraint on both the control and state variables considered…

The effectiveness of model-based versus model-free methods is a long-standing question in reinforcement learning (RL). Motivated by recent empirical success of RL on continuous control tasks, we study the sample complexity of popular model-based and model-free algorithms on the Linear Quadratic Regulator (LQR). We show…

This paper addresses the optimal control problem known as the Linear Quadratic Regulator in the case when the dynamics are unknown. We propose a multi-stage procedure, called Coarse-ID control, that estimates a model from a few experimental trials, estimates the error in that model with respect to the truth, and then d…

We consider adaptive control of the Linear Quadratic Regulator (LQR), where an unknown linear system is controlled subject to quadratic costs. Leveraging recent developments in the estimation of linear systems and in robust controller synthesis, we present the first provably polynomial time algorithm that provides high…

New algorithm achieves logarithmic regret for adversarial online control.

Multi-agent reinforcement learning has been successfully applied to a number of challenging problems. Despite these empirical successes, theoretical understanding of different algorithms is lacking, primarily due to the curse of dimensionality caused by the exponential growth of the state-action space with the number o…

Theory is developed for linear-quadratic at infinity generating families for Legendrian knots in R^3. It is shown that the unknot with maximal Thurston--Bennequin invariant of -1 has a unique linear-quadratic at infinity generating family, up to fiber-preserving diffeomorphism and stabilization. From this, invariant ge…