arXiv research

The paper studies optimal transport in linear quadratic systems and derives interpolation inequalities.

Study shows certainty equivalent policy minimizes regret in continuous-time systems.

Policy gradient converges to globally optimal policy in nearly linear-quadratic systems.

LqgOpt learns optimal control in unknown LQG systems with minimal regret.

New model-free algorithm achieves similar LQR regret guarantees.

The study sets limits on how well systems can be controlled adaptively.

New algorithms achieve logarithmic regret in learning linear quadratic control systems.

We consider adaptive control of the Linear Quadratic Regulator (LQR), where an unknown linear system is controlled subject to quadratic costs. Leveraging recent developments in the estimation of linear systems and in robust controller synthesis, we present the first provably polynomial time algorithm that provides high…

Develops a control framework for systemic risk under uncertainty.

We present the first computationally-efficient algorithm with $\widetilde O(\sqrt{T})$ regret for learning in Linear Quadratic Control systems with unknown dynamics. By that, we resolve an open question of Abbasi-Yadkori and Szepesvári (2011) and Dean, Mania, Matni, Recht, and Tu (2018).

Study optimizes resource allocation in noisy systems for better control.

Regularized least-squares approaches have been successfully applied to linear system identification. Recent approaches use quadratic penalty terms on the unknown impulse response defined by stable spline kernels, which control model space complexity by leveraging regularity and bounded-input bounded-output stability. T…

In this work, we propose a robust approach to design distributed controllers for unknown-but-sparse linear and time-invariant systems. By leveraging modern techniques in distributed controller synthesis and structured linear inverse problems as applied to system identification, we show that near-optimal distributed con…

QENDy learns quadratic dynamics from nonlinear systems data.

The paper is devoted to quadratic Poisson structures compatible with the canonical linear Poisson structures on trivial 1-dimensional central extensions of semisimple Lie algebras. In particular, we develop the general theory of such structures and study related families of functions in involution. We also show that th…

We study the problem of controlling linear time-invariant systems with known noisy dynamics and adversarially chosen quadratic losses. We present the first efficient online learning algorithms in this setting that guarantee $O(\sqrt{T})$ regret under mild assumptions, where $T$ is the time horizon. Our algorithms rely …

We study the constrained linear quadratic regulator with unknown dynamics, addressing the tension between safety and exploration in data-driven control techniques. We present a framework which allows for system identification through persistent excitation, while maintaining safety by guaranteeing the satisfaction of st…

This paper concerns the problem of learning control policies for an unknown linear dynamical system to minimize a quadratic cost function. We present a method, based on convex optimization, that accomplishes this task robustly: i.e., we minimize the worst-case cost, accounting for system uncertainty given the observed …

We study the problem of regret minimization in partially observable linear quadratic control systems when the model dynamics are unknown a priori. We propose ExpCommit, an explore-then-commit algorithm that learns the model Markov parameters and then follows the principle of optimism in the face of uncertainty to desig…

We consider a general time-inconsistent stochastic linear-quadratic differential game. The time-inconsistency arises from the presence of quadratic terms of the expected state as well as state-dependent term in the objective functionals. We define an equilibrium strategy, which is different from the classical one, and …

Paper uses Koopman operator and Nyström method for efficient nonlinear control.

Given a space it is easy to obtain the system of geodesic equations on it. In this paper the inverse problem of reconstructing the space from the geodesic equations is addressed. A procedure is developed for obtaining the metric tensor from the Christoffel symbols. The procedure is extended for determining if a second …

New bounds for adaptive control in high dimensions without fixed state space.

We consider the fundamental problem of solving quadratic systems of equations in $n$ variables, where $y_i = |\langle \boldsymbol{a}_i, \boldsymbol{x} \rangle|^2$, $i = 1, \ldots, m$ and $\boldsymbol{x} \in \mathbb{R}^n$ is unknown. We propose a novel method, which starting with an initial guess computed by means of a …

We study the performance of the certainty equivalent controller on Linear Quadratic (LQ) control problems with unknown transition dynamics. We show that for both the fully and partially observed settings, the sub-optimality gap between the cost incurred by playing the certainty equivalent controller on the true system …

This paper studies how gradient descent in control systems can perform well on unseen data.

Finding optimal feedback controllers for nonlinear dynamic systems from data is hard. Recently, Bayesian optimization (BO) has been proposed as a powerful framework for direct controller tuning from experimental trials. For selecting the next query point and finding the global optimum, BO relies on a probabilistic desc…

We study derivative-free methods for policy optimization over the class of linear policies. We focus on characterizing the convergence rate of these methods when applied to linear-quadratic systems, and study various settings of driving noise and reward feedback. We show that these methods provably converge to within a…

New algorithm learns linear dynamical systems from measurements.

We consider the optimal control problem for a linear conditional McKean-Vlasov equation with quadratic cost functional. The coefficients of the system and the weigh-ting matrices in the cost functional are allowed to be adapted processes with respect to the common noise filtration. Semi closed-loop strategies are intro…

Learning to make decisions from observed data in dynamic environments remains a problem of fundamental importance in a number of fields, from artificial intelligence and robotics, to medicine and finance. This paper concerns the problem of learning control policies for unknown linear dynamical systems so as to maximize…

We propose a model of inter-bank lending and borrowing which takes into account clearing debt obligations. The evolution of log-monetary reserves of $N$ banks is described by coupled diffusions driven by controls with delay in their drifts. Banks are minimizing their finite-horizon objective functions which take into a…

Policy gradient methods find Nash equilibrium in noisy games.

Paper develops PAC-Bayes bounds for unknown linear systems.

Improved stability analysis of neural network systems using Zames-Falb multipliers.

New algorithm learns LQR with $O(\sqrt{T})$ regret using Langevin dynamics and excitation.

New method handles robust and adaptive control of linear systems with non-convex costs.

The main challenge for adaptive regulation of linear-quadratic systems is the trade-off between identification and control. An adaptive policy needs to address both the estimation of unknown dynamics parameters (exploration), as well as the regulation of the underlying system (exploitation). To this end, optimism-based…

Paper solves time-inconsistent control problems with BSDEs.

Study policy gradient for large-agent mean-field control and game in continuous time.

TSAC achieves optimal frequentist regret in adaptive control of LQRs.

Study task-guided exploration in linear dynamical systems, improving sample complexity.

New algorithm achieves logarithmic regret for adversarial online control.

Policy gradient methods converge for LQR problems with noisy state dynamics.

We develop algorithms for the numerical computation of the quadratic hedging strategy in incomplete markets modeled by pure jump Markov process. Using the Hamilton-Jacobi-Bellman approach, the value function of the quadratic hedging problem can be related to a triangular system of parabolic partial integro-differential…

New study shows faster convergence of SGD and Kaczmarz methods.

This paper studies a class of continuous-time scalar-state stochastic Linear-Quadratic (LQ) optimal control problem with the linear control constraints. Applying the state separation theorem induced from its special structure, we develop the explicit solution for this class of problem. The revealed optimal control poli…

Non-bilinear observations make optimal control harder, showing non-convex costs and non-affine optimal controllers.