Efficient algorithm reduces control system learning regret to sqrt(T).
problem Learning Linear-Quadratic Regulators with unknown dynamics efficiently.
method First computationally-efficient algorithm with sqrt(T) regret.
result Resolves open question on control system learning.
The paper studies optimal transport in linear quadratic systems and derives interpolation inequalities.
problem Optimal transport problem in Linear Quadratic optimal control systems.
method Well-posedness of the Monge problem, regularity of optimal transport map, displacement interpolation of measures.
result Derivation of general interpolation inequalities for entropy functionals.
Study shows a linear quadratic regulator's imitation learning converges globally.
problem Global convergence of imitation learning for linear quadratic regulators.
method Analyzed alternating gradient algorithm and established Q-linear rate of convergence.
result Established a unique saddle point for globally optimal policy and reward function.
Algorithm reduces regret in partially observable systems by learning dynamics and using optimistic control.
problem Minimizing regret in partially observable linear quadratic control systems with unknown dynamics.
method ExpCommit algorithm that learns model parameters and uses optimism in uncertainty.
result End-to-end sublinear regret upper bound of O ~ ( T 2 / 3 ) \tilde{\mathcal{O}}(T^{2/3}) O ~ ( T 2/3 ) for ExpCommit. New algorithms achieve logarithmic regret in learning linear quadratic control systems.
problem Learning in Linear Quadratic Control systems with unknown parameters.
method Efficient algorithms for two scenarios: unknown A A A or B B B with certain conditions. result Regret scales logarithmically with the number of steps, not square root.
Certainty equivalent controllers perform nearly optimally in LQ control problems with unknown dynamics.
problem Optimizing control in systems with unknown transition dynamics.
method Analysis of certainty equivalent controllers and comparison to optimal LQ controllers, using perturbation bounds for discrete Riccati equations.
result Sub-optimality gap scales as the square of the parameter error, improving upon previous results.
RL solves discrete LQ control with Gaussian optimal policy.
problem Discrete-time linear-quadratic control problem.
method Entropy-based RL to find Gaussian optimal policy.
result RL algorithm solves mean-variance asset-liability management problem.
Logarithmic regret achieved in continuous-time linear-quadratic reinforcement learning.
problem Optimizing control actions in unknown continuous-time systems over a finite time horizon.
method Least-squares algorithm based on continuous-time observations and controls, with perturbation analysis and parameter estimation error analysis.
result Logarithmic regret bound of order O ( ( ln M ) ( ln ln M ) ) O((\ln M)(\ln\ln M)) O (( ln M ) ( ln ln M )) . LqgOpt learns optimal control in unknown LQG systems with minimal regret.
problem Adaptive control in partially observable linear quadratic Gaussian systems with unknown dynamics.
method Optimism in the face of uncertainty, predictor state evolution, closed-loop system identification, confidence bounds.
result Proves a regret upper bound of i l d e O ( T ) ilde{\mathcal{O}}(\sqrt{T}) i l d e O ( T ) for LQG systems. Safe control of systems with unknown dynamics using persistent excitation.
problem Tension between safety and exploration in data-driven control.
method System identification through persistent excitation, robust constraint satisfaction, and synthesis of feedback controllers.
result Non-asymptotic guarantees on estimation and controller performance.
Policy gradient converges to globally optimal policy in nearly linear-quadratic systems.
problem Finding optimal policies in nonlinear control systems with partial information.
method Policy gradient algorithm designed for nearly linear-quadratic regulators with small Lipschitz nonlinear components.
result Policy gradient algorithm converges to globally optimal policy with linear rate.
Study derivative-free methods for linear policies in linear-quadratic systems.
problem Optimizing policies in linear-quadratic systems with limited derivative information.
method Derivative-free methods applied to linear policies over various noise and reward feedback settings.
result These methods converge to near-optimal policies with a polynomial number of zero-order evaluations.
Policy gradient methods find Nash equilibrium in noisy games.
problem Finding Nash equilibrium in noisy games.
method Policy gradient methods with noise added.
result Policy gradient methods converge to Nash equilibrium in noisy games.
Study shows certainty equivalent policy minimizes regret in continuous-time systems.
problem Minimizing regret in continuous-time stochastic linear-quadratic systems.
method Theoretical analysis of randomized certainty equivalent policy.
result Establishes square-root of time regret bounds and linear scaling with parameters.
New algorithm learns LQR with O ( T ) O(\sqrt{T}) O ( T ) regret using Langevin dynamics and excitation.
problem Learning LQR with a O ( T ) O(\sqrt{T}) O ( T ) regret bound. method Thompson sampling with Langevin dynamics and excitation mechanism.
result Achieved O ( T ) O(\sqrt{T}) O ( T ) regret bound for LQR learning. Optimizes dividends with stability for risky businesses.
problem Maximizing dividends with stability in risky businesses.
method Linear-quadratic optimization for a general Lévy process.
result Derives optimal affine dividend strategies with stability.
Paper uses Koopman operator and Nyström method for efficient nonlinear control.
problem Control of nonlinear dynamical systems.
method Combines Koopman operator framework with Nyström approximation for kernel methods.
result Theoretical guarantees on the convergence rates of the approximated Riccati operator and regulator objective.
New algorithms improve blind source separation for linear-quadratic mixtures.
problem Blind source separation of linear-quadratic mixtures under separability assumptions.
method Proposed two algorithms: SNPALQ and BF. SNPALQ generalizes SNPA for LQ model, BF post-processes SNPALQ.
result Proven robustness and computational tractability of SNPALQ in separating sources even with noise.
Study on variance of policy gradient in simple RL environments.
problem Understanding variance of policy gradient estimators in continuous RL.
method Analyzes REINFORCE estimator in linear-quadratic environments with Gaussian noise.
result Derives and validates bounds on estimator variance empirically.
Global convergence proved for multi-agent LQRs with hierarchical actor-critic.
problem Challenges in understanding multi-agent reinforcement learning algorithms.
method Developed a hierarchical actor-critic algorithm for partially exchangeable agents.
result Global linear convergence to optimal policy proved.
Develops a control framework for systemic risk under uncertainty.
problem Systemic risk under model uncertainty.
method Linear-quadratic mean-field control framework with viscosity solutions and verification theorems.
result Explicit feedback controls derived from a coupled Riccati system, preserving analytical tractability.
Study on PG learning for LQ MFC problems with common noise, proving convergence and sample complexity.
problem Optimal policy learning in LQ MFC problems with common noise and entropy regularization.
method Comprehensive error analysis of PG algorithms in both model-based and model-free settings.
result Global linear convergence and sample complexity of PG algorithms in model-free setting.
Actor-critic converges globally in LQR with ergodic cost.
problem Theoretical understanding of actor-critic algorithm's global convergence.
method Nonasymptotic convergence analysis of actor-critic in linear quadratic regulator (LQR) setting.
result Actor-critic finds globally optimal policy and value function at a linear rate.
We consider a general time-inconsistent stochastic linear-quadratic differential game. The time-inconsistency arises from the presence of quadratic terms of the expected state as well as state-dependent term in the objective functionals. We define an equilibrium strategy, which is different from the classical one, and …
New theory extends LQ control to non-exponential discount scenarios.
problem Time-inconsistent deterministic LQ control problems.
method Extended equivalent relationship to non-exponential discount functions, studied Riccati equation solvability.
result Existence and uniqueness of linear equilibrium for time-inconsistent LQ problem.
Study optimizes resource allocation in noisy systems for better control.
problem Limited attention in stochastic systems with multiplicative noise.
method Analytical and numerical methods for optimal attention allocation.
result Effective resource allocation enhances noise estimation and control decisions.
Efficiently solves exploration-exploitation in LQR using Lagrangian relaxation.
problem Exploration-exploitation dilemma in linear quadratic regulator (LQR) setting.
method Relax optimistic optimization into a constrained extended LQR problem, then solve using Riccati equations.
result Computes ε ε ε -optimistic controller efficiently with O ( log ( 1 / ε ) ) O\big(\log(1/ε)\big) O ( log ( 1/ ε ) ) Riccati equations. Paper solves time-inconsistent control problems with BSDEs.
problem Time-inconsistent stochastic control in continuous time.
method Probabilistic representation via BSDEs.
result Equilibrium value function resolved for inconsistent cases.
Paper finds unique viscosity solution to complex control problems.
problem Complex stochastic control problems with singular terminal state constraints.
method Establishes existence of unique nonnegative continuous viscosity solution using novel comparison principle.
result Unique viscosity solution to HJB equation for linear-quadratic control problems.
Policy-gradient algorithms fail to converge to Nash equilibria in continuous state and action space games.
problem Policy-gradient algorithms lack convergence guarantees in multi-agent continuous state and action space games.
method Analysis of gradient-play in linear quadratic games, showing non-convexity and counterexamples.
result Policy-gradient algorithms can avoid Nash equilibria in certain continuous state and action space games.
Study solves HJB equations for time-inconsistent control problems.
problem Time-inconsistent deterministic linear quadratic control problems.
method Characterized solutions using Riccati equations with integral terms, proving uniqueness.
result Uniqueness of solutions to equilibrium HJB equations proved.
Simple stochastic Newton and cubic Newton methods with fast convergence.
problem Minimizing large numbers of smooth and strongly convex functions.
method Stochastic Newton and cubic Newton methods with simple local linear-quadratic rates.
result Local linear-quadratic convergence results with fast adaptation to problem's curvature.
Study optimizes trading in multiple assets with cross-effects.
problem Optimizing trade execution in multiple assets with cross-impact effects.
method Formulated as a stochastic control problem, extended to progressively measurable controls, solved using linear-quadratic control theory.
result Cross-hedging effects can be optimal, e.g., trading in an asset without an initial position.
Study policy gradient for large-agent mean-field control and game in continuous time.
problem Optimal policy learning for large number of agents in continuous-time mean-field systems.
method Policy gradient method applied to linear-quadratic mean-field control and game models.
result Policy gradient converges to optimal solution at a linear rate for both mean-field control and game.
Study cost-driven state representation learning for control from partial observations.
problem Learning state representation for control from partial and high-dimensional observations.
method Cost-driven state representation learning via predicting cumulative costs.
result Established finite-sample guarantees for near-optimal representation and controller.
This paper studies a class of continuous-time scalar-state stochastic Linear-Quadratic (LQ) optimal control problem with the linear control constraints. Applying the state separation theorem induced from its special structure, we develop the explicit solution for this class of problem. The revealed optimal control poli…
Study shows PI for LQR requires fewer policy improvement steps than policy evaluation steps.
problem Understanding the sample complexity of RL algorithms for continuous control tasks.
method Finite-time analysis of approximate policy iteration for LQR, quantifying policy improvement and evaluation complexities.
result Policy evaluation is the dominant factor in sample complexity, requiring ( n + d ) 3 / ε 2 (n+d)^3/\varepsilon^2 ( n + d ) 3 / ε 2 samples per step. Policy gradient methods converge for LQR problems with noisy state dynamics.
problem Finding optimal policies in noisy LQR problems over finite time horizons.
method Policy gradient methods with convergence guarantees for finite time and stochastic state dynamics.
result Global linear convergence for policy gradient methods in LQR problems with weak assumptions.
New model-free algorithm achieves similar LQR regret guarantees.
problem Model-free control of linear dynamical systems under quadratic costs.
method Online policy gradient scheme with policy space cost analysis.
result Achieves regret scaling with √T, matching model-based methods.
We consider adaptive control of the Linear Quadratic Regulator (LQR), where an unknown linear system is controlled subject to quadratic costs. Leveraging recent developments in the estimation of linear systems and in robust controller synthesis, we present the first provably polynomial time algorithm that provides high…
New algorithm achieves logarithmic regret for adversarial online control.
problem Online linear-quadratic control in systems with adversarial disturbances.
method Characterization of optimal offline control law, reduced to online learning with approximate advantage functions.
result First algorithm with logarithmic regret for arbitrary adversarial disturbance sequences.
Study shows model-based methods require fewer samples than model-free methods for LQR tasks.
problem Comparing model-based and model-free methods in reinforcement learning for continuous control tasks.
method An asymptotic analysis of sample complexity for policy evaluation in LQR tasks.
result Model-based methods require asymptotically less samples than model-free methods for policy evaluation in LQR tasks.
Policy optimization converges to Nash equilibria in zero-sum LQ games.
problem Finding Nash equilibria in zero-sum linear quadratic games.
method Developed three projected nested-gradient methods to converge to NE.
result Policy optimization methods converge to Nash equilibria in zero-sum LQ games.
This paper studies how gradient descent in control systems can perform well on unseen data.
problem The extent of a learned controller's ability to extrapolate to unseen initial states.
method Theoretical study of policy gradient in Linear Quadratic Regulator (LQR) problems, focusing on the role of exploration.
result The performance of a learned controller on unseen initial states depends on the degree of exploration induced by the system.
In this paper, we continue our study on a general time-inconsistent stochastic linear--quadratic (LQ) control problem originally formulated in [6]. We derive a necessary and sufficient condition for equilibrium controls via a flow of forward--backward stochastic differential equations. When the state is one dimensional…
Theory is developed for linear-quadratic at infinity generating families for Legendrian knots in R^3. It is shown that the unknot with maximal Thurston--Bennequin invariant of -1 has a unique linear-quadratic at infinity generating family, up to fiber-preserving diffeomorphism and stabilization. From this, invariant ge…
The paper solves TIC LQ control problems using stochastic differential games.
problem Time-inconsistent linear-quadratic stochastic control problems.
method Stochastic differential games, spike variation approach.
result Achieves Nash equilibrium for TIC problems, demonstrating impact of ambiguity aversion.
Study learns state representations from observations for control, proving guarantees.
problem Learning state representations from high-dimensional observations for control.
method Cost-driven approach, learning latent state model to predict costs.
result Proves finite-sample guarantees for near-optimal state representation and controller.