This work proposes an online learning approach to tighten constraints in stochastic control problems.
problem Solving chance-constrained stochastic optimal control problems is computationally challenging.
method Reformulate chance constraints as a binary regression problem and use a GP model to learn constraint-tightening parameters online.
result The approach tightens constraints more effectively, leading to lower costs in numerical experiments.
Iterative method learns unknown constraints for MPC control.
problem Learning to satisfy unknown polyhedral state constraints in iterative MPC.
method Collects and improves estimates of unknown constraints using collected data, designs an MPC controller to satisfy the estimated constraints.
result Robust and probabilistic guarantees of constraint satisfaction as a function of task iterations.
Study uses DRL with Lagrangian relaxation to solve temporal control tasks with STL constraints.
problem Optimal control problems with temporal logic constraints.
method Extended CMDP formulation, Lagrangian relaxation, two-phase constrained DRL algorithm.
result Demonstrated learning performance of the proposed algorithm through simulations.
The paper solves a consumption-investment problem with state-dependent lower bounds.
problem A life-time consumption-investment problem with a state-dependent lower bound on consumption.
method Transformed the problem into a state-independent control problem to apply standard theory.
result Explicit optimal strategies provided for both homogeneous and non-homogeneous constraints.
Controller-Augmented Hidden Markov Models (CHMMs) are a framework for constrained sequential inference.
problem Hidden Markov models fail under pathwise constraints like precedence, visitation, or monotonic state progression.
method CHMMs compile constraints into finite-state controllers, then use standard forward-backward and Viterbi recursions to compute exact constrained posteriors and paths.
result CHMMs provide exact constrained inference, monotone ascent in constrained EM, and linear complexity in controller cardinality.
New boundary and point constraints for controlling conformal surfaces.
problem Controlling the geometry of surfaces defined by minimizers of conformal variational problems.
method Introducing new boundary conditions, point constraints, and flux constraints to control the metric and conformal scale factor.
result Introduces intuitive controls for exploring a subspace of conformal immersions.
This paper proposes a method to safely adjust exploration in RL to satisfy constraints.
problem Unsafe exploration in reinforcement learning violates constraints on controlled object states.
method Automatic adjustment of exploration inputs and variance-covariance matrix for safety.
result The method guarantees satisfaction of joint chance constraints with specified probability.
Study optimal consumption with relaxed benchmarks and drawdown constraints.
problem Optimal consumption under relaxed benchmark tracking and consumption drawdown constraint.
method Transformed stochastic control problem into regular control problem with state-control constraints, then solved using dual transform and optimal consumption behavior.
result Closed-form solution for optimal investment and consumption in feedback form.
Solves optimal control with state constraints using probabilistic methods.
problem Optimal control of diffusion processes within state constraints.
method Probabilistic representation and optimal control under mild conditions.
result Explicit formulae for optimally controlled dynamics in examples.
Counterexample shows state-constrained optimal control problems can have Young measure gaps.
problem Existence of Young measure gaps in state-constrained optimal control problems.
method Provided a counterexample for smooth controllable systems state-constrained to the unit ball.
result Gap occurs in a regular setting with non-convex Lagrangian density.
New principle for optimal control with higher order differential constraints.
problem Optimal control problems with higher order differential constraints.
method Derivation of the Principle of Minimal Labour and generalization of Pontryagin Maximum Principle.
result Generalized Pontryagin Maximum Principle for higher order constraints.
Optimal control problems on Riemannian manifolds are solved by penalizing constraint violations.
problem Optimal control problems with velocity constraints on Riemannian manifolds.
method Penalizing constraint violations and showing convergence to hard-constrained solutions.
result Solutions to soft-constrained problems converge to solutions of hard-constrained problems as penalty parameter increases.
In order to satisfy safety conditions, an agent may be constrained from acting freely. A safe controller can be designed a priori if an environment is well understood, but not when learning is employed. In particular, reinforcement learned (RL) controllers require exploration, which can be hazardous in safety critical …
Meta-learning control algorithm with finite-time guarantees for unknown systems.
problem Online control of unknown linear systems with constraints.
method Provable regret guarantees for an iterative control algorithm.
result Regret bounds of O(T3/4) for controller cost and constraint violation. Improved text generation with constraints using discrete auto-regressive biasing.
problem Balancing fluency and constraint satisfaction in LLM outputs.
method Discrete Auto-regressive Biasing, leveraging gradients in discrete text space.
result Significantly improved constraint satisfaction with comparable fluency.
Framework for controlling multiple risks in AI models.
problem Enforcing multiple risk constraints in generative AI models.
method Formalizes problem, introduces two dynamic programming algorithms.
result Achieves nearly tight control of all constraint risks under mild assumptions.
New control theory for self-path-dependent problems solves unique constraints.
problem Optimal control with self-path-dependent constraints in stochastic systems.
method Introduces new HJB equations for variational inequalities with historical maximum controls.
result Value functions are viscosity solutions to HJB equations under Lipschitz conditions.
Paper proposes Vertex Networks for reinforcement learning of control systems with safety guarantees.
problem Challenges in reinforcement learning with hard state and action constraints.
method Vertex Networks incorporate safety constraints into policy network architecture, ensuring safety during exploration.
result Proposed Vertex Networks outperform vanilla reinforcement learning in benchmark control tasks.
Paper solves MV portfolio selection in jump-diffusion models with no-shorting constraint.
problem Mean-variance portfolio selection in jump-diffusion model with no-shorting constraint.
method Reduces problem to LQ control and finding a maximal point of a function, constructs viscosity solution.
result Explicit viscosity solution to Hamilton-Jacobi-Bellman equation, optimal controls derived.
Geometrically characterizes virtual nonlinear nonholonomic constraints using symplectic methods.
problem Characterizing virtual nonlinear nonholonomic constraints geometrically.
method Geometric characterization using symplectic structures and Chetaev equations.
result A unique control law exists to satisfy virtual constraints, and closed-loop dynamics are projections of uncontrolled dynamics.
Study magnetic Hamiltonian systems with constraints, deriving Hamilton-Jacobi equations.
problem Understanding dynamics of controlled magnetic Hamiltonian systems with constraints.
method Defined CMH system, derived Hamilton-Jacobi equations for different constraints.
result Invariant solutions of Hamilton-Jacobi equations under CMH-equivalence.
Control Barrier Functions (CBF) have been recently utilized in the design of provably safe feedback control laws for nonlinear systems. These feedback control methods typically compute the next control input by solving an online Quadratic Program (QP). Solving QP in real-time can be a computationally expensive process …
This paper optimizes dividend payout rates with a drawdown constraint in a stochastic model.
problem Optimizing dividend payout rates while avoiding drawdowns in a stochastic model.
method Solving a path-dependent stochastic control problem using Hamilton-Jacobi-Bellman equations and PDE methods.
result Explicit characterization of an optimal feedback control strategy, including two free boundaries and the running maximum surplus process.
We study the constrained linear quadratic regulator with unknown dynamics, addressing the tension between safety and exploration in data-driven control techniques. We present a framework which allows for system identification through persistent excitation, while maintaining safety by guaranteeing the satisfaction of st…
Robust model predictive control (MPC) is a well-known control technique for model-based control with constraints and uncertainties. In classic robust tube-based MPC approaches, an open-loop control sequence is computed via periodically solving an online nominal MPC problem, which requires prior model information and fr…
Combines Gaussian processes and polynomial chaos for stochastic control.
problem Uncertainties in dynamic models lead to performance issues in predictive control.
method Combines Gaussian processes with polynomial chaos expansions to estimate probability distributions of nonlinear functions.
result Demonstrates accurate approximation and closed-loop performance in stochastic nonlinear model predictive control.
In this paper, we show the implementation of deep neural networks applied in process control. In our approach, we based the training of the neural network on model predictive control. Model predictive control is popular for its ability to be tuned by the weighting matrices and by the fact that it respects the constrain…
We improve Riemannian metrics for constrained systems control.
problem Controlling mechanical systems with configuration constraints.
method Constructing complete Riemannian metrics by modifying incomplete ones.
result A controller can be found to satisfy a design criterion.
Paper studies constrained control games with a novel approximation method.
problem Games with constrained control directions.
method Approximation procedure based on L1-stability estimates and almost sure convergence. result Existence of game's value and optimal strategy for the stopper.
A novel controller for wheeled robots handles joystick inputs for smooth steering.
problem Steering control for differential-drive wheeled robots from indirect joystick inputs.
method Developed a geometric controller based on Darboux frame kinematics.
result Smooth trajectories achieved with safety constraints and no desired states.
We provide a probabilistic solution of a not necessarily Markovian control problem with a state constraint by means of a Backward Stochastic Differential Equation (BSDE). The novelty of our solution approach is that the BSDE possesses a singular terminal condition. We prove that a solution of the BSDE exists, thus part…
The problem of multi-hypothesis testing with controlled sensing of observations is considered. The distribution of observations collected under each control is assumed to follow a single-parameter exponential family distribution. The goal is to design a policy to find the true hypothesis with minimum expected delay whi…
New method controls renewable energy storage and portfolio selection with probabilistic constraints.
problem Control of McKean-Vlasov dynamics with probabilistic state constraints.
method Level-set approach for exact penalization and running maximum/integral cost.
result Extension to mean-field setting with machine learning algorithm.
We present a novel technique to solve the problem of managing optimally a pumped hydroelectric storage system. This technique relies on representing the system as a stochastic optimal control problem with state constraints, these latter corresponding to the finite volume of the reservoirs. Following the recent level-se…
This paper considers an optimal control of a big financial company with debt liability under bankrupt probability constraints. The company, which faces constant liability payments and has choices to choose various production/business policies from an available set of control policies with different expected profits and…
MuJAM learns traffic signal control policies that generalize to unseen intersections and traffic conditions.
problem Lack of transferability in reinforcement learning methods for traffic signal control.
method Model-based graph reinforcement learning with explicit coordination and generalization to both cyclic and acyclic constraints.
result MuJAM outperforms existing methods in zero-shot and larger transfer settings.
Safe learning of stochastic dynamics with safety constraints.
problem Learning controlled stochastic dynamics with safety constraints.
method Iterative expansion of a safe control set using kernel-based confidence bounds.
result The method ensures safe exploration and efficient estimation of system dynamics.
We provide a dynamic programming principle for stochastic optimal control problems with expectation constraints. A weak formulation, using test functions and a probabilistic relaxation of the constraint, avoids restrictions related to a measurable selection but still implies the Hamilton-Jacobi-Bellman equation in the …
Voltage control plays an important role in the operation of electricity distribution networks, especially with high penetration of distributed energy resources. These resources introduce significant and fast varying uncertainties. In this paper, we focus on reactive power compensation to control voltage in the presence…
This work establishes safe reinforcement learning for LQR with nonlinear baselines.
problem Safe reinforcement learning in LQR with unknown dynamics and safety constraints.
method General framework for nonlinear baselines, focusing on 1D spaces.
result Achieves optimal regret bounds for constrained reinforcement learning.
In this paper, we work in the framework of the Merton problem but we impose a drawdown constraint on the consumption process. This means that consumption can never fall below a fixed proportion of the running maximum of past consumption. In terms of economic motivation, this constraint represents a type of habit format…
We solve a class of control problems with fuel constraint by means of the log-Laplace transforms of J-functionals of Dawson-Watanabe superprocesses. This solution is related to the superprocess solution of quasilinear parabolic PDEs with singular terminal condition. For the probabilistic verification proof, we develo…
New method controls posterior collapse in VAEs without network architecture constraints.
problem Posterior collapse in VAEs reduces diversity of generated samples.
method Introduces Latent Reconstruction (LR) loss to control posterior collapse.
result Controls posterior collapse on various datasets without architectural constraints.
Optimizes bank capital structure under Basel III constraints, simplifying complex dynamics.
problem Optimizing risky investments, dividends, and capital structure under Basel III constraints.
method Formulated as a stochastic control problem, reducing dynamics to a one-dimensional process in leverage ratio.
result Simple policy: pay dividends at an upper barrier and recapitalize at the distress boundary.
Proposes methods to add constraints to neural networks to improve stability and generalization.
problem Improving stability and generalization of neural networks.
method Constraint-based regularization using stochastic gradient Langevin dynamics.
result Constraints help stabilize and improve the robustness of deep neural networks.
DePAint solves MARL for agents with local constraints, privacy, and no central controller.
problem Training multi-agent systems to optimize rewards while adhering to safety constraints in a decentralized setting.
method Formulated as a decentralized constrained multi-agent Markov Decision Problem, proposed DePAint method using momentum-based decentralized policy gradient.
result First privacy-preserving fully decentralized MARL algorithm considering both peak and average constraints.
The paper optimizes policies constrained to Schur stabilizing controllers using a Newton-type algorithm.
problem Optimizing policies under linear constraints in control systems.
method Newton-type algorithm on a manifold of Schur stabilizing controllers with a Riemannian metric.
result Local convergence guarantees for the Newton-type algorithm without relying on exponential mapping or retractions.
Algorithm safely learns from sub-optimal baseline policies while satisfying constraints.
problem Safe reinforcement learning with constraints when baseline policy is sub-optimal.
method Iterative policy optimization alternating between return maximization, baseline distance minimization, and constraint projection.
result Consistently outperforms baselines, achieving 10x fewer constraint violations and 40% higher reward.