Paper proposes tensor-based method for semiconductor manufacturing process control.
problem Challenges of traditional process control methods in high-dimensional image-based overlay errors.
method Builds a high-dimensional process model, proposes tensor-on-vector regression algorithms, designs EWMA controller for tensor data.
result The method reduces overlay errors using limited control recipes and is superior especially when disturbances are not stable.
MFRL-BI controls manufacturing processes without needing accurate models.
problem Model inaccuracies in complex manufacturing systems.
method Model-free reinforcement learning with Bayesian inference.
result Demonstrated to perform well in a nonlinear CMP process.
Study optimal control of diffusion processes with infimum or supremum costs.
problem Optimizing control of a diffusion process with costs dependent on its infimum or supremum.
method Introduced novel integral operators to solve two-dimensional singular control problems.
result Explicit solutions for optimal dividend problem with time-dependent preferences.
Safe RL for safety-critical tasks using CBFs and model-free RL.
problem Limited safety guarantees in RL for real-world applications.
method Combines model-free RL with CBFs and GP dynamics modeling.
result Demonstrates greater sample efficiency and safety in learning.
Anticipatory model generates music with control over events.
problem Controlling symbolic music generation.
method Interleaving event and control sequences to predict future events.
result Anticipatory model matches autoregressive models in performance and can infill control tasks.
We approach the development of models and control strategies of susceptible-infected-susceptible (SIS) epidemic processes from the perspective of marked temporal point processes and stochastic optimal control of stochastic differential equations (SDEs) with jumps. In contrast to previous work, this novel perspective is…
A linear model approximates Gaussian processes for efficient control.
problem Efficiently modeling and controlling Gaussian processes with many parameters.
method Developed a linear model using basis functions to approximate Gaussian processes.
result The linear model improves computational efficiency and feasibility of control strategies.
The paper solves a control problem using reflections to track a benchmark process.
problem Optimal consumption with a benchmark process that grows over time.
method Introduced two auxiliary state processes with reflections to transform the problem into a more tractable form.
result Established the existence of a unique classical solution to the dual PDE.
Unified approach for data-driven control of stochastic processes.
problem Developing practical strategies for stochastic control problems with unknown dynamics.
method Reduction to rate-optimal estimators of invariant distribution risk.
result Data-driven strategies can achieve better performance than known methods.
Paper combines control and statistics for learning process dynamics.
problem Learning dynamics of unknown continuous-time processes for control.
method Combining stochastic control and nonparametric statistics methods.
result Average cumulated regret is of uniform order O ( T − 1 / 3 ) O({T^{-1/3}}) O ( T − 1/3 ) . Optimizes dividend control in a bankruptcy process using a special Levy process.
problem Optimizing dividend payouts in a bankruptcy process.
method Using a non-standard spectrally negative Levy process with endogenous regime switching.
result Optimal dividend control is of the barrier type and the optimal barrier can be identified.
For controlled discrete-time stochastic processes we introduce a new class of dynamic risk measures, which we call process-based. Their main features are that they measure risk of processes that are functions of the history of a base process. We introduce a new concept of conditional stochastic time consistency and we …
Paper derives uniform error bounds for Gaussian process regression for safer control applications.
problem Quantifying model error in Gaussian process regression for safety-critical applications.
method Employing Gaussian process distribution and continuity arguments, derive uniform error bounds under weaker assumptions.
result Derives novel uniform error bounds for Gaussian process regression under weaker assumptions.
Paper uses CNN to predict process parameters from molten pool data in WLAM.
problem Achieving high quality in WLAM through real-time process control.
method Proposes a multi-modality CNN architecture to predict process parameters from molten pool sensor data.
result Improved prediction performance of multi-modal CNN compared to uni-modal approach.
Deep RL controller outperforms market making benchmarks in a Hawkes process model.
problem Optimal market making in financial markets.
method Deep reinforcement learning on a Hawkes process-based simulator.
result Deep RL controller outperforms benchmarks in various risk-reward metrics.
New RL framework learns optimal control policies directly.
problem Directly learning optimal control policies in complex systems.
method General Markov Decision Process (MDP) framework for reinforcement learning.
result Convergence and optimality of control-based RL methods established.
Solves optimal control for stochastic processes with absorbing states.
problem Optimal control of stochastic processes with absorbing states.
method Solves through system of partial differential equations.
result Explicit solution for Merton portfolio problem with default probability.
Combines Gaussian processes and polynomial chaos for stochastic control.
problem Uncertainties in dynamic models lead to performance issues in predictive control.
method Combines Gaussian processes with polynomial chaos expansions to estimate probability distributions of nonlinear functions.
result Demonstrates accurate approximation and closed-loop performance in stochastic nonlinear model predictive control.
New approach to control diffusion processes with soft constraints.
problem Finding an optimal diffusion process with a target terminal distribution.
method Generalized Schrödinger bridge problem with soft constraints, solving for a geometric mixture of target and other distributions.
result The terminal distribution of the optimally controlled process is a geometric mixture of the target and another distribution.
New method tackles model uncertainty in stochastic control using Bayesian nonparametrics.
problem Model uncertainty in stochastic control problems.
method Nonparametric Bayesian approach with Dirichlet process for unknown distributions, online learning, and Gaussian process surrogates.
result Demonstrates financial advantages of nonparametric Bayesian over parametric methods.
Optimizes dividend policies in a Brownian model with controlled rates.
problem Realistic optimal dividend policies in a stochastic control problem.
method Delayed linear control strategies for refracted diffusion processes.
result Optimality of delayed linear control strategies for dividend payments.
Two methods monitor high-dimensional processes via manifold fitting or learning.
problem Monitoring high-dimensional, dynamic industrial processes.
method Manifold fitting and learning approaches for online SPC.
result Manifold-fitting approach achieves performance competitive with classical methods.
For large-scale industrial processes under closed-loop control, process dynamics directly resulting from control action are typical characteristics and may show different behaviors between real faults and normal changes of operating conditions. However, conventional distributed monitoring approaches do not consider the…
Unified reinforcement learning and stochastic processes with action-driven processes.
problem Combining reinforcement learning and stochastic processes for efficient control.
method Action-driven processes, leveraging control-as-inference, and minimizing Kullback-Leibler divergence.
result Action-driven processes unify reinforcement learning and stochastic processes, equivalent to maximum entropy reinforcement learning.
We solve a complex trade execution problem by simplifying it into a known LQ control problem.
problem Optimal trade execution with stochastic price impact and resilience.
method Extending the problem to progressively measurable processes and reducing it to a LQ stochastic control problem.
result The solution to the LQ problem traces back to the solution of the original trade execution problem.
New method uses neural nets to control systems safely with disturbances.
problem Designing safe control laws for systems with disturbances.
method Imitation learning to train neural network controllers that satisfy CBF constraints.
result Demonstrated on a unicycle model with external disturbances.
We study stochastic differential equations (SDEs) whose drift and diffusion coefficients are path-dependent and controlled. We construct a value process on the canonical path space, considered simultaneously under a family of singular measures, rather than the usual family of processes indexed by the controls. This val…
This work proposes a new method for simultaneous probabilistic identification and control of an observable, fully-actuated mechanical system. Identification is achieved by conditioning stochastic process priors on observations of configurations and noisy estimates of configuration derivatives. In contrast to previous w…
We consider a zero-sum stochastic differential controller-and-stopper game in which the state process is a controlled diffusion evolving in a multi-dimensional Euclidean space. In this game, the controller affects both the drift and the volatility terms of the state process. Under appropriate conditions, we show that t…
Paper tackles risk-sensitive impulse control for continuous-time processes.
problem Risk-sensitive impulse control for continuous-time Feller-Markov processes.
method Probabilistic approach to solve Bellman equation and construct optimal strategy.
result Optimal strategy approximated by dyadic impulse strategies.
Paper studies long-run risk optimization with dyadic impulses for unbounded processes.
problem Long-run risk optimization problem with unbounded and non-uniformly ergodic processes.
method Adapting weight norm approach, combining geometric drift and local minorization property.
result Existence of solution to Bellman equation for risk-averse parameters.
In this paper we consider an energy storage optimization problem in finite time in a model with partial information that allows for a changing economic environment. The state process consists of the storage level controlled by the storage manager and the energy price process, which is a diffusion process the drift of w…
A framework integrates machine learning with robust control for safer, more reliable systems.
problem Combining machine learning with robust control for systems with stringent safety and reliability requirements.
method Integrates Gaussian Process Regression and state-of-the-art robust controller synthesis within a framework that provides rigorous guarantees.
result Demonstrated improved performance with more data while maintaining rigorous guarantees.
Neural PID controllers improve control system performance and are more interpretable.
problem Lack of interpretability in neural PID controllers limits their use in control engineering.
method Extensive study using four benchmark systems with and without noise and disturbances, applying GDNN to PID controllers.
result Neural PID controllers outperform standard PID and model-based control in most tasks.
Risk-controlled post-processing optimizes decision policies under risk constraints.
problem Optimizing decision policies with risk constraints for better outcomes.
method Developed a post-processing algorithm that selects a threshold based on fitted fallback policy and score, leveraging tools from algorithmic stability and stochastic processes.
result The post-processed policy achieves precise expected risk control under exchangeability and meets or nearly meets risk budgets while preserving more agreement with the baseline.
A model optimizes carbon emission reduction and allowance purchasing for companies.
problem Optimizing carbon emissions and allowance purchasing for companies.
method Established an optimal control model involving two stochastic processes with two control variables, converted into an HJB equation, proved existence and uniqueness of solution.
result Proved the existence and uniqueness of the solution to the HJB equation.
Optimizes costs in uncertain Markov systems using risk filters.
problem Optimizing costs in systems with model uncertainty and unknown parameters.
method Risk filters and Bellman principle of optimality applied to Bayesian framework.
result Derives the Bellman principle for non-standard risk-averse control problems.
Study uses statistical methods to solve control problems with probabilistic constraints.
problem Optimizing control of systems with low probability of failure.
method Monte Carlo algorithms and statistical regression techniques.
result Logistic and Gaussian process regression outperform other methods in estimating admissibility probability.
We solve a broad class of sequential decision-making problems with partially observed states.
problem Sequential decision-making under uncertainty with partially observed states.
method Modeling as a partially observed Markov decision process (POMDP) and separating state and modulation process.
result The approach allows for specialized approximate solution procedures.
Study shows how to control jump-diffusion processes with stable feedback controls in reinforcement learning.
problem Control jump-diffusion processes with unknown coefficients in reinforcement learning.
method Lipschitz continuous optimal feedback controls, stability analysis of forward-backward SDEs, least-squares algorithm.
result Achieves O ( N ln N ) O(\sqrt{N\ln N}) O ( N ln N ) regret for linear-convex learning problems with jumps. This paper develops numerical methods for finding optimal dividend pay-out and reinsurance policies. A generalized singular control formulation of surplus and discounted payoff function are introduced, where the surplus is modeled by a regime-switching process subject to both regular and singular controls. To approxima…
Study uses viscosity solutions to solve control problems involving measure-valued martingales.
problem Stochastic control problems with measure-valued martingale state processes.
method Viscosity solution approach exploiting structural properties of MVM processes.
result Value function is the unique viscosity solution to the HJB equation.
We consider a controlled diffusion process ( X t ) t ≥ 0 (X_t)_{t\ge 0} ( X t ) t ≥ 0 where the controller is allowed to choose the drift μ t μ_t μ t and the volatility σ t σ_t σ t from a set $\K(x) \subset \R\times (0,\infty)$ when X t = x X_t=x X t = x . By choosing the largest μ σ 2 \fracμ{σ^2} σ 2 μ at every point in time an extremal process is constructed which is under suita…
Nonparametric adaptive robust control tackles model uncertainty in stochastic processes.
problem Model uncertainty in stochastic processes.
method Adaptive robust control methodology using online learning and uncertainty reduction, empirical distribution, and Lagrangian duality.
result Nonparametric adaptive robust control approach is preferable to traditional robust frameworks.
Abstract: Surveying connections between ML and Control Theory.
problem Addressing the intersection of Machine Learning and Control Theory.
method Develops connections through reinforcement learning, supervised learning, deep learning, and stochastic gradient descent.
result Machine Learning and Control Theory are interconnected, with ML solving large control problems and Control Theory providing tools for ML.
ARPs improve exploration and sample efficiency in continuous control tasks.
problem Limited exploration in continuous control tasks leading to low sample efficiency.
method Introduce autoregressive policies (ARPs) with temporally coherent standard normal distributions.
result ARPs enhance exploration and sample efficiency in both simulated and real-world domains.
The paper tackles learning to control systems with unknown parameters using Brownian noise.
problem Learning to control systems with unknown parameters.
method Proposes algorithms based on moving empirical averages and integrates statistical methods with stochastic control theory.
result Achieves a logarithmic expected regret rate.
Optimal investment and risk control strategies for insurers are derived using a time-consistent approach.
problem Optimal investment and risk control for insurers under mean-variance criterion.
method Introducing a deterministic forward auxiliary process to formulate a time-consistent problem.
result Optimal strategy and value function obtained in closed-form for the new problem.