Paper develops online learning-based risk-averse MPC for uncertain systems.
problem Designing robust MPC for systems with unknown but inferable stochastic disturbances.
method Proposes a novel online learning framework using CVaR constraints and Dirichlet process mixture models.
result Demonstrates improved robustness and adaptability of MPC in handling time-varying disturbance distributions.
Paper uses imitation learning to create efficient insulin policies from MPC demonstrations.
problem Resource-constrained medical devices struggle with complex MPC optimizations and state estimation errors.
method Imitation learning of neural network policies from MPC-computed demonstrations, using Bayesian inference with Monte Carlo Dropout.
result Trained policies generalize well to different patient cohorts, outperforming traditional MPC with state estimation.
Proposes a recursive MPC scheme with probabilistic safety guarantees for uncertain dynamic systems.
problem Probabilistic safety guarantees for MPC in dynamic environments with unknown stochastic agents.
method Uses conformal prediction to derive high-confidence prediction regions and gradually relax safety constraints online.
result Ensures recursive feasibility of MPC schemes by relaxing safety constraints over time.
We present foundations for using Model Predictive Control (MPC) as a differentiable policy class for reinforcement learning in continuous state and action spaces. This provides one way of leveraging and combining the advantages of model-free and model-based approaches. Specifically, we differentiate through MPC by usin…
Bayesian optimisation tackles stochastic MPC hyper-parameter tuning.
problem Fine-tuning hyper-parameters in stochastic MPC models.
method Heteroscedastic Bayesian optimisation framework.
result Framework effectively tunes hyper-parameters in control problems.
MPC outperforms reactive budgeting in non-stationary return environments.
problem Optimizing budget allocation under non-stationary returns.
method Receding-horizon Model Predictive Control (MPC) compared to reactive policies.
result MPC consistently outperforms reactive budgeting when return dynamics are predictable.
Review of integrating Bayesian methods with neural network-based MPC.
problem Lack of standardized benchmarks and reliable analyses in Bayesian MPC.
method Systematic analysis of Bayesian methods in neural-network-based MPC.
result Need for standardized benchmarks, ablation studies, and transparent reporting.
This paper develops a framework for training and evaluating neural networks for MPC.
problem Lack of a general framework for characterizing learning approaches in MPC.
method Developed a framework using PyTorch and CVXPY, incorporating hit-and-run sampling for efficient training data generation.
result Proposed metrics for validating neural network-based MPC approaches.
Efficient learning-based MPC for unknown nonlinear systems with state constraints.
problem Control of discrete-time nonlinear systems with unknown dynamics and state constraints.
method Receding horizon reinforcement learning (r-LPC) using Koopman operator-based prediction model.
result Proven closed-loop recursive feasibility, robustness, and asymptotic stability under function approximation errors.
Iterative method learns unknown constraints for MPC control.
problem Learning to satisfy unknown polyhedral state constraints in iterative MPC.
method Collects and improves estimates of unknown constraints using collected data, designs an MPC controller to satisfy the estimated constraints.
result Robust and probabilistic guarantees of constraint satisfaction as a function of task iterations.
The paper explores how ReLU DNNs can represent MPC policies and vice versa.
problem Representing MPC policies as ReLU DNNs and vice versa.
method Developed an approximate method for identifying input-space in ReLU nets resulting in PWA functions over polyhedral regions. Studied inverse multiparametric linear or quadratic programs for reconstruction of constraints and cost functions given a PWA function.
result Identification and representation of MPC policies as ReLU DNNs and vice versa.
EHR-MPC optimizes sepsis treatment using digital twins and inference-time control.
problem Optimal sepsis treatment policies are contested and difficult to adapt during inference.
method EHR-MPC decouples learning patient dynamics from treatment optimization, enabling inference-time control over learned digital twins.
result EHR-MPC achieves comparable off-policy performance and improved simulation performance compared to RL baselines.
We prove that the kernels of the restrictions of symplectic Dirac or symplectic Dirac-Dolbeault operators on natural subspaces of polynomial valued spinor fields are finite dimensional on a compact symplectic manifold. We compute those kernels for the complex projective spaces. We construct injections of subgroups of t…
Model-based Reinforcement Learning (MBRL) allows data-efficient learning which is required in real world applications such as robotics. However, despite the impressive data-efficiency, MBRL does not achieve the final performance of state-of-the-art Model-free Reinforcement Learning (MFRL) methods. We leverage the stren…
Model predictive control (MPC) has become one of the well-established modern control methods for three-phase inverters with an output LC filter, where a high-quality voltage with low total harmonic distortion (THD) is needed. Although it is an intuitive controller, easy to understand and implement, it has the significa…
A novel Q-learning algorithm connects information theory to MPC, improving control tasks.
problem Combining model-free RL and MPC for real-world systems with expensive queries and uncertain dynamics.
method Developed a Q-learning algorithm that uses entropy regularization and leverages biased models.
result Improves control performance on sim-to-sim tasks compared to optimal control and RL from scratch.
Deep neural networks improve chemical reactor control using MPC.
problem Improving control of chemical reactors with neural networks.
method Training neural networks on model predictive control (MPC) for reactor control.
result Neural network can mimic MPC control inputs while maintaining constraints.
Enhances PlaNet for better planning in uncertain environments.
problem Improving deep planning networks for partially observable environments.
method Incorporates Bayesian inference to handle uncertainty in latent models and action candidates.
result Consistently improves asymptotic performance on continuous control tasks.
Differentially private method for synthetic data generation from vertically partitioned data.
problem Generating synthetic data from vertically partitioned data while preserving privacy.
method Differentially private stochastic gradient descent (DP-SGD) algorithm combined with secure multiparty computation (MPC).
result Comparable accuracy to non-partitioned data, demonstrating privacy-preserving synthetic data generation.
MPC framework reduces execution costs and schedule deviations in trading.
problem Executing large orders in markets under time and liquidity constraints.
method Model Predictive Control (MPC) framework balancing order completion, market impact, and opportunity cost.
result Significant reductions in slippage and schedule shortfall compared to benchmarks.
In recent studies on model-based reinforcement learning (MBRL), incorporating uncertainty in forward dynamics is a state-of-the-art strategy to enhance learning performance, making MBRLs competitive to cutting-edge model free methods, especially in simulated robotics tasks. Probabilistic ensembles with trajectory sampl…
Robo-advisors use MPC to create dynamic investment strategies.
problem Static allocation methods limit robo-advisors' effectiveness.
method Combines MPC with Hidden Markov Model and Black-Litterman for dynamic asset allocation.
result MPC-based strategies outperform static approaches in dynamic and risk-budgeting criteria.
Given a symplectic manifold (M,ω) admitting a metaplectic structure, and choosing a positive ω-compatible almost complex structure J and a linear connection ∇ preserving ω and J, Katharina and Lutz Habermann have constructed two Dirac operators D and ${\wt{D}}$ acting on sections of a bundle of sympl…
Simulation-to-real transfer is an important strategy for making reinforcement learning practical with real robots. Successful sim-to-real transfer systems have difficulty producing policies which generalize across tasks, despite training for thousands of hours equivalent real robot time. To address this shortcoming, we…
The paper proposes a control strategy for systems with sparse parameters using compressed sensing.
problem Control of linear systems with unknown sparse parameters under disturbances.
method Sparse estimation using Recursive Least Squares, improved with Basis Pursuit Denoising, and reformulated probabilistic constraints.
result The proposed algorithm outperforms existing methods in control design for systems with sparse impulse response parameters.
Paper tackles SMPC for linear systems with unknown noise distribution.
problem Stochastic MPC for linear systems with chance state constraints and unknown noise distribution.
method Reformulate chance constraints, design robust benchmark SMPC, and develop adaptive SMPC with online noise statistics learning.
result Adaptive SMPC guarantees time-uniform satisfaction of unknown reformulated state constraints with high probability.
We describe an optimal adversarial attack formulation against autoregressive time series forecast using Linear Quadratic Regulator (LQR). In this threat model, the environment evolves according to a dynamical system; an autoregressive model observes the current environment state and predicts its future values; an attac…
Optimizes asset allocation with illiquid assets using MPC.
problem Strategic asset allocation with illiquid alternative asset classes.
method Formulates illiquid dynamics as a random linear system and proposes a convex optimization based MPC policy.
result Performance close to a fully liquid scenario, despite time delay and uncertainty.
In this paper, we introduce an actor-critic algorithm called Deep Value Model Predictive Control (DMPC), which combines model-based trajectory optimization with value function estimation. The DMPC actor is a Model Predictive Control (MPC) optimizer with an objective function defined in terms of a value function estimat…
The paper tackles ride-hailing fleet repositioning with a calibrated demand approach.
problem Repositioning idle supply before future demand is observed in ride-hailing.
method A predict-then-optimize approach using calibrated demand regimes, a similarity gate, and spatial queue-regret decomposition.
result The spatial gate reduces mean wait time to 82.3s compared to 85.3s for a hand-tuned similarity gate and 85.8s for a distributional-only baseline.
Enhanced metrics for multiclass classification improve on existing methods.
problem Lack of decisive poor classification results in existing multiclass metrics.
method Introduces three new metrics derived from multivariate Pearson correlation coefficients.
result New metrics decisively indicate poor classification results.
MVPI framework optimizes risk in reinforcement learning, improving performance in robot simulations.
problem Optimizing risk in reinforcement learning control problems.
method Mean-Variance Policy Iteration (MVPI) framework for risk-averse control in MDPs.
result Risk-averse TD3 outperforms previous methods in robot simulation tasks.
Different models of capital exchange among economic agents have been proposed recently trying to explain the emergence of Pareto's wealth power law distribution. One important factor to be considered is the existence of risk aversion. In this paper we study a model where agents posses different levels of risk aversion,…
Bayesian Gaussian Processes improve exoplanet transit and Hubble constant inference.
problem Improving exoplanet transit and Hubble constant inference using Bayesian Gaussian Processes.
method Kernel-, mean- and noise-marginalised Gaussian Processes with evidence-based model comparison and transdimensional sampling.
result Inferred Hubble constant H0 values from cosmic chronometers, baryon acoustic oscillations and combined datasets are 66±6kms−1Mpc−1, 67±10kms−1Mpc−1 and 69±6kms−1Mpc−1, respectively. A new algorithm avoids worst-case outcomes in risky contexts.
problem Risk-averse behavior in contextual bandits is challenging.
method Developed a first risk-averse contextual bandit algorithm with online regret guarantees.
result First algorithm with an online regret guarantee for risk-averse contextual bandits.
A new measure quantifies how risk-averse different risk measures are.
problem Measuring the degree of risk aversion among different risk measures.
method Two axioms: normalization and linearity. Two formulas for the functional.
result Quantifies the degree of risk aversion among spectral risk measures.
Robo-advisors estimate clients' risk aversion using interactive questionnaires.
problem Estimating risk aversion of non-expert clients using adaptive questionnaires.
method Model risk aversion with cost functions and spectral risk measures. Use inverse reinforcement learning to design questions maximizing distinguishing power.
result Designing questions by maximizing distinguishing power achieves satisfactory accuracy in learning risk aversion with fewer than 50 questions.
Risk aversion is a key element of utility maximizing hedge strategies; however, it has typically been assigned an arbitrary value in the literature. This paper instead applies a GARCH-in-Mean (GARCH-M) model to estimate a time-varying measure of risk aversion that is based on the observed risk preferences of energy hed…
Modeling informed trading with risk-averse market makers.
problem Understanding informed trading and its impact on market liquidity and risk premia.
method Connections between optimal transport theory and Kyle's model, including new characterizations of profits and duality.
result Liquidity is lower, assets exhibit short-term reversals, and risk premia depend on market maker inventories, which are mean reverting.
Study risk-averse insider's behavior in dynamic signal asset pricing.
problem Analyzing risk-averse insider's dynamic signal in asset pricing.
method Employing a weak conditioning methodology to construct a Schrödinger bridge, deriving necessary conditions for equilibrium.
result Derive explicit closed-form solutions for important cases.
Privacy concern has been increasingly important in many machine learning (ML) problems. We study empirical risk minimization (ERM) problems under secure multi-party computation (MPC) frameworks. Main technical tools for MPC have been developed based on cryptography. One of limitations in current cryptographically priva…
The standard asset pricing models (the CCAPM and the Epstein-Zin non-expected utility model) counterintuitively predict that equilibrium asset prices can rise if the representative agent's risk aversion increases. If the income effect, which implies enhanced saving as a result of an increase in risk aversion, dominates…
Investigates how diversification preferences relate to risk attitudes.
problem Connecting diversification preferences to risk attitudes.
method Analyzes diversification preferences for various pairs of risks under different conditions.
result Diversification preferences for certain pairs of risks imply specific levels of risk aversion.
New insights into risk aversion for complex decision models.
problem Understanding risk aversion in non-monotone decision models.
method Characterization of probabilistic risk aversion for generalized rank-dependent functions.
result Probabilistic risk aversion is determined by the distortion function, which is convex or scaled quantile-spread mixtures.
Unified formula for optimal portfolio under piecewise hyperbolic risk aversion.
problem Optimizing portfolios with piecewise hyperbolic risk aversion utilities.
method Derive a unified closed-form formula for the optimal portfolio.
result Unified formula reflects risk aversion behaviors and risk-taking behaviors.
Investigates a Kyle model with imperfect information and risk aversion.
problem Tackles a Kyle model with imperfect information and risk-averse informed traders.
method Solves an optimal transport problem and a filtering problem under specific measures.
result Constructs an equilibrium for the Gaussian Kyle model with imperfect information and risk aversion.
Deep Hedging learns optimal strategies for various risk levels.
problem Finding optimal hedging policies for diverse risk aversions.
method Continuous Reinforcement Learning with actor-critic algorithm.
result Demonstrated effectiveness in a stochastic volatility model.
Risk-averse reinforcement learning optimizes option hedging.
problem Optimizing option hedging under risk aversion and realistic market conditions.
method Applied Trust Region Volatility Optimization (TRVO) to a vanilla option hedging environment.
result The derived hedging strategy outperforms Black & Scholes and is robust to market variations.