A new Bayesian MBRL method improves performance in robotics tasks.
problem Enhancing model-based reinforcement learning with uncertainty.
method Introduces variational inference MPC and probabilistic action ensembles with trajectory sampling (PaETS).
result Consistently improves performance on challenging locomotion tasks.
Enhances PlaNet for better planning in uncertain environments.
problem Improving deep planning networks for partially observable environments.
method Incorporates Bayesian inference to handle uncertainty in latent models and action candidates.
result Consistently improves asymptotic performance on continuous control tasks.
Differentially private method for synthetic data generation from vertically partitioned data.
problem Generating synthetic data from vertically partitioned data while preserving privacy.
method Differentially private stochastic gradient descent (DP-SGD) algorithm combined with secure multiparty computation (MPC).
result Comparable accuracy to non-partitioned data, demonstrating privacy-preserving synthetic data generation.
Paper develops online learning-based risk-averse MPC for uncertain systems.
problem Designing robust MPC for systems with unknown but inferable stochastic disturbances.
method Proposes a novel online learning framework using CVaR constraints and Dirichlet process mixture models.
result Demonstrates improved robustness and adaptability of MPC in handling time-varying disturbance distributions.
EHR-MPC optimizes sepsis treatment using digital twins and inference-time control.
problem Optimal sepsis treatment policies are contested and difficult to adapt during inference.
method EHR-MPC decouples learning patient dynamics from treatment optimization, enabling inference-time control over learned digital twins.
result EHR-MPC achieves comparable off-policy performance and improved simulation performance compared to RL baselines.
Paper uses imitation learning to create efficient insulin policies from MPC demonstrations.
problem Resource-constrained medical devices struggle with complex MPC optimizations and state estimation errors.
method Imitation learning of neural network policies from MPC-computed demonstrations, using Bayesian inference with Monte Carlo Dropout.
result Trained policies generalize well to different patient cohorts, outperforming traditional MPC with state estimation.
Bayesian Gaussian Processes improve exoplanet transit and Hubble constant inference.
problem Improving exoplanet transit and Hubble constant inference using Bayesian Gaussian Processes.
method Kernel-, mean- and noise-marginalised Gaussian Processes with evidence-based model comparison and transdimensional sampling.
result Inferred Hubble constant H0 values from cosmic chronometers, baryon acoustic oscillations and combined datasets are 66±6kms−1Mpc−1, 67±10kms−1Mpc−1 and 69±6kms−1Mpc−1, respectively. Proposes a recursive MPC scheme with probabilistic safety guarantees for uncertain dynamic systems.
problem Probabilistic safety guarantees for MPC in dynamic environments with unknown stochastic agents.
method Uses conformal prediction to derive high-confidence prediction regions and gradually relax safety constraints online.
result Ensures recursive feasibility of MPC schemes by relaxing safety constraints over time.
Improved MPC with neural networks and active sets for large-scale problems.
problem Large-scale model predictive control with guarantees on feasibility and stability.
method Combines offline-trained neural network and online primal active set solver.
result Achieves 2x reduction in online inference time for large problems.
We present foundations for using Model Predictive Control (MPC) as a differentiable policy class for reinforcement learning in continuous state and action spaces. This provides one way of leveraging and combining the advantages of model-free and model-based approaches. Specifically, we differentiate through MPC by usin…
Bayesian optimisation tackles stochastic MPC hyper-parameter tuning.
problem Fine-tuning hyper-parameters in stochastic MPC models.
method Heteroscedastic Bayesian optimisation framework.
result Framework effectively tunes hyper-parameters in control problems.
MPC outperforms reactive budgeting in non-stationary return environments.
problem Optimizing budget allocation under non-stationary returns.
method Receding-horizon Model Predictive Control (MPC) compared to reactive policies.
result MPC consistently outperforms reactive budgeting when return dynamics are predictable.
Review of integrating Bayesian methods with neural network-based MPC.
problem Lack of standardized benchmarks and reliable analyses in Bayesian MPC.
method Systematic analysis of Bayesian methods in neural-network-based MPC.
result Need for standardized benchmarks, ablation studies, and transparent reporting.
This paper develops a framework for training and evaluating neural networks for MPC.
problem Lack of a general framework for characterizing learning approaches in MPC.
method Developed a framework using PyTorch and CVXPY, incorporating hit-and-run sampling for efficient training data generation.
result Proposed metrics for validating neural network-based MPC approaches.
Efficient learning-based MPC for unknown nonlinear systems with state constraints.
problem Control of discrete-time nonlinear systems with unknown dynamics and state constraints.
method Receding horizon reinforcement learning (r-LPC) using Koopman operator-based prediction model.
result Proven closed-loop recursive feasibility, robustness, and asymptotic stability under function approximation errors.
Iterative method learns unknown constraints for MPC control.
problem Learning to satisfy unknown polyhedral state constraints in iterative MPC.
method Collects and improves estimates of unknown constraints using collected data, designs an MPC controller to satisfy the estimated constraints.
result Robust and probabilistic guarantees of constraint satisfaction as a function of task iterations.
The paper explores how ReLU DNNs can represent MPC policies and vice versa.
problem Representing MPC policies as ReLU DNNs and vice versa.
method Developed an approximate method for identifying input-space in ReLU nets resulting in PWA functions over polyhedral regions. Studied inverse multiparametric linear or quadratic programs for reconstruction of constraints and cost functions given a PWA function.
result Identification and representation of MPC policies as ReLU DNNs and vice versa.
Combines MFRL and MPC for data-efficient high performance in robotics.
problem Lack of data efficiency in MBRL for real-world applications.
method Combines Model-Free Reinforcement Learning (MFRL) and Model Predictive Control (MPC).
result Achieves MFRL's performance level with MBRL's data efficiency.
We prove that the kernels of the restrictions of symplectic Dirac or symplectic Dirac-Dolbeault operators on natural subspaces of polynomial valued spinor fields are finite dimensional on a compact symplectic manifold. We compute those kernels for the complex projective spaces. We construct injections of subgroups of t…
Model predictive control (MPC) has become one of the well-established modern control methods for three-phase inverters with an output LC filter, where a high-quality voltage with low total harmonic distortion (THD) is needed. Although it is an intuitive controller, easy to understand and implement, it has the significa…
A novel Q-learning algorithm connects information theory to MPC, improving control tasks.
problem Combining model-free RL and MPC for real-world systems with expensive queries and uncertain dynamics.
method Developed a Q-learning algorithm that uses entropy regularization and leverages biased models.
result Improves control performance on sim-to-sim tasks compared to optimal control and RL from scratch.
Deep neural networks improve chemical reactor control using MPC.
problem Improving control of chemical reactors with neural networks.
method Training neural networks on model predictive control (MPC) for reactor control.
result Neural network can mimic MPC control inputs while maintaining constraints.
Trial-and-error based reinforcement learning (RL) has seen rapid advancements in recent times, especially with the advent of deep neural networks. However, the majority of autonomous RL algorithms require a large number of interactions with the environment. A large number of interactions may be impractical in many real…
MPC framework reduces execution costs and schedule deviations in trading.
problem Executing large orders in markets under time and liquidity constraints.
method Model Predictive Control (MPC) framework balancing order completion, market impact, and opportunity cost.
result Significant reductions in slippage and schedule shortfall compared to benchmarks.
DMPC combines MPC and value function estimation for efficient control tasks.
problem Efficiently solve control tasks with sparse and binary reward signals.
method Actor-critic algorithm combining MPC and value function estimation.
result DMPC actor minimizes an upper bound of cross-entropy to optimal policy.
Robo-advisors use MPC to create dynamic investment strategies.
problem Static allocation methods limit robo-advisors' effectiveness.
method Combines MPC with Hidden Markov Model and Black-Litterman for dynamic asset allocation.
result MPC-based strategies outperform static approaches in dynamic and risk-budgeting criteria.
Given a symplectic manifold (M,ω) admitting a metaplectic structure, and choosing a positive ω-compatible almost complex structure J and a linear connection ∇ preserving ω and J, Katharina and Lutz Habermann have constructed two Dirac operators D and ${\wt{D}}$ acting on sections of a bundle of sympl…
Simulation-to-real transfer is an important strategy for making reinforcement learning practical with real robots. Successful sim-to-real transfer systems have difficulty producing policies which generalize across tasks, despite training for thousands of hours equivalent real robot time. To address this shortcoming, we…
The paper proposes a control strategy for systems with sparse parameters using compressed sensing.
problem Control of linear systems with unknown sparse parameters under disturbances.
method Sparse estimation using Recursive Least Squares, improved with Basis Pursuit Denoising, and reformulated probabilistic constraints.
result The proposed algorithm outperforms existing methods in control design for systems with sparse impulse response parameters.
Novel framework learns policies for MPC with near-optimal guarantees.
problem Safe and near-optimal policy learning for MPC.
method Primal-dual neural networks for supervised learning of policies and certificate policies.
result Guaranteed feasibility and optimality of learned policies.
Paper tackles SMPC for linear systems with unknown noise distribution.
problem Stochastic MPC for linear systems with chance state constraints and unknown noise distribution.
method Reformulate chance constraints, design robust benchmark SMPC, and develop adaptive SMPC with online noise statistics learning.
result Adaptive SMPC guarantees time-uniform satisfaction of unknown reformulated state constraints with high probability.
We describe an optimal adversarial attack formulation against autoregressive time series forecast using Linear Quadratic Regulator (LQR). In this threat model, the environment evolves according to a dynamical system; an autoregressive model observes the current environment state and predicts its future values; an attac…
Optimizes asset allocation with illiquid assets using MPC.
problem Strategic asset allocation with illiquid alternative asset classes.
method Formulates illiquid dynamics as a random linear system and proposes a convex optimization based MPC policy.
result Performance close to a fully liquid scenario, despite time delay and uncertainty.
Efficient privacy-preserving machine learning framework using random transformations.
problem Slow training and inference speed in privacy-preserving machine learning systems.
method Random transformations like linear and permutation, combined with arithmetic sharing.
result High efficiency and low computation cost in private machine learning.
The paper tackles ride-hailing fleet repositioning with a calibrated demand approach.
problem Repositioning idle supply before future demand is observed in ride-hailing.
method A predict-then-optimize approach using calibrated demand regimes, a similarity gate, and spatial queue-regret decomposition.
result The spatial gate reduces mean wait time to 82.3s compared to 85.3s for a hand-tuned similarity gate and 85.8s for a distributional-only baseline.
Enhanced metrics for multiclass classification improve on existing methods.
problem Lack of decisive poor classification results in existing multiclass metrics.
method Introduces three new metrics derived from multivariate Pearson correlation coefficients.
result New metrics decisively indicate poor classification results.
Privacy concern has been increasingly important in many machine learning (ML) problems. We study empirical risk minimization (ERM) problems under secure multi-party computation (MPC) frameworks. Main technical tools for MPC have been developed based on cryptography. One of limitations in current cryptographically priva…
PhysVarMix predicts diverse urban trajectories with physics constraints.
problem Predicting complex urban agent trajectories with multiple plausible scenarios.
method Physics-informed variational mixture model combining learning and physics constraints.
result Superior performance compared to existing methods on benchmark datasets.
Optimizes trading policies using future price forecasts.
problem Static reinforcement learning agents lack mechanisms for using price forecasts at inference time.
method FPILOT framework inspired by Model Predictive Control (MPC). Uses a predictive model to construct an allocation-based imagined return objective at each decision step.
result Consistent improvements in total return and risk-adjusted metrics across various policy learning algorithms.
Paper proposes Walsh-Hadamard Variational Inference for efficient approximate inference in large models.
problem Over-regularization in variational inference for large models.
method Walsh-Hadamard factorization strategies to reduce parameterization, accelerate computations, and increase posterior expressiveness.
result Efficient approximate inference achieved in over-parameterized models.
Variational inference is a scalable technique for approximate Bayesian inference. Deriving variational inference algorithms requires tedious model-specific calculations; this makes it difficult to automate. We propose an automatic variational inference algorithm, automatic differentiation variational inference (ADVI). …
Variational Inference shows promise for Bayesian GARCH model estimation.
problem Bayesian estimation of GARCH-family models using Monte Carlo sampling.
method Variational Inference as an alternative to Monte Carlo sampling.
result Variational Inference is a reliable and competitive method for Bayesian learning in GARCH-like models.
A tutorial on variational inference for high-dimensional models.
problem Approximating marginal likelihood and posterior in Bayesian models.
method Parametric approach to variational inference.
result Variational inference is now preferred for high-dimensional models and large datasets.
Variational inference is an umbrella term for algorithms which cast Bayesian inference as optimization. Classically, variational inference uses the Kullback-Leibler divergence to define the optimization. Though this divergence has been widely used, the resultant posterior approximation can suffer from undesirable stati…
TVO tightens variational inference bounds for deep models.
problem Improving variational inference bounds for deep models.
method Introduces thermodynamic variational objective (TVO) connecting variational inference and thermodynamic integration.
result TVO provides tighter lower bound to log marginal likelihood than ELBO.
Semi-Implicit Variational Inference (SIVI) is improved with SIVI-SM using score matching.
problem Intractable densities in variational distributions hinder SIVI training.
method SIVI-SM uses score matching to handle intractable densities in a minimax formulation.
result SIVI-SM outperforms ELBO-based SIVI methods in Bayesian inference tasks.
A new particle algorithm improves mean-field variational inference.
problem Efficiently approximating nonparametric posterior distributions in machine learning.
method Introduces PArticle VI (PAVI), a novel particle-based algorithm for nonparametric mean-field approximation.
result Obtains non-asymptotic error bounds for PArticle VI, providing the first end-to-end guarantee for particle-based MFVI.
We introduce incremental variational inference and apply it to latent Dirichlet allocation (LDA). Incremental variational inference is inspired by incremental EM and provides an alternative to stochastic variational inference. Incremental LDA can process massive document collections, does not require to set a learning …