DMPC combines MPC and value function estimation for efficient control tasks.
problem Efficiently solve control tasks with sparse and binary reward signals.
method Actor-critic algorithm combining MPC and value function estimation.
result DMPC actor minimizes an upper bound of cross-entropy to optimal policy.
Deep neural networks improve chemical reactor control using MPC.
problem Improving control of chemical reactors with neural networks.
method Training neural networks on model predictive control (MPC) for reactor control.
result Neural network can mimic MPC control inputs while maintaining constraints.
Robot-assisted dressing offers an opportunity to benefit the lives of many people with disabilities, such as some older adults. However, robots currently lack common sense about the physical implications of their actions on people. The physical implications of dressing are complicated by non-rigid garments, which can r…
Proposes a method to quantify the reliability of salient regions in deep learning models using p-values.
problem Difficulty in assessing the reliability of saliency maps generated by deep learning models.
method Proposes a selective inference framework to quantify the reliability of salient regions as selected hypotheses by deep learning models.
result The method can provably control the probability of false positive detections of salient regions.
We propose the use of Bayesian networks, which provide both a mean value and an uncertainty estimate as output, to enhance the safety of learned control policies under circumstances in which a test-time input differs significantly from the training set. Our algorithm combines reinforcement learning and end-to-end imita…
Study optimizes GCS operations with deep learning and reinforcement learning.
problem Maximizing storage performance in GCS with resource-efficient simulations.
method Introduces MLD model for fast flow prediction and well control optimization, combining deep learning and reinforcement learning.
result Achieves highest NPV while reducing computational resources by over 60%.
Deep RBVFs improve continuous control in RL.
problem Challenges in finding optimal actions for continuous actions in RL.
method Introduced deep radial-basis value functions (RBVFs) for continuous control.
result RBF-DQN significantly outperforms value-function-only baselines and is competitive with actor-critic algorithms.
We adapt Shapley values to explain model uncertainty, connecting it to information theory.
problem Explaining uncertainty in model predictions.
method Adapted Shapley value framework to quantify feature contributions to predictive uncertainty.
result Deep connections between Shapley values and information theory quantities.
Value functions are crucial for model-free Reinforcement Learning (RL) to obtain a policy implicitly or guide the policy updates. Value estimation heavily depends on the stochasticity of environmental dynamics and the quality of reward signals. In this paper, we propose a two-step understanding of value estimation from…
SCoRE provides risk control for selective prediction models.
problem Enforcing strict error control in selective prediction models.
method SCoRE framework based on conformal inference and hypothesis testing.
result SCoRE offers binary trust decisions with finite-sample error control.
FavMac maximizes value while controlling cost in multi-label prediction.
problem Value-maximizing predictions with strict cost control in multi-label scenarios.
method FavMac pipeline combining any multi-label classifier with online update mechanism.
result FavMac achieves higher value with strict cost control compared to baselines.
Effective plant growth and yield prediction is an essential task for greenhouse growers and for agriculture in general. Developing models which can effectively model growth and yield can help growers improve the environmental control for better production, match supply and market demand and lower costs. Recent developm…
Deep Galerkin Method estimates value function for mean-field control problem.
problem Optimal control of agents with average welfare as the objective.
method Apply DGM to estimate value function and distribution evolution.
result Neural network approximations converge to analytical solution.
Deep learning architectures have proved versatile in a number of drug discovery applications, including the modelling of in vitro compound activity. While controlling for prediction confidence is essential to increase the trust, interpretability and usefulness of virtual screening models in drug discovery, techniques t…
A novel model combines deep learning and extreme value theory for multivariate cyber risk prediction.
problem High dimensionality and heavy tails in multivariate cyber risk patterns.
method Combines deep learning for point predictions and extreme value theory for quantile predictions.
result The model provides satisfactory high quantile predictions and accurate point predictions.
ControlSHAP stabilizes Shapley value approximations using control variates.
problem High computational cost of exact Shapley values in blackbox models.
method ControlSHAP uses Monte Carlo control variates to stabilize Shapley value approximations.
result Significant reduction in Monte Carlo variability of Shapley estimates.
DSPPs improve predictive distributions in scalable regression tasks.
problem Improving predictive distributions in scalable regression tasks.
method Inspired by DGPs, DSPPs use mini-batch training and kernel basis functions for uncertainty control.
result DSPPs provide significantly better calibrated predictive distributions than other methods.
RKHS-SHAP uses Shapley values for kernel methods to provide feature attributions.
problem Feature attribution for kernel methods is often heuristic and not individualised.
method RKHS-SHAP uses Shapley values from coalition game theory to compute feature attributions for kernel machines efficiently.
result RKHS-SHAP can compute both Interventional and Observational Shapley values.
Deep learning solves complex stochastic control with jumps.
problem Solving high-dimensional stochastic control tasks with jumps.
method Model-based approach using two neural networks, iteratively trained with objectives derived from the Hamilton-Jacobi-Bellman equation.
result Demonstrates effectiveness in solving complex high-dimensional stochastic control tasks.
PIVEN predicts both specific values and prediction intervals.
problem Improving robustness of neural nets in regression tasks.
method PIVEN is a deep neural network that produces both a prediction interval and a specific value prediction.
result PIVEN produces tighter uncertainty bounds than state-of-the-art approaches for prediction intervals.
DVA framework attributes value of predictive models to features, configurations, and interactions.
problem Lack of explanation for how predictive models influence operational decisions.
method Shapley-based cooperative game theory applied to predict-then-optimize systems.
result DVA can guide targeted interventions to align model beliefs with operational performance.
Self-consistent models improve reinforcement learning by aligning predictions with future values.
problem Improving reinforcement learning by aligning model predictions with future values.
method Proposes multiple self-consistency updates to encourage a learned model and value function to be consistent with each other.
result Self-consistency helps both policy evaluation and control in both tabular and function approximation settings.
Researchers develop a method to control nonlinear systems with Koopman operator regression.
problem Controlling nonlinear systems with finite action spaces.
method Koopman operator regression for dynamics estimation and model predictive control for control.
result The method yields a linear switching predictive model for control.
The control of complex systems is of critical importance in many branches of science, engineering, and industry. Controlling an unsteady fluid flow is particularly important, as flow control is a key enabler for technologies in energy (e.g., wind, tidal, and combustion), transportation (e.g., planes, trains, and automo…
Method predicts hardware resource usage by control software with guaranteed linear convergence.
problem Predicting time-varying hardware resource availability in control software.
method Path structured multimarginal Schrödinger bridge (MSBP) for learning stochastic resource usage.
result Guaranteed linear convergence to accurate prediction of hardware resource utilization.
Optimizes investment model using LSTM for better risk control.
problem Enhancing risk control in multi-factor investment models.
method Combines LSTM with multi-factor investment model for factor selection and weight determination.
result LSTM model outperforms benchmark in risk control metrics.
Deep reinforcement learning controls drones without model knowledge.
problem Real-time robot control without engineered models.
method Learnt probabilistic model of drone dynamics, model-based reinforcement learning.
result Controller and value function optimized through generated latent trajectories.
MEMEC improves sample efficiency in reinforcement learning.
problem Lack of sample efficiency in reinforcement learning.
method Proposes MEMEC, a Boltzmann policy with state-dependent temperature for more principled exploration.
result MEMEC outperforms other methods on classic RL environments and Atari games.
Deep neural nets approximate high-dimensional HJB equations efficiently.
problem Approximating solutions to high-dimensional HJB equations.
method Deep neural networks for approximating solutions.
result Deep neural networks can approximate solutions without the curse of dimensionality.
Two new Koopman models improve nonlinear system prediction.
problem Predicting nonlinear, nonconvex dynamic systems.
method Convex and Extended Koopman Models using deep learning.
result Significantly improved predictive performance.
Deep learning predicts S&P 500 index direction.
problem Accurate stock price prediction remains challenging.
method Convolutional neural network model for S&P 500 index forecasting.
result Model achieves over 55% accuracy in predicting index direction.
Reinforcement learning algorithms such as the deep deterministic policy gradient algorithm (DDPG) has been widely used in continuous control tasks. However, the model-free DDPG algorithm suffers from high sample complexity. In this paper we consider the deterministic value gradients to improve the sample efficiency of …
Variational inference transforms posterior inference into parametric optimization thereby enabling the use of latent variable models where otherwise impractical. However, variational inference can be finicky when different variational parameters control variables that are strongly correlated under the model. Traditiona…
It has long been assumed that high dimensional continuous control problems cannot be solved effectively by discretizing individual dimensions of the action space due to the exponentially large number of bins over which policies would have to be learned. In this paper, we draw inspiration from the recent success of sequ…
In this paper, we propose a design of a model-free networked controller for a nonlinear plant whose mathematical model is unknown. In a networked control system, the controller and plant are located away from each other and exchange data over a network, which causes network delays that may fluctuate randomly due to net…
A method selects candidates based on predictions with statistical control.
problem Screening candidates for resource-intensive steps like hiring or drug discovery.
method Wraps around any prediction model to produce a subset of candidates with controlled false selection rate.
result Empirically demonstrates selection of candidates whose predictions exceed a data-dependent threshold.
Learning optimal feedback control laws capable of executing optimal trajectories is essential for many robotic applications. Such policies can be learned using reinforcement learning or planned using optimal control. While reinforcement learning is sample inefficient, optimal control only plans an optimal trajectory fr…
Paper introduces solving financial problems using time-stepped FBSDE and deep learning.
problem Quantitative finance problems under specific dynamics and instruments.
method Formulate as FBSDE, turn into control problems, time-step, solve with optimization and deep learning.
result Solves financial problems with new methods and deep learning.
Quantile deep learning improves time series prediction accuracy and uncertainty quantification.
problem Uncertainty in multi-step time series prediction.
method Developed a novel quantile regression deep learning framework for multi-step time series prediction.
result Integrating quantile loss function with deep learning provides additional predictions for selected quantiles without loss in accuracy.
Deep learning method proves convergence for high-dimensional PDEs.
problem Solving high-dimensional nonlinear PDEs for mean field control problems.
method Deep Galerkin method (DGM) for Hamilton-Jacobi-Bellman (HJB) equations.
result DGM converges to the true value function of mean field control problems.
Paper proposes exploiting Q function structures for better planning and RL.
problem Value-based methods in planning and RL.
method Exploiting low-rank structure of Q function using Matrix Estimation techniques.
result Improved planning and RL performance on 'low-rank' tasks.
We demonstrate that there is significant redundancy in the parameterization of several deep learning models. Given only a few weight values for each feature it is possible to accurately predict the remaining values. Moreover, we show that not only can the parameter values be predicted, but many of them need not be lear…
Dissipative SymODEN learns dynamics with dissipation and control from data.
problem Learning dynamics with dissipation and control from observed data.
method Dissipative SymODEN encodes port-Hamiltonian dynamics into a deep learning architecture.
result The learned model reveals key aspects of the system, such as inertia, dissipation, and potential energy.
Paper uses deep reinforcement learning for better control of rocket engines during start-up phases.
problem Lack of optimal control during transient phases of liquid rocket engines.
method Deep reinforcement learning approach for optimal control of a gas-generator engine's continuous start-up phase.
result Deep reinforcement learning controller achieves highest performance and minimal computational effort.
Data-efficient learning in continuous state-action spaces using very high-dimensional observations remains a key challenge in developing fully autonomous systems. In this paper, we consider one instance of this challenge, the pixels to torques problem, where an agent must learn a closed-loop control policy from pixel i…
New deep learning method solves stochastic control problems.
problem Solving strongly coupled FBSDEs for stochastic control.
method Modified deep BSDE method with new loss function.
result Empirical convergence of the new method for three problems.
Develops a framework to control risk in online learning models.
problem Rigorous uncertainty quantification for online learning models.
method A framework for constructing uncertainty sets that provably control risk.
result Guarantees risk control at any user-specified level even with distribution shifts.
Extends conformal prediction for controlling expected risk of monotone loss functions.
problem Controlling expected risk of monotone loss functions.
method Generalizes split conformal prediction with coverage guarantee, extending to distribution shift, quantile risk, multiple, adversarial, and expectations of U-statistics.
result Tight up to an O(1/n) factor, with worked examples in computer vision and natural language processing.