Paper proposes a new method to optimize robot body structure and control policy.
problem Optimizing robot body structure and control policy in a coupled manner.
method Revisits co-design problem as a Stackelberg game, incorporating control adaptation dynamics.
result Stackelberg PPO outperforms standard PPO in stability and performance.
Meta-reinforcement learning improves fault-adaptive control efficiency.
problem Adaptive control under abrupt system faults with strict time constraints.
method Model-agnostic meta learning (MAML) with a fault library of prior policies.
result Improved sample efficiency and quick adaptation to new faults.
TSAC achieves optimal frequentist regret in adaptive control of LQRs.
problem Adaptive control of stabilizable linear-quadratic regulators with unknown dynamics.
method Thompson Sampling (TS) for adaptive control of LQRs, with a novel early exploration strategy.
result Achieves i l d e O ( T ) ilde O(\sqrt{T}) i l d e O ( T ) regret, optimal for multidimensional systems. New algorithm reduces control error in systems with changing dynamics.
problem Online control of systems with time-varying linear dynamics.
method Introduces adaptive regret metric and a novel meta-algorithm.
result First adaptive regret bound for online convex optimization with memory.
Survey combines FL and control for better adaptability and privacy.
problem Combining FL and control for better adaptability and privacy.
method Combining Federated Learning (FL) and control methods.
result Combining FL and control enhances adaptability, scalability, generalization, and privacy.
In this paper we propose a new methodology for solving an uncertain stochastic Markovian control problem in discrete time. We call the proposed methodology the adaptive robust control. We demonstrate that the uncertain control problem under consideration can be solved in terms of associated adaptive robust Bellman equa…
Adaptive traffic control uses deep RL to improve decision-making.
problem Improving traffic control using deep RL.
method Integrates recent deep RL techniques into a novel DQN-based algorithm (TC-DQN+) for traffic control.
result Proposes a new reward function for traffic control.
This paper tackles adaptive control of unknown Markov jump systems with sample complexity and regret bounds.
problem Adaptive control of unknown Markov jump systems with changing dynamics.
method Identification-based adaptive control using a system identification algorithm and certainty equivalent control.
result The proposed adaptive control scheme achieves O ( T ) \mathcal{O}(\sqrt{T}) O ( T ) regret, improving to O ( p o l y l o g ( T ) ) \mathcal{O}(polylog(T)) O ( p o l y l o g ( T )) with partial knowledge. AdaptOn achieves logarithmic regret in adaptive control of unknown partially observable linear systems.
problem Adaptive control in partially observable linear dynamical systems.
method AdaptOn algorithm that estimates system dynamics through online learning and gradient descent.
result AdaptOn achieves a logarithmic regret bound of polylog(T) after T steps.
Paper proposes adaptive control for unknown systems using reinforcement learning.
problem Adaptive control for unknown, linearizable systems.
method On-policy reinforcement learning for discrete-time, stochastic systems.
result Stability and tracking errors concentrate near zero with high probability.
Agents acting in the natural world aim at selecting appropriate actions based on noisy and partial sensory observations. Many behaviors leading to decision mak- ing and action selection in a closed loop setting are naturally phrased within a control theoretic framework. Within the framework of optimal Control Theory, o…
New robust control method for uncertain systems using bootstrapped noise.
problem Designing controllers robust to model uncertainties in finite data.
method Least-squares model estimator, bootstrap resampling, multiplicative noise LQR.
result Significantly outperforms certainty equivalent controllers in numerical tests.
In this paper a neural network heuristic dynamic programing (HDP) is used for optimal control of the virtual inertia based control of grid connected three phase inverters. It is shown that the conventional virtual inertia controllers are not suited for non inductive grids. A neural network based controller is proposed …
Nonparametric adaptive robust control tackles model uncertainty in stochastic processes.
problem Model uncertainty in stochastic processes.
method Adaptive robust control methodology using online learning and uncertainty reduction, empirical distribution, and Lagrangian duality.
result Nonparametric adaptive robust control approach is preferable to traditional robust frameworks.
LqgOpt learns optimal control in unknown LQG systems with minimal regret.
problem Adaptive control in partially observable linear quadratic Gaussian systems with unknown dynamics.
method Optimism in the face of uncertainty, predictor state evolution, closed-loop system identification, confidence bounds.
result Proves a regret upper bound of i l d e O ( T ) ilde{\mathcal{O}}(\sqrt{T}) i l d e O ( T ) for LQG systems. Combines control variates and adaptive importance sampling for Monte Carlo integration.
problem Improving Monte Carlo integration accuracy with control variates and adaptive sampling.
method A quadrature rule combining control variates and adaptive importance sampling.
result Non-asymptotic bound on the probabilistic error of the procedure.
New method handles robust and adaptive control of linear systems with non-convex costs.
problem Robust and adaptive control of linear systems with unknown parameters.
method Combining non-asymptotic linear regression, interval prediction, and tree-based planning.
result First end-to-end suboptimality analysis for robust and adaptive MPC with non-convex costs.
Designs adaptive controller for networked control systems with wireless data transmission.
problem Adaptive control in networked systems with unreliable wireless channels.
method Upper Confidence Bounds for Networked Control Systems (UCB-NCS) learning rule.
result Non-asymptotic performance guarantees with a regret bound of O(C√T).
FinFlowRL learns from experts to optimize financial control in changing markets.
problem Traditional finance control methods fail in real-world, non-stationary markets.
method Imitation-Reinforcement Learning framework that pretrains on expert strategies and finetunes in noise space.
result Consistently outperforms individually optimized experts across diverse market conditions.
We consider adaptive control of the Linear Quadratic Regulator (LQR), where an unknown linear system is controlled subject to quadratic costs. Leveraging recent developments in the estimation of linear systems and in robust controller synthesis, we present the first provably polynomial time algorithm that provides high…
New bounds for adaptive control in high dimensions without fixed state space.
problem Adaptive control of linear systems in high or infinite dimensions.
method Novel perturbation bound for certainty equivalence, scaling with prediction error.
result First regret bounds for LQR in infinite dimensional systems, independent of ambient dimension.
Latent FxLMS accelerates ANC by adapting along low-dimensional filter weights.
problem Improving active noise control with neural adaptive filters.
method Training an auto-encoder on filter coefficients, constraining weights to latent variables, and updating in latent space.
result Latent FxLMS converges in fewer steps with comparable error to standard FxLMS.
Novel Bayesian meta-reinforcement learning framework improves traffic signal control robustness.
problem Lack of robustness and stability in adaptation for traffic signal control.
method Value-based Bayesian meta-reinforcement learning framework BM-DQN with fast-adaptation variation and DQN fast-update advantage.
result Framework adapts more quickly and robustly to new scenarios than previous methods.
This paper improves MARL for networked systems through new protocols and discount factors.
problem Improving control in networked systems using multi-agent reinforcement learning.
method Formulated as a spatiotemporal Markov decision process, introduced a spatial discount factor, and proposed NeurComm.
result Appropriate spatial discount factor enhances learning curves of non-communicative MARL algorithms.
CAD-DA controls anomaly detection under domain adaptation.
problem Valid statistical inference after domain adaptation.
method Conditional Selective Inference to handle domain adaptation effects.
result Valid statistical inference under domain adaptation achieved.
New model-based methods adapt pre-trained policies to unseen environments efficiently.
problem High sample complexity in reinforcement learning limits practical applications.
method Combines online learning and adaptive control to adapt policies in unseen environments.
result Proves policies can quickly recover trajectories from source to target environments.
NCT simplifies one-step generator adaptation to new controls.
problem Adapting one-step generators to new control conditions.
method Noise Consistency Training (NCT) integrates new controls without retraining.
result NCT achieves state-of-the-art controllable generation in a single pass.
We investigate the adaptive robust control framework for portfolio optimization and loss-based hedging under drift and volatility uncertainty. Adaptive robust problems offer many advantages but require handling a double optimization problem (infimum over market measures, supremum over the control) at each instance. Mor…
The study sets limits on how well systems can be controlled adaptively.
problem Learning to control unknown linear Gaussian systems with quadratic costs.
method Combining ideas from experiment design, estimation theory, and perturbation bounds of information matrices.
result Regret lower bounds of the order of T \sqrt{T} T in the time horizon T T T accurately capture control-theoretic parameters. FinFlowRL combines imitation and reinforcement learning for better financial control.
problem Traditional stochastic control methods fail in real-world finance due to changing market conditions.
method FinFlowRL uses imitation learning to pretrain an adaptive meta policy, then finetunes it with reinforcement learning.
result FinFlowRL consistently outperforms individual strategies across various market conditions.
Paper proposes a privacy-preserving method to control false discoveries.
problem Protecting individual information in hypothesis tests while controlling false discoveries.
method Differentially private adaptive FDR control method with privacy guarantee.
result The method controls the FDR metric exactly at a user-specified level with privacy.
Adaptive optimal control using value iteration initiated from a stabilizing control policy is theoretically analyzed in terms of stability of the system during the learning stage without ignoring the effects of approximation errors. This analysis includes the system operated using any single/constant resulting control …
Convolutional neural networks (CNNs) are commonly used for image classification tasks, raising the challenge of their application on data flows. During their training, adaptation is often performed by tuning the learning rate. Usual learning rate strategies are time-based i.e. monotonously decreasing. In this paper, we…
Adapts model-based advice to stabilize black-box policies for nonlinear control.
problem Stabilizing machine-learned policies for nonlinear control with limited model information.
method Proposes an adaptive λ λ λ -confident policy to combine black-box and model-based advice. result Proves the stability of the adaptive λ λ λ -confident policy and its competitive ratio. Syntax designs adaptive trials for subpopulations with potential benefits.
problem Identifying subpopulations with positive treatment effects in diverse patient populations.
method Adaptive patient recruitment and synthetic control estimation.
result Syntax outperforms conventional trial designs in identifying beneficial subpopulations.
IG-RL learns adaptive traffic signals for any network, outperforming existing methods.
problem Adaptive traffic signal control for large networks with combinatorial state and action spaces.
method Graph-Convolutional Networks for decentralized, flexible control.
result IG-RL generalizes to new networks and traffic conditions without additional training.
The study optimizes wind farm yaw control using Gaussian process regression and high-fidelity simulations.
problem Improving yaw control inputs for maximum power production in wind farms.
method Gaussian process regression and modifier adaptation scheme based on high-fidelity simulation data.
result Both modifier adaptation and Bayesian optimization improve power production with smaller yaw misalignments.
Study time-inconsistent control problems with model uncertainty, solving portfolio selection.
problem Time-inconsistent Markovian control problems under model uncertainty.
method Combining sub-game perfect strategies with adaptive robust stochastic methods.
result Solved numerically the mean-variance portfolio selection problem.
Power system emergency control is generally regarded as the last safety net for grid security and resiliency. Existing emergency control schemes are usually designed off-line based on either the conceived "worst" case scenario or a few typical operation scenarios. These schemes are facing significant adaptiveness and r…
Paper tackles non-Markovian control problems with new learning methods.
problem Non-Markovian stochastic control problems with unknown parameters.
method Off-model training and importance sampling for deep neural network approximation.
result Quantitative error bounds for adaptive learning under model uncertainty.
New method controls false discoveries in real-time data streams.
problem Online testing of hypotheses with strict error constraints and no future data.
method Structure-adaptive sequential testing (SAST) with alpha-investment algorithm.
result Substantial power gain over existing online testing rules.
Optimal control strategy uses random noise to adaptively control systems with unknown parameters.
problem Online adaptive control of linear quadratic regulator with unknown system parameters.
method Certainty equivalent control with exploratory random noise, refined estimates of system matrices.
result Achieves optimal regret scaling as Θ(√(d_u^2 d_x T)) with self-bounding ODE method.
In a voice-controlled smart-home, a controller must respond not only to user's requests but also according to the interaction context. This paper describes Arcades, a system which uses deep reinforcement learning to extract context from a graphical representation of home automation system and to update continuously its…
Optimal control in changing systems without strong convexity assumptions.
problem Adversarial changes in convex costs for unknown linear systems.
method Non-convex lower confidence bounds and computationally-efficient regret minimization.
result Achieves T \smash{\sqrt{T}} T -regret rate, optimal compared to best stabilizing controller. This paper improves Adam's performance in machine learning tasks.
problem Improving the generalization ability of adaptive gradient methods.
method Develops a control theoretic framework to propose AdamSSM, a new variant of Adam.
result AdamSSM improves generalization accuracy and convergence compared to recent adaptive gradient methods.
Truncated backpropagation through time (TBPTT) is a popular method for learning in recurrent neural networks (RNNs) that saves computation and memory at the cost of bias by truncating backpropagation after a fixed number of lags. In practice, choosing the optimal truncation length is difficult: TBPTT will not converge …
Paper proposes a deep RL approach for traffic signal control balancing efficiency and equity.
problem Inefficient and inflexible traffic signal controllers.
method Deep reinforcement learning with a novel reward function combining efficiency and equity.
result The proposed algorithm achieves state-of-the-art performance on various traffic scenarios.
A new framework for adaptive behavior using reusable value profiles.
problem Adaptive behavior in changing environments requires switching among value-control regimes, but maintaining separate parameters for each situation is impractical.
method Introduces value profiles: reusable bundles of parameters assigned to hidden states, allowing for state-conditional strategy recruitment without independent parameters for each context.
result Profile-based models outperform simpler alternatives in probabilistic reversal learning, suggesting belief-dependent control of adaptive behavior.