KQT-EWMA monitors multivariate data streams online with flexible and practical change detection.
problem Online monitoring of multivariate data streams for detecting changes.
method Combines Kernel-QuantTree histogram and EWMA statistic for non-parametric monitoring.
result Controls Average Run Length (ARL0) while achieving comparable detection delays.
New method detects changes without partitioning distributions.
problem Detects changes in unknown probability distributions without partitioning.
method Aggregates e-processes over possible changepoints and infimum over candidate no-change distributions.
result Achieves first-order asymptotically optimal detection delay.
Active Reinforcement Learning (ARL) is a twist on RL where the agent observes reward information only if it pays a cost. This subtle change makes exploration substantially more challenging. Powerful principles in RL like optimism, Thompson sampling, and random exploration do not help with ARL. We relate ARL in tabular …
ARL uses queries to learn rewards, focusing on cost vs. reward value.
problem How to efficiently use queries to learn rewards in reinforcement learning.
method Proposed and evaluated heuristic approaches for ARL in multi-armed bandits and MDPs.
result Challenging aspects of ARL highlighted, including intractability of value computation.
ARL makes market makers resilient to adversarial conditions.
problem Creating resilient market makers against adversarial attacks.
method Transformed single-agent RL into a zero-sum game between market maker and adversary.
result ARL leads to risk-averse behavior and significant performance improvements.
ARL and Hawkes processes improve market-making strategies with variable volatility.
problem Enhancing market-making strategies to adapt to varying volatility levels and self-exciting behaviors.
method Integrates ARL, Hawkes processes, and variable volatility levels; shifts from Poisson to Hawkes process.
result 4-action MM trained in low-volatility environment adapts to high-volatility conditions, providing stable performance.
Non-parametric estimators improve quickest changepoint detection under irregular sequence lengths.
problem Limited and irregular sequence lengths hinder application of ARL and ADD in QCD.
method Analogies with survival analysis to model detection probabilities under truncation.
result KM-ARL and KM-ADD non-parametric estimators are asymptotically unbiased.
ARL algorithm reduces adversarial MDP to bandit problems for reliable policy learning.
problem Learning reliable policies in non-stationary, adversarial MDPs.
method Adversarial Reinforcement Learning (ARL) algorithm that converts MDP to a sequence of adversarial bandit problems.
result Achieves optimal regret bound of O ( S A T H 3 ) O(\sqrt{SATH^3}) O ( S A T H 3 ) . ARL-GEN adapts to the smallest model class in nested families for RL with improved regret.
problem Model selection for Reinforcement Learning with nested model families.
method Adaptive Reinforcement Learning (ARL-GEN) with value targeted regression and model selection module.
result ARL-GEN achieves a matching regret to an oracle with knowledge of the true model class.
ARL improves fairness without protected features, showing AUC improvements for worst-case groups.
problem Training fairness in ML without known protected features.
method Adversarially Reweighted Learning (ARL) using non-protected features and task labels.
result ARL improves Rawlsian Max-Min fairness with notable AUC improvements for worst-case groups.
Optimizes quickest change detection with bounded means under ARL constraint.
problem Quickest detection of changepoints with bounded means under ARL constraint.
method Derives universal lower and upper bounds for detection delay.
result Achieves universal lower bound in the bounded mean detection setting.
We study the problem of detecting an abrupt change to the signal covariance matrix. In particular, the covariance changes from a "white" identity matrix to an unknown spiked or low-rank matrix. Two sequential change-point detection procedures are presented, based on the largest and the smallest eigenvalues of the sampl…
Research uses deep learning and copulas to predict multivariate survival data.
problem Handling right-censored and correlated multivariate survival data.
method Integrates deep learning, copula functions, and survival analysis. Uses copula-based activation functions to model nonlinear dependencies.
result Enhanced prediction accuracy for multivariate survival responses.
Efficient online kernel CUSUM detects changes quickly and accurately.
problem Detecting changes in online data streams efficiently.
method Online kernel CUSUM using maximum kernel statistics.
result Increased sensitivity to small changes compared to existing methods.
ARL bridges non-Markovian decision processes with reinforcement learning, improving foresight and stability.
problem Inaccurate foresight in non-Markovian environments due to state-based methods' limitations.
method Lifted state space into a signature-augmented manifold, using a self-consistent field approach to anticipate future path-law.
result ARL achieves deterministic evaluation of expected returns with reduced computational complexity and variance.
We present new algorithms for detecting the emergence of a community in large networks from sequential observations. The networks are modeled using Erdos-Renyi random graphs with edges forming between nodes in the community with higher probability. Based on statistical changepoint detection methodology, we develop thre…
We study sequential change-point detection procedures based on linear sketches of high-dimensional signal vectors using generalized likelihood ratio (GLR) statistics. The GLR statistics allow for an unknown post-change mean that represents an anomaly or novelty. We consider both fixed and time-varying projections, deri…
New method detects changes in high-dimensional data from small samples.
problem Detecting changes in high-dimensional data with limited samples.
method Angular kernel scan framework for detecting marginal distributional shifts.
result Exact population mean factorization and asymptotically distribution-free test.
We develop a mixture procedure for multi-sensor systems to monitor data streams for a change-point that causes a gradual degradation to a subset of the streams. Observations are assumed to be initially normal random variables with known constant means and variances. After the change-point, observations in the subset wi…
Paper finds exact global optima for adversarial representation learning.
problem Obtaining data representations invariant to sensitive attributes.
method Spectral learning for linear functions, kernel representation for non-linear functions.
result Exact closed-form expression for global optima with performance guarantees.
NN-CUSUM detects changes in high-dimensional data using neural networks.
problem Detecting abrupt changes in high-dimensional data.
method Neural network-based CUSUM for online change-point detection.
result NN-CUSUM performs well in detecting changes in high-dimensional data.
Neural PID controllers improve control system performance and are more interpretable.
problem Lack of interpretability in neural PID controllers limits their use in control engineering.
method Extensive study using four benchmark systems with and without noise and disturbances, applying GDNN to PID controllers.
result Neural PID controllers outperform standard PID and model-based control in most tasks.
Boosting improves control of complex systems.
problem Improving performance of controllers for dynamical systems.
method Proposes a boosting framework for online control of dynamical systems.
result An efficient boosting algorithm that combines weak controllers into a more accurate one.
Derives optimal control conditions using calculus of variations.
problem Optimizing Markov control in stochastic control problems.
method Calculus of variations approach to derive necessary conditions.
result Solves the Merton portfolio optimization problem.
Framework simplifies vision-based control and goal discovery.
problem Learning proportional control from visual data.
method Introduces NewtonianVAE for proportional control and goal discovery.
result Dramatic simplification and acceleration of vision-based controllers.
The paper introduces new tests for global controllability in hybrid systems.
problem Global controllability in hybrid systems with discrete events.
method Geometric formulation of hybrid systems and analysis of jump points.
result Hybrid systems can be globally controllable even if continuous systems are not.
This work tackles force control for contact-rich manipulation tasks with rigid robots using RL.
problem Challenges in working with real robotic hardware, especially position-controlled robots.
method Combines RL with traditional force control techniques, implementing parallel position/force control and admittance control.
result Validated methods on both simulation and real robot (UR3 e-series) for force control.
Paper studies constrained control games with a novel approximation method.
problem Games with constrained control directions.
method Approximation procedure based on L 1 L^1 L 1 -stability estimates and almost sure convergence. result Existence of game's value and optimal strategy for the stopper.
RL applied to TCLs for power consumption control.
problem Optimizing power consumption using TCLs with RL.
method Modelica-based reinforcement learning (Q-learning) for stochastic TCLs.
result Q-learning parameters affect controller performance.
A framework integrates machine learning with robust control for safer, more reliable systems.
problem Combining machine learning with robust control for systems with stringent safety and reliability requirements.
method Integrates Gaussian Process Regression and state-of-the-art robust controller synthesis within a framework that provides rigorous guarantees.
result Demonstrated improved performance with more data while maintaining rigorous guarantees.
Paper optimizes energy-based controller for swinging up a pendulum using entropy search.
problem Finding optimal parameters for energy-based controllers is hard.
method Bayesian optimization (Entropy Search) applied to energy-based controller design.
result Optimal controller improves performance of a swinging pendulum.
Paper uses deep reinforcement learning for better control of rocket engines during start-up phases.
problem Lack of optimal control during transient phases of liquid rocket engines.
method Deep reinforcement learning approach for optimal control of a gas-generator engine's continuous start-up phase.
result Deep reinforcement learning controller achieves highest performance and minimal computational effort.
Neural ODEs control graph dynamics with low energy feedback.
problem Controlling complex dynamical systems on graphs.
method Neural Ordinary Differential Equation Control (NODEC) framework.
result NODEC learns low-energy control signals for graph dynamical systems.
Just as an explicit parameterisation of system dynamics by state, i.e., a choice of coordinates, can impede the identification of general structure, so it is too with an explicit parameterisation of system dynamics by control. However, such explicit and fixed parameterisation by control is commonplace in control theory…
Unified control theory and machine learning for safety in uncertain systems.
problem Safety guarantees for systems with measurement model uncertainty.
method Measurement-Robust Control Barrier Functions (MR-CBFs) for control synthesis.
result MR-CBFs ensure safety in perception systems with measurement model uncertainty.
Paper studies optimal control for a specific geometric problem.
problem Optimal control problem associated with the Paneitz obstacle problem.
method Existence and regularity results for optimal controls.
result Existence of optimal controls and their properties.
Optimizes dividend policies in a Brownian model with controlled rates.
problem Realistic optimal dividend policies in a stochastic control problem.
method Delayed linear control strategies for refracted diffusion processes.
result Optimality of delayed linear control strategies for dividend payments.
New method uses neural nets to control systems safely with disturbances.
problem Designing safe control laws for systems with disturbances.
method Imitation learning to train neural network controllers that satisfy CBF constraints.
result Demonstrated on a unicycle model with external disturbances.
Paper proposes a new method to optimize robot body structure and control policy.
problem Optimizing robot body structure and control policy in a coupled manner.
method Revisits co-design problem as a Stackelberg game, incorporating control adaptation dynamics.
result Stackelberg PPO outperforms standard PPO in stability and performance.
Neural network HDP improves virtual inertia control for non-inductive grids.
problem Traditional virtual inertia controllers are not suitable for non-inductive grids.
method Adaptive neural network heuristic dynamic programming (HDP) for optimal control.
result The proposed HDP controller outperforms traditional controllers in virtual inertia control.
Non-bilinear observations make optimal control harder, showing non-convex costs and non-affine optimal controllers.
problem Optimal control from bilinear observations in linear systems is challenging.
method Analytical and numerical methods to study the non-convex cost-to-go and non-affine optimal controllers.
result The Separation Principle does not hold for bilinear observations, leading to non-convex costs and non-affine optimal controllers.
Defense strategy improves controller robustness against adversarial attacks.
problem Adversarial attacks on learning-enabled controllers in CPS.
method Two-stage defense strategy treating controller and environment as black-boxes with unknown dynamics.
result Defense strategy effectively improves controller robustness in realistic control domains.
This paper considers control systems defined on Lie algebroids. After deriving basic controllability tests for general control systems, we specialize our discussion to the class of mechanical control systems on Lie algebroids. This class of systems includes mechanical systems subject to holonomic and nonholonomic const…
Survey of theoretical foundations for policy optimization in control.
problem Understanding the theoretical properties of gradient-based methods in control and reinforcement learning.
method Interdisciplinary review of optimization landscape, convergence, and sample complexity for various control problems.
result Recent theoretical results on stability and robustness in learning-based control.
Neuro-inspired RL solves complex control problems with fewer controllers.
problem Solving nonlinear control problems with unknown dynamics efficiently.
method Hierarchical RL framework combining limb coordination and reinforcement learning.
result Local LQR controllers combined with a reinforcement learner solve global nonlinear problems.
Survey combines FL and control for better adaptability and privacy.
problem Combining FL and control for better adaptability and privacy.
method Combining Federated Learning (FL) and control methods.
result Combining FL and control enhances adaptability, scalability, generalization, and privacy.
New method for handling multi-dimensional singular controls with jump costs in mean-field problems.
problem Handling jump costs in multi-dimensional singular controls.
method Introducing two-layer parametrisations to interpolate jumps on both distributional and pathwise levels.
result Derivation of a DPP and characterisation of the value function as a minimal super-solution to a quasi-variational inequality.
Researchers use Meyer-σ-fields to model information flow in irreversible investment problems.
problem Modeling information flows in stochastic control problems with jumps.
method Using Meyer-σ-fields as a tool to model information flow.
result Different signals on exogenous jumps lead to different optimal controls.