A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New algorithm for nonstationary multi-armed bandits with optimal performance.
Reinforcement learning (RL) methods learn optimal decisions in the presence of a stationary environment. However, the stationary assumption on the environment is very restrictive. In many real world problems like traffic signal control, robotic applications, one often encounters situations with non-stationary environme…
Self-supervised policy adapts after deployment without rewards.
New method recovers causal DAGs from general environments without strict assumptions.
Framework for robust decision making in changing environments with privacy constraints.
New bandit algorithm detects and adapts to seasonal changes in rewards.
OSAMD adapts online to changing distributions with limited labels.
A new buffer system improves continual learning in RL agents by adapting to changing environments.
Researchers use DT to transfer policies from one environment to another using causal reasoning.
Bayesian model for multi-environment prediction with latent variable changes.
Autonomous lane changing is a critical feature for advanced autonomous driving systems, that involves several challenges such as uncertainty in other driver's behaviors and the trade-off between safety and agility. In this work, we develop a novel simulation environment that emulates these challenges and train a deep r…
AdaRL adapts quickly to new environments with minimal data.
High-dimensional always-changing environments constitute a hard challenge for current reinforcement learning techniques. Artificial agents, nowadays, are often trained off-line in very static and controlled conditions in simulation such that training observations can be thought as sampled i.i.d. from the entire observa…
Proposes MSS to identify causal structure from heterogeneous environments.
Proposes a method to adapt models in nonstationary environments using ℓ1 regularization.
A model is developed to study the effectiveness of innovation and its impact on structure creation and structure change on agent-based societies. The abstract model that is developed is easily adapted to any particular field. In any interacting environment, the agents receive something from the environment (the other a…
We propose a novel approach to address one aspect of the non-stationarity problem in multi-agent reinforcement learning (RL), where the other agents may alter their policies due to environment changes during execution. This violates the Markov assumption that governs most single-agent RL methods and is one of the key c…
Novel graph-spanning algorithm detects changes in high-dimensional data.
New method detects anomalies in systems influenced by their environment.
New algorithms detect changes in non-stationary MABs for better performance.
This paper proposes a formal approach to online learning and planning for agents operating in a priori unknown, time-varying environments. The proposed method computes the maximally likely model of the environment, given the observations about the environment made by an agent earlier in the system run and assuming know…
This thesis improves OCO algorithms for dynamic data environments.
BAM integrates new data while selectively remembering past observations.
A new method shapes reinforcement learning environments by abstracting large state spaces.
Multi-armed bandit algorithms have become a reference solution for handling the explore/exploit dilemma in recommender systems, and many other important real-world problems, such as display advertisement. However, such algorithms usually assume a stationary reward distribution, which hardly holds in practice as users' …
Robust OPE framework uses human inputs to improve policy evaluation in changing environments.
Tree-based regularization improves latent variable inference from related datasets.
AAMDRL uses DRL to manage assets in noisy, changing environments.
New algorithms detect and react to multiple change points in online learning.
Learning in a non-stationary environment is an inevitable problem when applying machine learning algorithm to real world environment. Learning new tasks without forgetting the previous knowledge is a challenge issue in machine learning. We propose a Kalman Filter based modifier to maintain the performance of Neural Net…
The paper proposes a method to adapt machine learning models to changing conditions.
We propose algorithms for online principal component analysis (PCA) and variance minimization for adaptive settings. Previous literature has focused on upper bounding the static adversarial regret, whose comparator is the optimal fixed action in hindsight. However, static regret is not an appropriate metric when the un…
AACC improves RL performance in changing environments.
We study the non-stationary stochastic multiarmed bandit (MAB) problem and propose two generic algorithms, namely, the limited memory deterministic sequencing of exploration and exploitation (LM-DSEE) and the Sliding-Window Upper Confidence Bound# (SW-UCB#). We rigorously analyze these algorithms in abruptly-changing a…
The paper tackles policy learning in dynamic environments using causal methods.
It has been postulated that a good representation is one that disentangles the underlying explanatory factors of variation. However, it remains an open question what kind of training framework could potentially achieve that. Whereas most previous work focuses on the static setting (e.g., with images), we postulate that…
Concept drift is formally defined as the change in joint distribution of a set of input variables X and a target variable y. The two types of drift that are extensively studied are real drift and virtual drift where the former is the change in posterior probabilities p(y|X) while the latter is the change in distributio…
Optimizes latency and false alarm probability in change detection problems.
Study adapts combinatorial semi-bandit for piecewise stationary, causally related rewards.
New algorithm tackles non-stationary delayed feedback in recommender systems.
To deal with changing environments, a new performance measure -- adaptive regret, defined as the maximum static regret over any interval, was proposed in online learning. Under the setting of online convex optimization, several algorithms have been successfully developed to minimize the adaptive regret. However, existi…
It has been postulated that a good representation is one that disentangles the underlying explanatory factors of variation. However, it remains an open question what kind of training framework could potentially achieve that. Whereas most previous work focuses on the static setting (e.g., with images), we postulate that…
This work introduces a method to attribute model performance drops to distribution shifts.
Develops RL algorithm for lifelong non-stationary environments.
A key challenge in online learning is that classical algorithms can be slow to adapt to changing environments. Recent studies have proposed "meta" algorithms that convert any online learning algorithm to one that is adaptive to changing environments, where the adaptivity is analyzed in a quantity called the strongly-ad…
Lane change is a challenging task which requires delicate actions to ensure safety and comfort. Some recent studies have attempted to solve the lane-change control problem with Reinforcement Learning (RL), yet the action is confined to discrete action space. To overcome this limitation, we formulate the lane change beh…
Autonomous robots need to interact with unknown, unstructured and changing environments, constantly facing novel challenges. Therefore, continuous online adaptation for lifelong-learning and the need of sample-efficient mechanisms to adapt to changes in the environment, the constraints, the tasks, or the robot itself a…
Optimization in the presence of sharp (non-Lipschitz), unpredictable (w.r.t. time and amount) changes is a challenging and largely unexplored problem of great significance. We consider the class of piecewise Lipschitz functions, which is the most general online setting considered in the literature for the problem, and …