Two new algorithms reduce group regret in abruptly changing multi-player bandit problems.
problem Reducing group regret in multi-player bandit problems in environments that change suddenly.
method Design of two novel algorithms: RR-SW-UCB# and SW-DLP.
result Expected cumulative group regret converges to zero over time.
We study the non-stationary stochastic multiarmed bandit (MAB) problem and propose two generic algorithms, namely, the limited memory deterministic sequencing of exploration and exploitation (LM-DSEE) and the Sliding-Window Upper Confidence Bound# (SW-UCB#). We rigorously analyze these algorithms in abruptly-changing a…
Meta-algorithm reduces exploration steps in changing CMPs.
problem Navigating in environments where transition probabilities change abruptly.
method MNM learning meta-algorithm, exploration steps as performance measure.
result Upper bound on exploration steps in terms of changes.
DS-TS adapts to abrupt and smooth changes in bandit problems.
problem Non-stationary multi-armed bandit problems with abrupt and smooth changes.
method Discounted Thompson Sampling with Gaussian priors.
result Achieves nearly optimal regret bound for both abrupt and smooth changes.
Study incentivizes exploration in non-stationary MAB with compensation.
problem Incentivized exploration for non-stationary stochastic bandits with biased feedback.
method Proposed algorithms for abruptly-changing and continuously-changing non-stationary environments.
result Achieves sublinear regret and compensation over time.
New bandit algorithm detects and adapts to seasonal changes in rewards.
problem Adapting to abrupt changes in user preferences during events.
method Detects and adapts to seasonal changes in reward function.
result Outperforms state-of-the-art algorithms for non-stationary environments.
New strategy for gambling with changing environments achieves sub-linear regret.
problem Optimizing rewards in a non-stationary X \mathcal{X} X -armed bandit problem. method Proposes a novel strategy for environments that change behavior abruptly.
result Proves the proposed strategy attains sub-linear cumulative regret.
Study online ranking in non-stationary environments with abrupt preference changes.
problem Online learning to rank in a non-stationary cascade model where user preferences change abruptly.
method Cascading non-stationary bandits, proposing CascadeDUCB and CascadeSWUCB algorithms.
result Upper and lower bounds on regret for cascading non-stationary bandits algorithms.
A new algorithm for non-stationary linear bandits with improved regret bound.
problem Non-stationary linear bandit problem with time-varying rewards.
method D-LinUCB, a discounted linear regression algorithm with exponential weights.
result Upper bound on dynamic regret of order d^{2/3} B_T^{1/3}T^{2/3}, optimal in slowly-varying and abruptly-changing environments.
New TS algorithms improve performance in non-stationary multi-armed bandit problems.
problem Sequential decision-making with evolving action rewards.
method Sliding-window Thompson sampling approaches with different priors.
result Unified regret upper bound for arbitrary non-stationary MABs.
Robust PCA methods are typically batch algorithms which requires loading all observations into memory before processing. This makes them inefficient to process big data. In this paper, we develop an efficient online robust principal component methods, namely online moving window robust principal component analysis (OMW…
New GLB algorithm handles non-stationary data with forgetting.
problem Non-stationary GLB with non-convex projection or burn-in phases.
method Self-concordant GLB with sliding window or exponential weights for forgetting.
result Novel confidence-based algorithm for maximum likelihood estimator.
In this paper, a geometric function is introduced to reflect the attenuation speed of impact of one firm's default to its partner. If two firms are competitions (copartners), the default intensity of one firm will decrease (increase) abruptly when the other firm defaults. As time goes on, the impact will decrease gradu…
Online detection of abrupt changes in high-dimensional data streams.
problem Detecting abrupt changes in high-dimensional, streaming data with multiple subspaces.
method Dynamic sparse subspace learning approach with multiple structural change-point model, Bayesian information criterion for penalty coefficients selection, and Pruned Exact Linear Time algorithm.
result Effectiveness demonstrated through simulation and real gesture data studies.
Paper proposes a beta distribution method for detecting concept drift in adaptive classifiers.
problem Adaptive classifiers need to detect and respond to concept drift in real-time data streams.
method The paper introduces a beta distribution model to monitor model error and identify abnormal behavior as drift.
result The method effectively detects abrupt changes in model error, improving classifier performance.
New algorithm limits regret in changing MDPs.
problem Reinforcement learning in MDPs with time-varying rewards and transitions.
method Proposed an algorithm with performance guarantees for non-stationary policies.
result First variational regret bound for general RL setting.
This paper tackles RL in non-stationary environments, improving decision-making.
problem Develop optimal RL decisions in non-stationary environments.
method Adapted change point algorithm for detecting model changes and developed an RL algorithm.
result RL algorithm maximizes long-run reward in changing environments.
New algorithm for nonstationary multi-armed bandits with optimal performance.
problem Nonstationary multi-armed bandits with changing model parameters over time.
method Adaptive Resetting Bandit (ADR-bandit) algorithm using adaptive windowing techniques.
result ADR-bandit achieves nearly optimal performance in both abrupt and gradual changes.
Task offloading is a promising technology to exploit the benefits of fog computing. An effective task offloading strategy is needed to utilize the computational resources efficiently. In this paper, we endeavor to seek an online task offloading strategy to minimize the long-term latency. In particular, we formulate a s…
A geometric transition is a continuous path of geometric structures that changes type, meaning that the model geometry, i.e. the homogeneous space on which the structures are modeled, abruptly changes. In order to rigorously study transitions, one must define a notion of geometric limit at the level of homogeneous spac…
Deep RL agent improves lane changing in unpredictable traffic.
problem Uncertainty in other drivers' behaviors and safety vs agility trade-off.
method Developed a deep reinforcement learning agent in a simulated highway environment.
result Significantly better performance in noisy environments compared to heuristic methods.
Self-supervised policy adapts after deployment without rewards.
problem Generalizing reinforcement learning policies across different environments.
method Uses self-supervision to train policies in new environments without reward signals.
result Significant improvements in generalization across diverse environments.
New method recovers causal DAGs from general environments without strict assumptions.
problem Recovering causal DAGs from real-world data with varying distributions.
method Formalizes desiderata for causal representation learning in general environments, leveraging sufficient change conditions up to third-order derivatives.
result Fully recovers latent DAG and identifies latent variables up to minor indeterminacies under nonparametric mixing.
Proposes a Kalman Filter modifier to improve neural network performance in changing environments.
problem Maintaining performance of neural networks in non-stationary environments.
method Kalman Filter based modifier to adapt to changes.
result The proposed model adapts better to changes with a 0.4% accuracy drop compared to 90% for conventional models.
Enhances deep RL learning with stable policy updates.
problem Stability and performance degradation in deep RL methods.
method EVEREST method providing conservative updates with confidence bounds.
result Significant improvements in continuous control and Atari benchmarks.
RL agent learns to smoothly change lanes in a dynamic driving environment.
problem Challenging lane change control with safety and comfort.
method Formulated continuous action for lane change in DDPG algorithm, defined reward function for learning.
result Successfully changed lanes with 100% success rate in diverse driving situations.
Framework for robust decision making in changing environments with privacy constraints.
problem Interactive decision making in changing environments with constraints.
method Hybrid Decision Making with Structured Observations (hybrid DMSO) framework, local differentially private decision making, query-based learning, robust and smooth decision making.
result Strong connections and bounds derived for DEC, SQ dimension, local minimax complexity, learnability, and joint differential privacy.
New algorithms minimize regret in changing environments for piecewise Lipschitz functions.
problem Optimizing in unpredictable, time-varying conditions for piecewise Lipschitz functions.
method Introduced shifting regret as a metric, and provided bounds for O ( s d T log T + s T 1 − β ) O(\sqrt{sdT\log T}+sT^{1-β}) O ( s d T log T + s T 1 − β ) . result Improved bounds for β β β -dispersed functions, with empirical validation in online clustering. Adaptive PCA algorithms for changing environments.
problem Static adversarial regret is not suitable for changing environments.
method Online adaptive algorithms for PCA and variance minimization with sub-linear adaptive regret guarantees.
result The proposed algorithms adapt to changing environments.
Paper introduces CRLMaze, a new benchmark for continual reinforcement learning in 3D non-stationary environments.
problem Challenges of training reinforcement learning agents in high-dimensional, always-changing environments.
method End-to-end model-free continual reinforcement learning strategy.
result Competitive results in a complex 3D non-stationary task, outperforming four baselines.
The study uses State Switching Markov Autoregressive models to identify and predict market regimes.
problem Adapting to abrupt changes in financial markets and identifying stable investment strategies.
method State Switching Markov Autoregressive models and Wyckoff Price Regimes.
result A dynamically adaptive trading system that outperforms traditional alphas.
Adaptive policies for multi-agent RL improve adaptiveness in changing environments.
problem Non-stationarity in multi-agent reinforcement learning where other agents may alter their policies.
method Train multiple adaptive policies for each agent and a policy predictor to select the best policy at execution time.
result Agents trained with our method outperform state-of-the-art methods in all tested environments.
OSAMD adapts online to changing distributions with limited labels.
problem Models struggle with continual distribution shifts and expensive labeling in changing environments.
method Online Active Continual Adaptation with OSAMD, an online teacher-student structure and margin-based criterion.
result OSAMD achieves favorable dynamic regret bounds under changing environments with limited labels.
New algorithm for multi-player bandits in changing environments.
problem Sequential action selection with collisions and adversarial losses.
method First provable Multi-player Bandit algorithm for changing environments.
result Resolves open problem in adversarial multi-player settings.
A new buffer system improves continual learning in RL agents by adapting to changing environments.
problem Improving RL agents' ability to learn from changing environments over time.
method Multi-timescale replay buffer combined with invariant risk minimization.
result The method shows improvement over baselines in continual learning settings.
Researchers use DT to transfer policies from one environment to another using causal reasoning.
problem Adapting to changes in environmental dynamics in reinforcement learning.
method Applying causal counterfactual reasoning to Decision Transformer (DT) architecture for policy transfer.
result DT successfully transfers a learned policy to new environments while retaining most of the reward.
Bayesian model for multi-environment prediction with latent variable changes.
problem Prediction in environments with changing latent variable distributions.
method Bayesian model with empirical Bayes prior and amortized variational algorithm.
result Method outperforms previous approaches in new environments.
AdaRL adapts quickly to new environments with minimal data.
problem Quickly adapting to new environments in reinforcement learning.
method AdaRL uses a parsimonious graphical representation to encode changes across domains.
result AdaRL can efficiently adapt policies to target domains with few samples.
Adding metadata abruptly changes network inference outcomes.
problem Understanding the impact of metadata on network inference.
method Investigated the effect of metadata on network inference problems.
result Metadata causes abrupt transitions in inference outcomes.
Paper proposes a method for learning and planning in time-varying environments.
problem Learning and planning in unknown, time-varying environments.
method Computes the maximally likely model of the environment using maximum likelihood estimation.
result Generalizes learning algorithms for time-invariant Markov decision processes to time-varying ones.
Proposes MSS to identify causal structure from heterogeneous environments.
problem Distribution shifts between environments violate i.i.d. data assumption.
method Sparse mechanism shift hypothesis, score-based approach.
result Identifies entire causal structure with high probability.
Proposes a method to adapt models in nonstationary environments using ℓ1 regularization.
problem Adapting models to nonstationary environments in machine learning.
method Integrates ℓ1 regularization of differences between source and target parameters.
result Effective balance of stability and plasticity in model adaptation.
A model is developed to study the effectiveness of innovation and its impact on structure creation and structure change on agent-based societies. The abstract model that is developed is easily adapted to any particular field. In any interacting environment, the agents receive something from the environment (the other a…
Universal algorithm minimizes adaptive regret for various convex functions.
problem Minimizing adaptive regret in changing environments for multiple convex functions.
method Borrowing MetaGrad's idea of multiple learning rates and using sleeping experts.
result First universal algorithm for minimizing adaptive regret of convex functions.
Novel graph-spanning algorithm detects changes in high-dimensional data.
problem Detecting changes in high-dimensional data across various distributions.
method Graph-spanning algorithm designed for offline and online data.
result Achieves high detection power with minimal separation rate.
New method detects anomalies in systems influenced by their environment.
problem Detecting anomalies in systems under environmental influence.
method Adversarial learning and time series representation learning.
result Successfully addresses label sparsity and subjectivity in anomaly detection.
New algorithms detect changes in non-stationary MABs for better performance.
problem Non-stationary MAB environments where arm reward distributions change over time.
method Modular Detection Augmented Bandit (DAB) procedures with improved performance lower bounds.
result Modular DAB procedures achieve order-optimal regret bounds for various change detectors and bandit algorithms.
This thesis improves OCO algorithms for dynamic data environments.
problem Sequential, changing data in big data environments.
method Designing algorithms to adapt to changing environments.
result Improved algorithms for online resource allocation.