This paper tackles RL in non-stationary environments, improving decision-making.
problem Develop optimal RL decisions in non-stationary environments.
method Adapted change point algorithm for detecting model changes and developed an RL algorithm.
result RL algorithm maximizes long-run reward in changing environments.
Proposes a Kalman Filter modifier to improve neural network performance in changing environments.
problem Maintaining performance of neural networks in non-stationary environments.
method Kalman Filter based modifier to adapt to changes.
result The proposed model adapts better to changes with a 0.4% accuracy drop compared to 90% for conventional models.
Optimizes particle filtering for non-stationary environments.
problem Tracking and adapting to non-stationary environments in online prediction.
method Formulated an efficient particle filtering method using online mirror descent algorithm.
result Achieves optimal particle efficiency in non-stationary environments.
Predictive sampling improves on Thompson sampling for non-stationary bandit environments.
problem Thompson sampling fails in non-stationary bandit environments.
method Proposes predictive sampling, which deprioritizes actions based on information loss rate.
result Predictive sampling outperforms Thompson sampling in all tested non-stationary environments.
Transformers achieve near-optimal dynamic regret in non-stationary reinforcement learning.
problem Understanding and handling non-stationary environments in reinforcement learning.
method Demonstrated that transformers can achieve nearly optimal dynamic regret bounds in non-stationary settings.
result Transformers can approximate and learn strategies for non-stationary environments, matching or outperforming existing expert algorithms.
Paper introduces CRLMaze, a new benchmark for continual reinforcement learning in 3D non-stationary environments.
problem Challenges of training reinforcement learning agents in high-dimensional, always-changing environments.
method End-to-end model-free continual reinforcement learning strategy.
result Competitive results in a complex 3D non-stationary task, outperforming four baselines.
New algorithms reduce dynamic regret in non-stationary RL environments.
problem Optimizing policies in environments that change over time.
method POWER and POWER++ algorithms for policy optimization with dynamic regret analysis.
result POWER++ improves dynamic regret by actively adapting to non-stationarity.
Proposes a new algorithm for non-stationary bandits.
problem Non-stationary reward distributions in contextual bandits.
method Multiscale changepoint detection for adaptive learning.
result Regret bound analysis and superior performance in experiments.
New algorithm tackles non-stationary delayed feedback in recommender systems.
problem Challenges in learning from delayed feedback in non-stationary environments.
method Developed a UCRL-based algorithm for non-stationary, delayed bandits with intermediate observations.
result Sublinear regret guarantees for the proposed algorithm in non-stationary delayed environments.
New algorithm reduces dynamic regret in non-stationary BCO problems.
problem Sequential decision-making with non-stationary environments.
method Proposes a novel algorithm for dynamic regret measurement.
result Achieves optimal dynamic regret for one-point and two-point feedback models.
A new algorithm for non-stationary linear bandits with improved regret bound.
problem Non-stationary linear bandit problem with time-varying rewards.
method D-LinUCB, a discounted linear regression algorithm with exponential weights.
result Upper bound on dynamic regret of order d^{2/3} B_T^{1/3}T^{2/3}, optimal in slowly-varying and abruptly-changing environments.
Study online ranking in non-stationary environments with abrupt preference changes.
problem Online learning to rank in a non-stationary cascade model where user preferences change abruptly.
method Cascading non-stationary bandits, proposing CascadeDUCB and CascadeSWUCB algorithms.
result Upper and lower bounds on regret for cascading non-stationary bandits algorithms.
Paper proposes a hybrid model for financial time series prediction using sentiment analysis.
problem Challenges in forecasting in non-stationary, complex environments with heterogeneous data.
method Hybrid model combining GANs with NLP-based sentiment analysis.
result Hybrid model enhances robustness in non-stationary environments.
Melanie improves predictive performance in non-stationary data streams by transferring knowledge between multiple sources.
problem Concept drift in data streams leads to poor predictive performance.
method Melanie uses multiple sub-classifiers to learn different aspects from various sources and compose an ensemble for the target concept.
result Melanie improves predictive performance over existing algorithms by leveraging multiple sources.
Study non-stationary MDPs using worst-case RL, proposing RATS algorithm.
problem Robust zero-shot planning in non-stationary stochastic environments.
method Model-Based Reinforcement Learning, worst-case approach.
result RATS algorithm demonstrates benefits over reference methods.
BAM integrates new data while selectively remembering past observations.
problem Slow adaptation and convergence to incorrect parameter values in non-stationary environments.
method Bayes' theorem with adaptive memory selection.
result BAM generalizes and demonstrates continuous adaptation in changing environments.
New RL algorithm tackles non-stationary environments with flexible policy updates.
problem Non-stationary reinforcement learning with time-varying rewards and transition probabilities.
method Model-free policy-based algorithm NS-NAC with restart-based exploration and dynamic learning rates.
result Dynamic regret of i l d e O ( ∣ S ∣ 1 / 2 ∣ A ∣ 1 / 2 Δ T 1 / 6 T 5 / 6 ) ilde{\mathscr O}(|S|^{1/2}|A|^{1/2}Δ_T^{1/6}T^{5/6}) i l d e O ( ∣ S ∣ 1/2 ∣ A ∣ 1/2 Δ T 1/6 T 5/6 ) for both algorithms. Efficient algorithms for non-stationary contexts reduce regret to optimal bounds.
problem Contextual bandits in non-stationary environments where distributions change over time.
method Developed algorithms that adapt to changes in distribution using statistical tests.
result Achieved optimal regret bounds for non-stationary environments, matching or improving existing results.
A new memory system handles non-stationary environments by self-sizing and retaining memories.
problem Non-stationary environments where memories arrive over time and must be distinguished from noise.
method A self-sizing continual associative memory that generalizes Hopfield's network, handling adaptation and novelty.
result The memory system grows to the intrinsic memory demand of the environment and retains memories without forgetting.
A framework for prototype-based classifiers in changing data environments.
problem Learning in non-stationary environments with concept drift.
method Analytical methods from statistical physics applied to LVQ systems.
result Basic LVQ algorithms are suitable for non-stationary environments, but weight decay does not improve performance.
New method optimizes policies in non-stationary environments.
problem Optimizing policies in non-stationary, context-dependent environments.
method Two-phase approach: offline learning and online adaptation.
result Our method outperforms existing approaches in both synthetic and real-world datasets.
Study incentivizes exploration in non-stationary MAB with compensation.
problem Incentivized exploration for non-stationary stochastic bandits with biased feedback.
method Proposed algorithms for abruptly-changing and continuously-changing non-stationary environments.
result Achieves sublinear regret and compensation over time.
Two algorithms reduce regret in non-stationary MAB problems.
problem Non-stationary stochastic multiarmed bandit problems.
method LM-DSEE and SW-UCB# algorithms for abruptly-changing and slowly-varying environments.
result Expected cumulative regret is upper bounded by sublinear functions of time.
Study on consistency of ML methods for moving objects in non-stationary environments.
problem Consistency of machine learning methods for moving objects in non-stationary environments.
method Least squares, ridge regression, and ℓ s \ell_s ℓ s -penalized least squares methods under non-stationary spatial-temporal sampling. result Consistency and asymptotic normality of the estimates under weak conditions.
New algorithms adaptively calibrate predictions in non-stationary environments, matching optimal rates.
problem Designing online prediction algorithms that adapt to varying levels of non-stationarity.
method Epoch-based scheduling and non-uniform partitioning of the prediction space.
result Achieves adaptive calibration guarantees under multiple measures with optimal rates.
New method extends supervised learning for non-stationary control problems.
problem Optimal control in non-stationary, reset-free environments.
method Prospective Learning with Control (PLuC) using Empirical Risk Minimization (ERM).
result ERM asymptotically achieves Bayes optimal policy in non-stationary environments.
DAL enhances black-box bandit algorithms for non-stationary environments.
problem Non-stationary environments in bandit problems.
method DAL combines any stationary bandit algorithm with a change detector.
result DAL consistently outperforms state-of-the-art methods in various non-stationary scenarios.
Paper uses RL for market making, improving stability in non-stationary markets.
problem Optimizing market making strategies in non-stationary limit order book dynamics.
method Reinforcement Learning (Proximal-Policy Optimization) applied to a simulator.
result RL agent outperforms closed-form optimal solution in non-stationary markets.
Study non-stationary bandits with resource constraints.
problem Maximize reward in a non-stationary environment with resource constraints.
method Propose new non-stationarity measure and use primal-dual analysis.
result Upper and lower bounds for non-stationary BwK problem.
Framework models supervised learning in non-stationary data.
problem Non-stationary data in supervised learning.
method Statistical physics methods applied to LVQ and neural networks.
result LVQ and ReLU have different sensitivity to concept drift.
New algorithm improves A/B testing for dynamic environments.
problem Manual A/B testing is inefficient and prone to human error.
method Expert learning approach with Follow-The-Best-Interval (FTBI) algorithm.
result FTBI outperforms current methods in dynamic environments.
Hop Sampling improves GNNs in non-stationary environments by preventing overfitting.
problem Non-stationary environments cause concept drift, making GNNs overfit to training graphs.
method Randomly selects the number of propagation steps in GNNs to prevent overfitting.
result Improves GNNs' prediction accuracy by 7.97% and 16.93% in LINE Coupon recommender systems.
LC-SAC tackles non-stationary dynamics in reinforcement learning.
problem Degradation of deep RL methods in non-stationary environments.
method LC-SAC uses latent context encoders and contrastive loss for dynamic information capture.
result LC-SAC outperforms SAC on environments with drastic dynamics changes.
Unified approach for non-stationary and clustered bandits.
problem Solving non-stationary and clustered bandits with overlapping solutions.
method Test of homogeneity for seamless integration of non-stationary and clustered bandits.
result Unified solution framework for change detection and cluster identification.
Building models, or maps, of robot environments is a highly active research area; however, most existing techniques construct unstructured maps and assume static environments. In this paper, we present an algorithm for learning object models of non-stationary objects found in office-type environments. Our algorithm exp…
Study MNL-Bandit in non-stationary settings with optimal regret bound.
problem Optimizing decisions in a non-stationary environment for multi-armed bandit problems.
method Develops an algorithm with worst-case expected regret bound and introduces new techniques to handle non-stationarity.
result Optimal regret bound proven for the MNL-Bandit problem in non-stationary environments.
New pricing algorithm learns demand curves and optimizes prices in dynamic markets.
problem Dynamic pricing in markets with incomplete demand information and shifting conditions.
method Actor-Critic Information-Directed Pricing (ACIDP) using IDS algorithms and auditing procedures.
result ACIDP outperforms UCB and TS in market environment shifts.
New algorithms reduce dynamic regret for convex and smooth functions in non-stationary environments.
problem Online convex optimization in non-stationary environments.
method Proposed novel online algorithms exploiting smoothness to reduce dynamic regret.
result Dynamic regret improved to O ( T ) \mathcal{O}(T) O ( T ) for convex and smooth functions. The study questions the necessity of accurate models in MBRL.
problem The difficulty of achieving and utilizing accurate models in complex or changing environments.
method Introduces a meta-learning algorithm focused on model usefulness rather than accuracy.
result Meta-learning models enable faster learning than accurate models built with domain-specific knowledge.
This paper tackles continuous domain adaptation with a new approach.
problem Learning in non-stationary environments, especially domain drift.
method Variational domain-agnostic feature replay, composed of inference, generative, and solver modules.
result Demonstrates the effectiveness of the proposed approach for practical usage.
New bandit algorithm detects and adapts to changing user preferences.
problem Dynamic user preferences in recommender systems.
method Contextual bandit algorithm that detects changes and updates strategy.
result Upper regret bound analysis shows effectiveness in non-stationary environments.
Study tackles non-stationary bandit convex optimization with new algorithms.
problem Minimizing regret in non-stationary environments with various measures of non-stationarity.
method Proposed Tilted Exponentially Weighted Average with Sleeping Experts (TEWA-SE) for strongly convex losses and clipped Exploration by Optimization (cExO) for general convex losses.
result Proved minimax-optimality of TEWA-SE for strongly convex losses and introduced cExO for general convex losses.
New approach turns optimal stationary RL into non-stationary RL without prior knowledge.
problem Optimal RL in non-stationary environments without prior knowledge of non-stationarity.
method Black-box reduction of optimal stationary RL algorithms to non-stationary RL.
result Achieves optimal dynamic regret bounds in various RL settings.
Paper introduces novel Bandit algorithms for non-stationary environments in finance.
problem Non-stationary reward distributions in financial markets.
method Introduces Adaptive Discounted Thompson Sampling (ADTS) and Combinatorial Adaptive Discounted Thompson Sampling (CADTS) for non-stationary environments in portfolio optimization.
result Bandit Networks improve portfolio optimization performance by 20% compared to classical models.
MPC outperforms reactive budgeting in non-stationary return environments.
problem Optimizing budget allocation under non-stationary returns.
method Receding-horizon Model Predictive Control (MPC) compared to reactive policies.
result MPC consistently outperforms reactive budgeting when return dynamics are predictable.
Adaptive sparse GP model for non-stationary data.
problem Non-stationary environments require efficient model updates.
method Reformulated variational sparse GP with adaptive forgetting factor and single inducing point update.
result Fast convergence and efficient model update in non-stationary environments.
A new method optimizes in nonstationary environments with many arms efficiently.
problem Optimizing in nonstationary environments with a large number of arms.
method Gaussian interpolation to learn continuous Lipschitz reward functions in nonstationary environments.
result Efficiently learns continuous Lipschitz reward functions with O ∗ ( T ) \mathcal{O}^*(\sqrt{T}) O ∗ ( T ) cumulative regret. Bayes Factor Surprise enables rapid adaptation to changing environments.
problem Learning in volatile, non-stationary stochastic environments.
method Bayesian inference in a hierarchical model with a Bayes Factor Surprise probability ratio.
result Novel surprise-based algorithms improve parameter estimation and performance.