Transformers achieve near-optimal dynamic regret in non-stationary reinforcement learning.
problem Understanding and handling non-stationary environments in reinforcement learning.
method Demonstrated that transformers can achieve nearly optimal dynamic regret bounds in non-stationary settings.
result Transformers can approximate and learn strategies for non-stationary environments, matching or outperforming existing expert algorithms.
Proposes a method to forecast non-stationary time series.
problem Challenges of non-stationary conditional distributions in deep learning.
method Bayesian dynamic model + deep conditional distribution model.
result Adapts to non-stationary time series better than state-of-the-art solutions.
New algorithm tackles non-stationary reinforcement learning with general function approximation.
problem Understanding non-stationary MDPs with function approximation.
method Dynamic Bellman Eluder (DBE) dimension for complexity, sliding window mechanism, confidence set design.
result Upper bound on dynamic regret for proposed SW-OPEA algorithm.
Framework infers Langevin dynamics from stochastic observations of latent systems.
problem Inferring non-stationary Langevin dynamics from indirect stochastic observations.
method Non-parametric framework explicitly modeling stochastic observation process and non-stationary latent dynamics.
result Correct inference of non-stationary dynamics requires accounting for non-equilibrium states and observation duration.
Algorithm adapts to non-stationary rewards without prior knowledge.
problem Optimizing decisions in non-stationary environments without prior knowledge of changes.
method Optimization-based algorithm that restarts when non-stationarity is detected.
result Achieves tighter dynamic regret bound and is nearly minimax optimal.
New algorithms reduce dynamic regret in non-stationary RL environments.
problem Optimizing policies in environments that change over time.
method POWER and POWER++ algorithms for policy optimization with dynamic regret analysis.
result POWER++ improves dynamic regret by actively adapting to non-stationarity.
ETGPSSM efficiently models high-dimensional, non-stationary systems with reduced complexity.
problem Prohibitive computational and parametric complexity in high-dimensional, non-stationary dynamical systems.
method ETGPSSM integrates a single shared GP with input-dependent normalizing flows for scalable and flexible modeling.
result ETGPSSM outperforms existing models in computational efficiency and accuracy.
New definition resolves ambiguity in non-stationary bandit classification.
problem Ambiguity in classifying non-stationary bandits using existing definitions.
method Introducing a formal definition that resolves ambiguity and provides a unified approach.
result Unified approach applicable to both Bayesian and frequentist formulations, resolves classification issues.
Improved algorithm for adaptive dueling bandits with near-optimal regret bound.
problem Non-stationary dueling bandits with unknown number of preference changes.
method Elimination-based rescheduling algorithm for adaptive dynamic regret.
result Near-optimal i l d e O ( S e x t t t C W T ) ilde{O}(\sqrt{S^{ exttt{CW}} T}) i l d e O ( S e x ttt C W T ) dynamic regret bound. New algorithm reduces dynamic regret without prior function change knowledge.
problem Non-stationary stochastic optimization with bandit feedback.
method Fixed step sizes combined with multi-scale sampling framework.
result Achieves optimal dynamic regret without prior function change knowledge.
New RL algorithm tackles non-stationary environments with flexible policy updates.
problem Non-stationary reinforcement learning with time-varying rewards and transition probabilities.
method Model-free policy-based algorithm NS-NAC with restart-based exploration and dynamic learning rates.
result Dynamic regret of i l d e O ( ∣ S ∣ 1 / 2 ∣ A ∣ 1 / 2 Δ T 1 / 6 T 5 / 6 ) ilde{\mathscr O}(|S|^{1/2}|A|^{1/2}Δ_T^{1/6}T^{5/6}) i l d e O ( ∣ S ∣ 1/2 ∣ A ∣ 1/2 Δ T 1/6 T 5/6 ) for both algorithms. LC-SAC tackles non-stationary dynamics in reinforcement learning.
problem Degradation of deep RL methods in non-stationary environments.
method LC-SAC uses latent context encoders and contrastive loss for dynamic information capture.
result LC-SAC outperforms SAC on environments with drastic dynamics changes.
Efficient methods reduce projections in non-stationary online learning.
problem Optimizing dynamic and adaptive regret in non-stationary online learning environments.
method Presented efficient methods reducing the number of projections per round from O ( log T ) O(\log T) O ( log T ) to 1 1 1 . result Reduced number of projections per round from O ( log T ) O(\log T) O ( log T ) to 1 1 1 for optimizing dynamic and adaptive regret. DARLING tackles non-stationary RL with guarantees, improving dynamic regret.
problem Non-stationary reinforcement learning in unknown change points.
method Detection Augmented Reinforcement Learning (DARLING) for tabular and linear MDPs.
result DARLING matches minimax lower bounds in tabular and linear MDPs.
Bayesian method improves SOM training for dynamic data.
problem Training Self Organizing Maps (SOM) for non-stationary data.
method Variational Bayesian approach with adaptive neighborhood radius.
result Method outperforms other adaptive methods in high dimensions.
Unified approach for non-stationary linear bandits with dynamic regret.
problem Non-stationary linear bandits with round-specific feasible actions and drifting reward models.
method Unified misspecification-reduction viewpoint, restarting algorithms with misspecification-dependent regret guarantees.
result Optimal \(T^{2/3}P_T^{1/3}\) dynamic-regret dependence for both linear bandits and contextual linear bandits.
New method reduces dynamic regret for non-stationary bandits.
problem Non-stationary stochastic multi-armed bandit problem with changing optimal arm.
method Proposes a method achieving near-optimal dynamic regret without prior knowledge of changes.
result Achieves O ~ ( K N ( S + 1 ) ) \widetilde O(\sqrt{K N(S+1)}) O ( K N ( S + 1 ) ) dynamic regret. MetaCURL tackles non-stationary MDPs with optimal dynamic regret.
problem Online learning in non-stationary Markov decision processes.
method MetaCURL uses a meta-algorithm with multiple black-box algorithms and a sleeping expert framework.
result Achieves optimal dynamic regret without prior knowledge of MDP changes.
Algorithm minimizes control regret for non-stationary LQR systems.
problem Control of non-stationary LQR systems with unknown dynamics.
method Adaptive non-stationarity detection and OLS estimator with small bias.
result Achieves optimal dynamic regret of $ ilde{\mathcal{O}}\left(V_T^{2/5}T^{3/5}
ight)$ .
Paper introduces Decentralized Non-stationary Competing Bandits ( exttt{DNCB}) for dynamic matching markets.
problem Understanding dynamic two-sided matching markets with competing agents.
method Proposes a decentralized asynchronous learning algorithm ( exttt{DNCB}) for non-stationary environments.
result Obtains sub-linear (logarithmic) regret of exttt{DNCB} in dynamic settings.
New algorithm for non-stationary bandits with slow drifts.
problem Minimizing dynamic regret in non-stationary bandits with slowly varying rewards.
method Extends Successive Elimination to non-stationary bandits with a novel gap profile characterization.
result First instance-dependent regret upper bound for slowly varying non-stationary bandits.
A new fuzzy time series method for non-stationary data.
problem Forecasting in non-stationary environments with concept drift.
method Non-Stationary Fuzzy Time Series (NSFTS) with time-varying parameters.
result The method can adapt to dynamic changes in the stochastic process.
One of the primary aspects of sustainable development involves accurate understanding and modeling of environmental phenomena. Many of these phenomena exhibit variations in both space and time and it is imperative to develop a deeper understanding of techniques that can model space-time dynamics accurately. In this pap…
New algorithms minimize dynamic regret in non-stationary online learning.
problem Universal dynamic regret minimization under exp-concave and smooth losses.
method Strongly Adaptive algorithms with a path variational based on second order differences of the comparator sequence.
result Achieve a dynamic regret of i l d e O ( d 2 n 1 / 5 C n 2 / 5 ∨ d 2 ) ilde O(d^2 n^{1/5} C_n^{2/5} \vee d^2) i l d e O ( d 2 n 1/5 C n 2/5 ∨ d 2 ) , optimal modulo dependencies. New definitions of ESP for quantum reservoir computing handle non-stationary systems.
problem Traditional ESP does not apply to non-stationary systems.
method Introduce two new categories of ESP: non-stationary ESP and subset/subspace ESP.
result Demonstrates correspondence between non-stationary ESP and QRC with NARMA tasks.
A method based on wavelet transform and genetic programming is proposed for characterizing and modeling variations at multiple scales in non-stationary time series. The cyclic variations, extracted by wavelets and smoothened by cubic splines, are well captured by genetic programming in the form of dynamical equations. …
New algorithm tracks changes in infinite action space rewards.
problem Non-stationary Lipschitz bandits with infinite actions.
method Adaptive tracking of significant shifts using hierarchical discretization.
result Achieves minimax-optimal dynamic regret bound of O ~ ( i l d e L 1 / 3 T 2 / 3 ) \mathcal{\widetilde{O}}( ilde{L}^{1/3}T^{2/3}) O ( i l d e L 1/3 T 2/3 ) . We discuss the finite sample theoretical properties of online predictions in non-stationary time series under model misspecification. To analyze the theoretical predictive properties of statistical methods under this setting, we first define the Kullback-Leibler risk, in order to place the problem within a decision the…
New RL method tackles dynamic MDPs with evolving rewards and states.
problem Dynamic MDPs with evolving rewards and states.
method Sliding Window Upper-Confidence bound for Reinforcement Learning (SWUCRL2-CW) and Bandit-over-Reinforcement Learning (BORL).
result Achieves dynamic regret bound for non-stationary MDPs.
Paper proposes a hybrid model for financial time series prediction using sentiment analysis.
problem Challenges in forecasting in non-stationary, complex environments with heterogeneous data.
method Hybrid model combining GANs with NLP-based sentiment analysis.
result Hybrid model enhances robustness in non-stationary environments.
Classifies cosmological Finsler spacetimes, finding viable non-stationary models.
problem Locating viable non-stationary Finsler spacetimes in cosmology.
method Locally classified all possible cosmological homogeneous and isotropic Landsberg-type Finsler structures in 4-dimensions.
result Identified unique Finsler, non-Berwaldian Landsberg generalization of Friedmann-Lemaitre-Robertson-Walker geometry.
Active learning aims to reduce annotation cost by predicting which samples are useful for a human teacher to label. However it has become clear there is no best active learning algorithm. Inspired by various philosophies about what constitutes a good criteria, different algorithms perform well on different datasets. Th…
New algorithms reduce dynamic regret for convex and smooth functions in non-stationary environments.
problem Online convex optimization in non-stationary environments.
method Proposed novel online algorithms exploiting smoothness to reduce dynamic regret.
result Dynamic regret improved to O ( T ) \mathcal{O}(T) O ( T ) for convex and smooth functions. New algorithm reduces decision switching in dynamic environments.
problem Online learning with memory and non-stationary environments.
method Dynamic policy regret, novel ensemble approach, meta-base decomposition.
result Proves optimal dynamic policy regret for memory length, non-stationarity, and time horizon.
dLDS models neural dynamics as sparse combinations of simpler components.
problem Understanding complex neural dynamics at a population level.
method Proposes a decomposed dynamical system model trained through dictionary learning.
result Model efficiently captures and demix diverse neural dynamics.
New approach turns optimal stationary RL into non-stationary RL without prior knowledge.
problem Optimal RL in non-stationary environments without prior knowledge of non-stationarity.
method Black-box reduction of optimal stationary RL algorithms to non-stationary RL.
result Achieves optimal dynamic regret bounds in various RL settings.
Framework uses RL with dynamic embedding to outperform benchmarks in volatile markets.
problem Challenges in high-dimensional, non-stationary, and noisy market information.
method Dynamic embedding of market information using generative autoencoders and online meta-learning in a reinforcement learning framework.
result Framework outperforms common portfolio benchmarks and PTO approach during market stress.
New algorithm optimizes resource allocation in non-stationary networks.
problem Optimal resource allocation in non-stationary RMABs is computationally hard.
method Sliding-Window Online Whittle (SW-Whittle) policy for non-stationary transition kernels.
result Sub-linear dynamic regret achieved with unknown variation budget.
We present a novel approach for fully non-stationary Gaussian process regression (GPR), where all three key parameters -- noise variance, signal variance and lengthscale -- can be simultaneously input-dependent. We develop gradient-based inference methods to learn the unknown function and the non-stationary model param…
Most contextual bandit algorithms minimize regret against the best fixed policy, a questionable benchmark for non-stationary environments that are ubiquitous in applications. In this work, we develop several efficient contextual bandit algorithms for non-stationary environments by equipping existing methods for i.i.d. …
Adaptive smooth non-stationary bandits achieve optimal regret rates without knowing parameters.
problem Smooth non-stationary bandits with Hölder class rewards.
method Established optimal dynamic regret rate and adaptive algorithm.
result Optimal dynamic regret can be attained adaptively without knowing Hölder exponent and coefficient.
NVMDP framework tackles non-stationary MDPs with varying discount rates.
problem Challenges in non-stationary environments and infinite-horizon formulations for reinforcement learning.
method Introduces NVMDP framework that accommodates non-stationarity and varying discount rates.
result NVMDPs provide a flexible mechanism to shape optimal policies without altering state or action spaces.
Paper uses RL for market making, improving stability in non-stationary markets.
problem Optimizing market making strategies in non-stationary limit order book dynamics.
method Reinforcement Learning (Proximal-Policy Optimization) applied to a simulator.
result RL agent outperforms closed-form optimal solution in non-stationary markets.
This paper proposes a hierarchical feature extractor for non-stationary streaming time series based on the concept of switching observable Markov chain models. The slow time-scale non-stationary behaviors are considered to be a mixture of quasi-stationary fast time-scale segments that are exhibited by complex dynamical…
New algorithm reduces dynamic regret in non-stationary dueling bandits using a weighted Borda score.
problem Designing algorithms with low dynamic regret in non-stationary dueling bandits.
method Introducing a novel weighted Borda score framework to analyze the Condorcet problem and establish improved bounds.
result First optimal and adaptive dynamic regret upper bound of i l d e O ( i l d e L 1 / 3 K 1 / 3 T 2 / 3 ) ilde{O}( ilde{L}^{1/3} K^{1/3} T^{2/3} ) i l d e O ( i l d e L 1/3 K 1/3 T 2/3 ) . Paper introduces novel Bandit algorithms for non-stationary environments in finance.
problem Non-stationary reward distributions in financial markets.
method Introduces Adaptive Discounted Thompson Sampling (ADTS) and Combinatorial Adaptive Discounted Thompson Sampling (CADTS) for non-stationary environments in portfolio optimization.
result Bandit Networks improve portfolio optimization performance by 20% compared to classical models.
New algorithm tackles non-stationary RL with near-optimal regret bounds.
problem Model-free reinforcement learning in non-stationary Markov decision processes.
method Proposed RestartQ-UCB algorithm with Freedman-type bonus terms.
result Achieves near-optimal dynamic regret bound in non-stationary RL.
New pricing algorithm learns demand curves and optimizes prices in dynamic markets.
problem Dynamic pricing in markets with incomplete demand information and shifting conditions.
method Actor-Critic Information-Directed Pricing (ACIDP) using IDS algorithms and auditing procedures.
result ACIDP outperforms UCB and TS in market environment shifts.