Bayesian convolutional deep sets improve ambiguity in stationary process modeling.
problem Ambiguity in translation equivariant functional representations due to insufficient data points.
method Introduce Bayesian convolutional deep sets with task-dependent stationary prior.
result Improves representation quality compared to kernel smoother and non-parametric models.
Algorithm adapts to non-stationary rewards without prior knowledge.
problem Optimizing decisions in non-stationary environments without prior knowledge of changes.
method Optimization-based algorithm that restarts when non-stationarity is detected.
result Achieves tighter dynamic regret bound and is nearly minimax optimal.
New approach turns optimal stationary RL into non-stationary RL without prior knowledge.
problem Optimal RL in non-stationary environments without prior knowledge of non-stationarity.
method Black-box reduction of optimal stationary RL algorithms to non-stationary RL.
result Achieves optimal dynamic regret bounds in various RL settings.
We use diffusion models to sample from complex GP priors in climate data.
problem Sampling from non-stationary Gaussian process priors is computationally hard.
method Replace GP prior with a diffusion model surrogate and use training-free guidance algorithms.
result Generated distributions are close to GP priors and can be fine-tuned.
DS-TS adapts to abrupt and smooth changes in bandit problems.
problem Non-stationary multi-armed bandit problems with abrupt and smooth changes.
method Discounted Thompson Sampling with Gaussian priors.
result Achieves nearly optimal regret bound for both abrupt and smooth changes.
Master algorithm fails to detect non-stationarity in practical settings.
problem Non-Stationary Reinforcement Learning without prior knowledge.
method Master algorithm tested under various conditions, including piecewise stationary multi-armed bandits.
result Master's non-stationarity detection is ineffective for practical horizons, leading to performance similar to random restarting.
Generative models improve inverse problems by providing tailored priors.
problem Analyzing the error in inverse problems solved with generative priors.
method Quantitative error bounds for minimum Wasserstein-2 generative models.
result The error in the posterior due to the generative prior is bounded by the prior's error in Wasserstein-1 distance.
New algorithm reduces dynamic regret without prior function change knowledge.
problem Non-stationary stochastic optimization with bandit feedback.
method Fixed step sizes combined with multi-scale sampling framework.
result Achieves optimal dynamic regret without prior function change knowledge.
DAL enhances black-box bandit algorithms for non-stationary environments.
problem Non-stationary environments in bandit problems.
method DAL combines any stationary bandit algorithm with a change detector.
result DAL consistently outperforms state-of-the-art methods in various non-stationary scenarios.
This paper distills financial indicators into neural networks to reduce noise and improve accuracy.
problem Reduction of non-stationary noise in financial time series data.
method Co-distillation of smaller networks trained on indicators to transfer prior knowledge and reduce overfitting.
result The proposed method outperforms traditional methods in terms of speed and accuracy on real financial datasets.
New definition resolves ambiguity in non-stationary bandit classification.
problem Ambiguity in classifying non-stationary bandits using existing definitions.
method Introducing a formal definition that resolves ambiguity and provides a unified approach.
result Unified approach applicable to both Bayesian and frequentist formulations, resolves classification issues.
We introduce GLR-klUCB, a novel algorithm for the piecewise iid non-stationary bandit problem with bounded rewards. This algorithm combines an efficient bandit algorithm, kl-UCB, with an efficient, parameter-free, changepoint detector, the Bernoulli Generalized Likelihood Ratio Test, for which we provide new theoretica…
New Hida-Matérn kernels enable flexible process priors and efficient GP inference.
problem Flexible modeling of stationary processes with oscillatory components.
method Introducing a new class of covariance functions (Hida-Matérn kernels) and their state space representations.
result Efficient Gaussian Process inference and improved numerical stability.
New GP kernels avoid mean reversion without losing smoothness.
problem Pathological behavior in stationary GP regression.
method Improper Gaussian processes with non-positive kernels.
result Stationary, non-reverting covariance functions.
New method reduces dynamic regret for non-stationary bandits.
problem Non-stationary stochastic multi-armed bandit problem with changing optimal arm.
method Proposes a method achieving near-optimal dynamic regret without prior knowledge of changes.
result Achieves O ~ ( K N ( S + 1 ) ) \widetilde O(\sqrt{K N(S+1)}) O ( K N ( S + 1 ) ) dynamic regret. Bayesian nonparametric method segments multi-sequence time series data.
problem Temporal segmentation of multi-sequence time series data into stationary segments.
method Gaussian process priors and nonparametric distribution for segment partitioning.
result Model effectively segments synthetic and real-time series data.
Bayesian optimization improves with nonstationary covariance functions.
problem Stationary covariance functions fail to capture prior information in high dimensions.
method Proposes nonstationary covariance functions to encode prior information and adaptively promote local exploration.
result Nonstationary covariance functions increase sample efficiency in high dimensions.
New method tracks shifts in infinite-armed bandits without prior knowledge.
problem Tracking shifts in non-stationary infinite-armed bandits.
method Blackbox conversion of finite-armed MAB to infinite-armed non-stationary, randomized elimination.
result First parameter-free optimal regret bounds for all reservoir regularity regimes.
Vroom optimizes in unpredictable conditions without derivatives.
problem Optimizing in non-stationary, adversarial environments.
method Zeroth-order online learning with vanishing regret.
result Achieves favorable rates in stochastic settings.
Develops Gaussian processes on non-Euclidean spaces with symmetries.
problem Invariance to symmetries in non-Euclidean spaces.
method Constructive techniques for stationary Gaussian processes on compact and non-compact spaces.
result Makes non-Euclidean Gaussian processes compatible with standard software.
Develops Gaussian processes on non-compact Lie groups.
problem Invariance to symmetries in non-Euclidean spaces.
method Constructive techniques for stationary Gaussian processes.
result Makes non-Euclidean Gaussian processes compatible with standard software.
Optimizes spectral density estimation for stationary and nonstationary processes.
problem Estimating spectral density of time series with complex structure.
method Optimally adaptive Bayesian spectral density estimation using smoothing spline covariance structure.
result Optimal eigendecomposition provides superior performance compared to alternative covariance functions.
MetaCURL tackles non-stationary MDPs with optimal dynamic regret.
problem Online learning in non-stationary Markov decision processes.
method MetaCURL uses a meta-algorithm with multiple black-box algorithms and a sleeping expert framework.
result Achieves optimal dynamic regret without prior knowledge of MDP changes.
Novel Bayesian approach for non-stationary linear contextual bandits.
problem Non-stationary linear contextual bandits.
method Weighted Sequential Bayesian (WSB) inference.
result Established frequentist regret guarantees for new algorithms.
New algorithm tracks changes in infinite action space rewards.
problem Non-stationary Lipschitz bandits with infinite actions.
method Adaptive tracking of significant shifts using hierarchical discretization.
result Achieves minimax-optimal dynamic regret bound of O ~ ( i l d e L 1 / 3 T 2 / 3 ) \mathcal{\widetilde{O}}( ilde{L}^{1/3}T^{2/3}) O ( i l d e L 1/3 T 2/3 ) . DARLING tackles non-stationary RL with guarantees, improving dynamic regret.
problem Non-stationary reinforcement learning in unknown change points.
method Detection Augmented Reinforcement Learning (DARLING) for tabular and linear MDPs.
result DARLING matches minimax lower bounds in tabular and linear MDPs.
New RL algorithm tackles non-stationary environments with flexible policy updates.
problem Non-stationary reinforcement learning with time-varying rewards and transition probabilities.
method Model-free policy-based algorithm NS-NAC with restart-based exploration and dynamic learning rates.
result Dynamic regret of i l d e O ( ∣ S ∣ 1 / 2 ∣ A ∣ 1 / 2 Δ T 1 / 6 T 5 / 6 ) ilde{\mathscr O}(|S|^{1/2}|A|^{1/2}Δ_T^{1/6}T^{5/6}) i l d e O ( ∣ S ∣ 1/2 ∣ A ∣ 1/2 Δ T 1/6 T 5/6 ) for both algorithms. MAML optimizes shared priors for subtasks in a nonconvex meta-objective.
problem Understanding global optimality of MAML for nonconvex meta-objectives.
method Characterizes optimality gap of MAML stationary points via first-order optimization methods.
result Establishes global optimality of MAML for both RL and supervised learning.
The paper tackles lifelong learning in multi-armed bandits, aiming to minimize average regret over multiple tasks.
problem Minimizing average regret in multi-armed bandits over multiple tasks.
method Confidence interval tuning of UCB algorithms and greedy algorithms applied to a bandit over bandit approach.
result Empirical improvement over previous work in the mortal bandit problem.
ETGPSSM efficiently models high-dimensional, non-stationary systems with reduced complexity.
problem Prohibitive computational and parametric complexity in high-dimensional, non-stationary dynamical systems.
method ETGPSSM integrates a single shared GP with input-dependent normalizing flows for scalable and flexible modeling.
result ETGPSSM outperforms existing models in computational efficiency and accuracy.
One of the challenges in model-based control of stochastic dynamical systems is that the state transition dynamics are involved, and it is not easy or efficient to make good-quality predictions of the states. Moreover, there are not many representational models for the majority of autonomous systems, as it is not easy …
The deep image prior was recently introduced as a prior for natural images. It represents images as the output of a convolutional network with random inputs. For "inference", gradient descent is performed to adjust network parameters to make the output match observations. This approach yields good performance on a rang…
The Probably Approximately Correct (PAC) Bayes framework (McAllester, 1999) can incorporate knowledge about the learning algorithm and (data) distribution through the use of distribution-dependent priors, yielding tighter generalization bounds on data-dependent posteriors. Using this flexibility, however, is difficult,…
We consider the multi armed bandit problem in non-stationary environments. Based on the Bayesian method, we propose a variant of Thompson Sampling which can be used in both rested and restless bandit scenarios. Applying discounting to the parameters of prior distribution, we describe a way to systematically reduce the …
New features reduce computational cost of variational inference.
problem Efficiently compute ELBO with reduced computational cost.
method Developed features that reduce O ( M 3 ) O(M^3) O ( M 3 ) to O ( i l d e N T + M T ) O( ilde{N}T+MT) O ( i l d e N T + M T ) for large M M M . result Unbiased ELBO estimation with reduced computational complexity.
Proposes a flexible MGP model for dynamic, sparse correlations.
problem Handling dynamic and sparse correlations in multivariate data.
method Non-stationary MGP with dynamic spike-and-slab prior and EM algorithm.
result Captures dynamic and sparse correlations effectively.
New TS algorithms improve performance in non-stationary multi-armed bandit problems.
problem Sequential decision-making with evolving action rewards.
method Sliding-window Thompson sampling approaches with different priors.
result Unified regret upper bound for arbitrary non-stationary MABs.
New algorithms adaptively calibrate predictions in non-stationary environments, matching optimal rates.
problem Designing online prediction algorithms that adapt to varying levels of non-stationarity.
method Epoch-based scheduling and non-uniform partitioning of the prediction space.
result Achieves adaptive calibration guarantees under multiple measures with optimal rates.
This paper refines the weighted strategy for non-stationary parametric bandits, improving regret bounds.
problem Non-stationary environments with gradual drifting patterns.
method Refined analysis framework for the weighted strategy in linear and generalized linear bandits.
result A simpler weight-based algorithm with improved regret bounds compared to previous studies.
New algorithm tackles non-stationary RL with near-optimal regret bounds.
problem Model-free reinforcement learning in non-stationary Markov decision processes.
method Proposed RestartQ-UCB algorithm with Freedman-type bonus terms.
result Achieves near-optimal dynamic regret bound in non-stationary RL.
Proposes PE-GP-UCB for time-varying Bayesian optimisation.
problem Time-varying Gaussian process bandits with unknown prior.
method PE-GP-UCB algorithm, relying on consistency of function values with priors.
result Regret bound provided for the proposed algorithm.
A novel algorithm for best-arm identification in non-stationary linear bandits reduces error probability.
problem Non-stationary environments in A/B testing scenarios.
method Proposes a novel algorithm P 1 \mathsf{P1} P1 - R A G E \mathsf{RAGE} RAGE for robust best-arm identification. result Error probability decreases as exp ( − T Δ ( 1 ) 2 / d ) \exp(-TΔ^2_{(1)}/d) exp ( − T Δ ( 1 ) 2 / d ) , demonstrating robustness to non-stationarity. This paper introduces the Partition Tree Weighting technique, an efficient meta-algorithm for piecewise stationary sources. The technique works by performing Bayesian model averaging over a large class of possible partitions of the data into locally stationary segments. It uses a prior, closely related to the Context T…
Solves imaging inverse problems using a VAE prior and joint MAP optimization.
problem Solving ill-posed inverse problems in imaging.
method Joint Posterior Maximization with a VAE prior, using alternate optimization algorithms and stochastic encoding.
result Converges to high-quality solutions close to bi-convex, outperforming non-convex MAP approaches.
The goal of imitation learning is for an apprentice to learn how to behave in a stochastic environment by observing a mentor demonstrating the correct behavior. Accurate prior knowledge about the correct behavior can reduce the need for demonstrations from the mentor. We present a novel approach to encoding prior knowl…
This paper refines the weighted strategy for non-stationary parametric bandits and MDPs, improving regret bounds.
problem Non-stationary environments with gradual drifting patterns.
method Refined analysis framework for the weighted strategy, leading to simpler and more efficient algorithms.
result Improved regret bounds for linear bandits, generalized linear bandits, and self-concordant bandits.
Study on inventory control with changing demand, proposing adaptive algorithms.
problem Inventory control with non-stationary demand distributions.
method Adaptive online algorithms optimizing base-stock policies.
result Sharp separation in adaptability across different inventory models.
ELBO of VAEs converges to a sum of three entropies.
problem Understanding the convergence of ELBO in VAEs.
method Analytical derivation of ELBO convergence for standard Gaussian VAEs.
result ELBO converges to a sum of three entropies at stationary points.