CtrlNS learns latent factors and distribution shifts from sparse transitions without prior knowledge.
problem Lack of prior knowledge of domain variables limits causal temporal representation learning.
method Sparse transition assumption and identifiability results from theoretical perspective.
result Effective in identifying distribution shifts and latent factors without prior knowledge.
End-to-end probabilistic inference improves audio signal processing.
problem Efficiently processing large audio signals with varying characteristics.
method Formulated a spectral mixture Gaussian process model with nonstationary priors, enabling infinite-horizon Gaussian process regression.
result The method outperforms standard techniques in processing audio signals with hundreds of thousands of data points.
Optimizes spectral density estimation for stationary and nonstationary processes.
problem Estimating spectral density of time series with complex structure.
method Optimally adaptive Bayesian spectral density estimation using smoothing spline covariance structure.
result Optimal eigendecomposition provides superior performance compared to alternative covariance functions.
Bayesian optimization improves with nonstationary covariance functions.
problem Stationary covariance functions fail to capture prior information in high dimensions.
method Proposes nonstationary covariance functions to encode prior information and adaptively promote local exploration.
result Nonstationary covariance functions increase sample efficiency in high dimensions.
Bayesian optimization has shown to be a fundamental global optimization algorithm in many applications: ranging from automatic machine learning, robotics, reinforcement learning, experimental design, simulations, etc. The most popular and effective Bayesian optimization relies on a surrogate model in the form of a Gaus…
New algorithm for nonstationary GLBs reduces computation and memory costs.
problem Nonstationary generalized linear bandits with unknown time-varying parameters.
method Discounted Online Mirror Descent (DOMD) for parameter estimation.
result Dynamic regret bounds of order O(1) per round in drifting and piecewise-stationary environments. Gaussian processes (GPs) provide a nonparametric representation of functions. However, classical GP inference suffers from high computational cost and it is difficult to design nonstationary GP priors in practice. In this paper, we propose a sparse Gaussian process model, EigenGP, based on the Karhunen-Loeve (KL) expan…
CBGP boosts GP covariance to model spatiotemporal irregularities.
problem Overfitting and overconfident uncertainty in nonstationary GP models.
method Boosting covariance priors, partially-whitened observations, gradient descent-like procedure.
result Accurate and reliable SBAS ionospheric corrections in challenging space weather.
Flexible nonstationary Gaussian process with neural network parameters.
problem Limited expressiveness of stationary Gaussian processes.
method Nonstationary kernels with neural network parameters trained jointly.
result Better accuracy and log-score compared to stationary and hierarchical models.
Bayesian optimization has become a fundamental global optimization algorithm in many problems where sample efficiency is of paramount importance. Recently, there has been proposed a large number of new applications in fields such as robotics, machine learning, experimental design, simulation, etc. In this paper, we foc…
Unified formulation bridges adversarial and nonstationary bandits.
problem Handling time-varying reward distributions in multi-armed bandit problems.
method Unified oracle that switches between adversarial and nonstationary bandit oracles based on window size.
result Optimal regret achieved with matching lower bound.
Paper analyzes algorithms for nonstationary saddle-point optimization problems.
problem Nonstationary saddle-point optimization problems in game theory, reinforcement learning, and machine learning.
method Proposes extragradient and Frank-Wolfe algorithms for online and bandit settings.
result Establishes sub-linear regret bounds for the proposed algorithms.
A new approach learns to represent context for nonstationary bandits.
problem Nonstationary contextual bandits where patterns change over time.
method Combines recurrent neural networks with contextual linear bandit algorithm.
result Consistently outperforms handcrafted historical contexts and other methods.
The analysis of nonstationary time series is of great importance in many scientific fields such as physics and neuroscience. In recent years, Gaussian process regression has attracted substantial attention as a robust and powerful method for analyzing time series. In this paper, we introduce a new framework for analyzi…
Model nonstationary spatial processes using normalizing flows.
problem Difficult selection of spatial warping functions.
method Neural autoregressive flows (NAFs) for complex, high-dimensional warpings.
result NAFs model has greater representational capacity than other spatial process models.
New algorithm for nonstationary multi-armed bandits with optimal performance.
problem Nonstationary multi-armed bandits with changing model parameters over time.
method Adaptive Resetting Bandit (ADR-bandit) algorithm using adaptive windowing techniques.
result ADR-bandit achieves nearly optimal performance in both abrupt and gradual changes.
ConvNets improve nonstationary covariance estimation for large-scale spatial data.
problem Estimating nonstationary spatial covariance functions on large scales.
method Convolutional Neural Networks (ConvNets) for subregion identification and selection.
result Enhanced accuracy in parameter estimation using ConvNet-based partitioning.
Bayesian topological learning improves EEG signal analysis for brain state classification.
problem Challenges in classifying and analyzing noisy, nonlinear, nonstationary EEG signals.
method Persistent homology with Bayesian framework to track topological features and incorporate prior knowledge.
result Bayesian topological learning outperforms existing methods for noisy EEG classification.
The paper tackles causal discovery and forecasting in nonstationary environments using state-space models.
problem Challenges in identifying causal relations and forecasting in nonstationary time series.
method Exploiting a particular type of state-space model to represent nonstationary processes, allowing changes in causal strengths and noise variances.
result Nonstationarity helps identify causal structure and improves forecasting.
SORSCNs improve nonstationary data modeling by self-organizing and adjusting network parameters.
problem Nonstationary data challenges traditional models in continuous learning.
method SORSCNs autonomously adjust network parameters and structure in real-time using adaptive algorithms.
result SORSCNs outperform other models in generalizing to nonstationary data.
Paper develops bandit algorithms for nonstationary nonconvex optimization.
problem Nonstationary online nonconvex optimization problems.
method Proposes and analyzes bandit algorithms for nonconvex functions with nonstationary regret.
result Develops bandit versions of Newton's method for nonstationary nonconvex optimization.
We propose a method to clean covariance matrices of nonstationary systems by using time-independent eigenvalues.
problem Noise in covariance matrices of nonstationary systems with time-independent eigenvalues.
method Data-driven approach to use independent eigenvalues encoding long-term influence of future on present.
result Our method outperforms optimal stationary methods for filtering covariance matrix and its inverse.
We use diffusion models to sample from complex GP priors in climate data.
problem Sampling from non-stationary Gaussian process priors is computationally hard.
method Replace GP prior with a diffusion model surrogate and use training-free guidance algorithms.
result Generated distributions are close to GP priors and can be fine-tuned.
New method identifies nonstationary causal structures in time series data.
problem Identifying causal relationships in time series data that change over time.
method High-order Markov Switching Models for regime-dependent causal discovery.
result Scalable approach for estimating high-order regime-dependent causal structures.
New method for identifying causal relationships in financial time series data.
problem Identifying causal relationships in nonstationary financial time series data.
method Refined constraint-based causal discovery algorithm (CD-NOTS) for nonstationary time series data.
result CD-NOTS effectively identifies causal connections in financial applications.
SyMPLER improves time series forecasting in nonstationary environments with explainable models.
problem Nonstationary time series forecasting with limited interpretability.
method Dynamic piecewise-linear approximations based on Statistical Learning Theory generalization bounds.
result SyMPLER achieves comparable performance to black-box and explainable models while maintaining interpretability.
This paper presents a novel approach for approximate integration over the uncertainty of noise and signal variances in Gaussian process (GP) regression. Our efficient and straightforward approach can also be applied to integration over input dependent noise variance (heteroscedasticity) and input dependent signal varia…
Markovian RNN adapts to nonstationary data using HMM for better time series prediction.
problem Nonstationary sequential data in real-life applications.
method Markovian RNN with HMM for regime switching and end-to-end optimization.
result Significant performance gains over vanilla RNN and Markov Switching ARIMA.
AREBA algorithm improves learning from imbalanced, nonstationary data.
problem Learning from imbalanced, nonstationary data in online settings.
method Adaptive REBAlancing (AREBA) algorithm that selectively includes examples to maintain class balance.
result AREBA significantly outperforms other algorithms in learning speed and quality.
BerlinUCB learns from episodic rewards in nonstationary contexts.
problem Online learning with episodic rewards in nonstationary environments.
method BerlinUCB integrates clustering for self-supervision.
result BerlinUCB outperforms standard contextual bandits in various scenarios.
Gaussian processes retain the linear model either as a special case, or in the limit. We show how this relationship can be exploited when the data are at least partially linear. However from the perspective of the Bayesian posterior, the Gaussian processes which encode the linear model either have probability of nearly…
A novel nonstationary permanental process relaxes kernel constraints and captures complex data patterns.
problem Limitations of existing permanental processes in terms of kernel types and stationarity.
method Sparse spectral representation of nonstationary kernels and hierarchical stacking of spectral feature mappings.
result Enhanced model expressiveness and reduced computational complexity.
A new estimator corrects bias in high-dimensional predictive regressions.
problem Bias in high-dimensional predictive regressions.
method IVX-desparsified LASSO (XDlasso) estimator.
result Corrects both shrinkage and Stambaugh bias.
Most real world phenomena such as sunlight distribution under a forest canopy, minerals concentration, stock valuation, exhibit nonstationary dynamics i.e. phenomenon variation changes depending on the locality. Nonstationary dynamics pose both theoretical and practical challenges to statistical machine learning algori…
LEAP identifies latent causal variables from temporal data.
problem Recovering time-delayed latent causal variables from general temporal data.
method Proposes LEAP, a framework that extends VAEs with constraints for temporally causal latent processes.
result Successfully identifies temporally causal latent processes from observed variables under various dependency structures.
New method infers causal relationships from nonstationary time series data.
problem Challenges in inferring causal relationships from nonstationary time series data.
method Proposes a new class of restricted SCM with time-varying filters and stationary noise, leveraging asymmetry from nonstationarity.
result Demonstrates effectiveness of the proposed methodology on various synthetic and real datasets.
Develops nonstationary MOGP kernels for better performance.
problem Limited applicability of existing MOGP kernels for nonstationary data.
method Harmonizable spectral mixture kernels for nonstationary MOGP.
result Automatic identification of nonstationary behavior in data.
New RL algorithm tackles nonstationary MDPs with linear approximations and varying rewards.
problem Nonstationary reinforcement learning with evolving reward and state transition functions.
method Developed a new algorithm LSVI-UCB-Restart with periodic restart, and parameter-free Ada-LSVI-UCB-Restart for unknown variation budgets.
result First minimax dynamic regret lower bound for nonstationary linear MDPs and linear MDPs lower bound.
New method tests independence with single nonstationary time series.
problem Testing independence in nonstationary nonlinear time series.
method Time-varying nonlinear regression, local long-run covariance estimation, strong Gaussian approximation.
result First framework for conditional independence testing with a single realization of a nonstationary nonlinear process.
A new method warps inputs to learn nonstationary kernels efficiently.
problem Learning nonstationary patterns in data with varying smoothness.
method Sparse spectrum Gaussian processes with input warping as conditional Gaussian measures.
result Efficient learning of nonstationary patterns with fewer parameters.
Most environmental phenomena, such as wind profiles, ozone concentration and sunlight distribution under a forest canopy, exhibit nonstationary dynamics i.e. phenomenon variation change depending on the location and time of occurrence. Non-stationary dynamics pose both theoretical and practical challenges to statistica…
Proposes Genetic Thompson Sampling for multi-armed bandits, improving performance in nonstationary settings.
problem Improving sequential decision making tasks of online learning agents using multi-armed bandits.
method Integrates genetic principles into Thompson Sampling for multi-armed bandits.
result Significantly outperforms baselines in nonstationary settings.
The paper provides bounds for regression schemes using nonstationary training samples.
problem Developing confidence intervals for nonparametric regression with nonstationary data.
method The approach involves Rademacher and Vapnik-Chervonenkis theories to analyze the cost and optimality of regression schemes.
result The paper establishes nonasymptotic bounds for regression schemes and optimality in L2-distance. Develops a new method to discover causal relationships from nonstationary time series data.
problem Challenges in inferring causal relationships from observational data, especially for nonstationary time series.
method State-Dependent Causal Inference (SDCI) for conditionally stationary time series.
result SDCI can recover underlying causal dependencies with provable identifiability for state-dependent causal structures.
We present online prediction methods for time series that let us explicitly handle nonstationary artifacts (e.g. trend and seasonality) present in most real time series. Specifically, we show that applying appropriate transformations to such time series before prediction can lead to improved theoretical and empirical p…
Study shows nonstationary bandits require T-dependent regret even with minimal nonstationarity.
problem Understanding satisficing regret in nonstationary multi-armed bandits.
method Developed a novel Fano-based framework for nonstationary bandits with a post-interaction reference construction.
result Optimal regret scales with T even with minimal nonstationarity, contrasting with stationary case.
DyEnsemble improves BCI accuracy by adapting to nonstationary neural signals.
problem Nonstationary neural signals in BCI cause decoding errors.
method Dynamic ensemble modeling that learns and combines diverse models online.
result DyEnsemble outperforms Kalman filters, especially with noisy signals.
A new method for nonstationary Gaussian processes using Fourier features.
problem Efficient simulation of nonstationary Gaussian processes with high-dimensional distributions.
method Discretizes the spectral representation of nonstationary processes, avoiding probability measure assumptions.
result An efficient low-rank approximation of nonstationary spectral densities, consistent and positive semi-definite.