OMLE combines optimism and MLE for efficient sequential decision making.
problem Efficiently solving sequential decision making problems, especially in partially observable settings.
method Combines optimism for exploration and maximum likelihood estimation for model learning.
result OMLE learns near-optimal policies for a wide range of sequential decision making problems.
We consider the problem of sequential learning from categorical observations bounded in [0,1]. We establish an ordering between the Dirichlet posterior over categorical outcomes and a Gaussian posterior under observations with N(0,1) noise. We establish that, conditioned upon identical data with at least two observatio…
We solve a broad class of sequential decision-making problems with partially observed states.
problem Sequential decision-making under uncertainty with partially observed states.
method Modeling as a partially observed Markov decision process (POMDP) and separating state and modulation process.
result The approach allows for specialized approximate solution procedures.
Recently, a lot of effort has been paid to the efficient computation of Kriging predictors when observations are assimilated sequentially. In particular, Kriging update formulae enabling significant computational savings were derived in Barnes and Watson (1992), Gao et al. (1996), and Emery (2009). Taking advantage of …
New distributed EnKF method for non-sequential assimilation of large datasets.
problem Computational intensity and order dependencies in traditional EnKF.
method Distributed computing for full model error covariance matrix.
result Non-sequential assimilation outperforms sequential in performance.
Describes state variables in sequential decision problems, linking them to Markovian and non-Markovian models.
problem Sequential decision problems, especially in active learning and POMDPs, where decisions affect what is observed and learned.
method Canonical framework and novel two-agent perspective of POMDPs, defining state variables to claim Markovian or non-Markovian models.
result Properly modeled sequential decision problems are Markovian, while real decision problems are often non-Markovian.
Proposes LDIDPs for efficient sequential data generation from latent dynamical models.
problem Challenges in generating high-fidelity sequential samples from latent dynamical models.
method Utilizes implicit diffusion processes to sample from latent dynamical processes.
result Demonstrates accurate learning of dynamics and efficient generation of high-quality sequential data.
Develops methods for inference after detecting a change in sequential data.
problem Inference after a detected change in sequential data.
method General framework for constructing confidence sets using only data up to a stopping time.
result First general method for sequential changepoint localization with theoretical guarantees.
Proposes a new RNN model for grouped sequential data with varying time intervals.
problem Implicitly models fixed time intervals between observations and lacks group-level effects.
method Mixed membership framework for RNN, learning group-level base parameter.
result Demonstrates dynamic topic modeling with evolving topic distributions over time.
Algorithm clusters items by sequentially selecting features, minimizing observations.
problem Clustering items based on bandit feedback with many features.
method Sequential Halving algorithm for feature selection.
result Accurate recovery of item partition with minimal observations.
The spectral energy distribution (SED) is a relatively easy way for astronomers to distinguish between different astronomical objects such as galaxies, black holes, and stellar objects. By comparing the observations from a source at different frequencies with template models, astronomers are able to infer the type of t…
New method for robust policy evaluation in offline reinforcement learning with sequentially exogenous unobserved confounders.
problem Offline reinforcement learning in domains with unobserved confounders.
method Orthogonalized robust fitted-Q-iteration with closed-form solutions and bias-correction.
result Effective in simulations and real-world data, improving robustness and computational ease.
Paper tests Markov assumption in sequential decision making.
problem Testing the Markov assumption in sequential decision making.
method Forward-Backward Learning procedure to test MA without assuming parametric forms.
result The proposed test plays a crucial role in identifying optimal policies in complex decision processes.
Solves high-dimensional observation learning for control models.
problem Learning dynamics from high-dimensional images is challenging.
method Proposes a Beta DVBF approach to handle latent and observable space discrepancies.
result Demonstrates improved model learning from high-dimensional observations.
PoPPy simplifies point process modeling and analysis.
problem Efficient modeling and analysis of sequential data.
method Flexible design and efficient learning of point process models.
result PoPPy enables large-scale point process analysis, simulation, and prediction.
Paper proposes semi-supervised learning with triplet Markov chains.
problem Lack of labels in training data.
method Variational Bayesian inference for semi-supervised learning.
result Derives semi-supervised algorithms for various sequential models.
New algorithm for clustering data streams with no substitutions.
problem Clustering sequential data with no point substitution.
method Careful risk estimation, linear bin division, multiscale center selection.
result First constant approximation algorithm for random-order k-median clustering.
PrefOpt simplifies optimization with human preference feedback.
problem Design and optimization guided by human preferences.
method Extends latent variable model for binary preferences to include equivalent preference observations.
result Simplified sequential optimization tasks incorporating human feedback.
NegPSpan extracts negative sequential patterns efficiently.
problem Mining negative sequential patterns in sequence datasets.
method PrefixSpan depth-first scheme with maxgap constraints.
result NegPSpan extracts meaningful negative sequential patterns.
New method optimizes decision-making in uncertain environments.
problem Optimal decision-making under partial observability.
method Nested sequential Monte Carlo algorithm for continuous POMDPs.
result Demonstrated effectiveness on continuous POMDP benchmarks.
Paper tackles distribution learning from indirect sequential samples.
problem Learning discrete distribution P X P_X P X from indirect and sequential samples. method Iterative algorithm choosing functions and combining samples.
result Asymptotically consistent estimation possible under certain conditions.
Develops methods for finding counterfactual explanations in sequential decision making.
problem Finding counterfactual explanations for sequential decision making processes.
method Formal characterization of sequential actions and states using Markov decision processes and Gumbel-Max structural causal model. Introduces a polynomial time algorithm based on dynamic programming.
result Algorithm finds optimal counterfactual explanations for sequential decision making.
Detects changes in classifier scores to identify shifts in class priors.
problem Label shift changes in classification data.
method Sequential changepoint detection of classifier scores.
result Outperforms other detection procedures in label shift settings.
A new classifier updates sequentially using maximum margin principles.
problem Sequential data collection and partial labeling.
method Maximum margin classifier with Maximum Entropy Discrimination principle, kernel representation, and regularization.
result Improved performance compared to non-sequential classifiers.
A new particle filter uses diffusion models to improve state estimation from noisy data.
problem Sequentially estimating the state of a dynamical system from noisy and incomplete observations.
method Uses a diffusion model to simulate and predict system dynamics, incorporating noisy observations to refine predicted states.
result An unbiased particle filtering method that rigorously fuses observational data with diffusion model simulations.
Optimal unimodal fitting for linear loss functions in a sequential, efficient manner.
problem Optimal unimodal transformation of univariate model scores under linear loss functions.
method Proposes a sequential approach to estimate the optimal rectangular fit for observed samples with each new sample.
result Sequential approach achieves optimal efficiency with logarithmic time complexity per iteration.
New algorithms extract low-dimensional representations from sequential data, revealing insights into complex processes.
problem Challenges in extracting low-dimensional representations from sequential, high-dimensional, sparse, and noisy data.
method Developed new clustering algorithms based on Block Markov Chains theory, validated on real-world data.
result These algorithms can successfully extract low-dimensional representations from real-world sequential data, revealing insights into complex processes.
SDIFT generates full-field dynamics from sparse, irregular data.
problem Modeling and reconstructing physical dynamics from sparse, off-grid observations.
method SDIFT uses a functional Tucker model and sequential diffusion for generating full-field evolution from irregular sparse observations.
result Significant improvements in reconstruction accuracy and computational efficiency compared to state-of-the-art approaches.
Algorithm optimizes quantized isotonic regression with log-linear time updates.
problem Optimizing quantized isotonic regression estimations.
method Modified PAVA algorithm for sequential optimization.
result Log-linear time updates for optimal quantized mapping.
New method estimates treatment effects over time with unobserved confounders.
problem Estimating treatment effects from observational data with unobserved confounders.
method Sequential Deconfounder using Gaussian process latent variable model.
result Unbiased estimates of individualized treatment responses over time.
An efficient algorithm for k-median clustering in a sequential setting without substitutions.
problem Clustering a sequence of examples without being able to substitute centers later.
method An efficient algorithm with a multiplicative approximation factor of twice the offline algorithm's factor, and an optimal offline algorithm.
result The efficient algorithm achieves a good approximation of the optimal offline solution.
A new model for sequential memory using temporal predictive coding.
problem Forming accurate memory of sequential stimuli in the brain.
method Proposes a novel PC-based model called temporal predictive coding (tPC).
result Shows that tPC models can accurately memorize and retrieve sequential inputs.
AUCRSS detects change points in partially observed multivariate autocorrelated data.
problem Detecting change points in multivariate autocorrelated data with limited sensing resources.
method Adaptive Upper Confidence Region (AUCRSS) with state space model (SSM), adaptive sampling policy, and generalized likelihood ratio test.
result The method outperforms existing approaches in detecting change points efficiently.
Reduces change detection to estimation using confidence sequences.
problem Detecting changes in data streams with minimal delay and false alarms.
method Reduction from sequential change detection to sequential estimation using confidence sequences.
result Change detection scheme with minimal structural assumptions and strong guarantees.
Sequential tests for two-sample and independence testing using betting strategies.
problem Testing sequential data for two-sample and independence without kernel selection issues.
method Prediction-based betting strategies that adaptively determine distribution and joint distribution.
result Prediction-based tests outperform kernel-based approaches in high-dimensional or structured data settings.
DAIS improves AIS by resampling, avoiding gradient issues.
problem Low effective sample size in DAIS.
method DAIS with resampling step to improve efficiency.
result Resampling step avoids gradient variance issues.
A new method for Bayesian inference using diffusion models.
problem Bayesian inference in simulator-based models.
method Score-based diffusion models trained with a sequential training procedure.
result Comparable or superior performance compared to existing methods.
Composite likelihood inference of fractional Gaussian processes with sequentially optimal subset selection
problem Estimating parameters in time series
method Composite likelihood method
result The method reduces computational cost
The paper reviews methods for testing randomness and exchangeability in sequential data.
problem Testing the randomness and exchangeability of sequential data.
method Conformal martingales for online testing of randomness and exchangeability.
result Valid and efficiency results for various conformal martingale-based procedures.
New insights on active sequential prediction for mean estimation.
problem Active sequential prediction-powered mean estimation problem.
method Combining uncertainty-based suggestion with a constant probability, analyzing non-asymptotic bounds, and using no-regret learning.
result The optimal query probability is close to the constraint when using no-regret learning.
Extends GP models for sequential data, scalable and robust.
problem Handling sequential input-output observations in multi-task settings.
method Variational inference with sparse approximations and recursive GP priors.
result Tractable continual learning with KL divergences and recursive reconstruction.
New method improves generative model performance by fully conditioning variational posteriors.
problem Inaccurate inference due to partial conditioning of variational posteriors in sequential LVMs.
method Introduces fully-conditioned approximate posteriors to improve generative model performance.
result Improves generative modelling and multi-step prediction performance.
New framework improves robust inference in HMMs under model misspecification.
problem Inference in general state-space HMMs under likelihood misspecification.
method Generalized Bayesian Inference (GBI) and Sequential Monte Carlo (SMC) methods.
result Improved performance in object tracking and Gaussian process regression.
The paper tackles long-term treatment effects with persistent confounders using sequential short-term outcomes.
problem Estimating long-term treatment effects with persistent unmeasured confounders.
method Exploiting the sequential structure of short-term outcomes, the paper develops three novel identification strategies and corresponding estimators.
result The proposed methods outperform existing approaches in handling persistent confounders.
Investigates sequential problems on graph structures and large action spaces.
problem Sequential decision-making on graph structures and large action spaces.
method Spectral bandits, side observations, influence maximization, kernel bandits, polymatroid bandits, function optimization, infinitely many-arms bandits.
result Contributions to graph and structured bandits.
Study optimal policy regret in partially observable Markov games with adaptive opponents.
problem Optimal sequential decision-making in partially observable environments against strategic, adaptive opponents.
method An epoch-based optimistic maximum-likelihood algorithm that selects one policy per epoch using confidence sets built cumulatively from past data.
result Achieves i l d e O ( T ) ilde{O}(\sqrt{T}) i l d e O ( T ) policy regret for fixed problem parameters, with explicit dependence on horizon, adversary memory, confidence radius, and aggregate Eluder dimension. New bounds for sequential linear regression with square loss, uniformly over all features.
problem Uniform regret bounds for online linear regression with square loss over arbitrary deterministic sequences.
method Derives new regret bounds for online linear regression with square loss, using variants of online non-linear ridge regression.
result Optimal constant of 1 in front of the d B 2 ln T d B^2 \ln T d B 2 ln T term for known features, and asymptotic regret bound of d B 2 ln T d B^2 \ln T d B 2 ln T for sequentially revealed features. Estimates causal effects from a single time-series without further assumptions.
problem Estimating causal effects from a single time-series without additional assumptions.
method Proposes a general class of averages of conditional causal parameters, estimated using a targeted maximum likelihood estimator (TMLE).
result Asymptotic consistency and normality of the TMLE for estimating causal parameters.