The paper analyzes covariate shift in nonparametric regression with Markovian data.
problem Covariate shift in regression problems with Markovian data.
method Extension of nonparametric convergence rates to Markovian dependence structures, using Hölder smoothness assumptions and similarity measures.
result Precise convergence rates for Nadaraya-Watson kernel estimators under specific Markovian conditions.
Develops correlation number for specific potentials and Hitchin representations.
problem Analyzing correlation numbers for potentials with entropy gaps and Hitchin representations.
method Defines a correlation number for pairs of cusped Hitchin representations and explores its connection to the Manhattan curve.
result Establishes a connection between the correlation number and the Manhattan curve, revealing rigidity properties.
New framework for robust reinforcement learning policies in uncertain environments.
problem Robust reinforcement learning policies in environments with distributional shifts.
method Comprehensive modeling framework centered around robust Markov decision processes (RMDPs).
result Existence and conditions for the dynamic programming principle (DPP) in RMDPs.
Paper addresses uncertainty in model generalization under regime shifts.
problem Uncertainty in model generalization under regime changes.
method Proposes a framework to quantify and separate regime mismatch and sensitivity.
result Obtains exact decomposition and minimax lower bound for regime-aware models.
Study shows how to count and equidistribute cusped Hitchin representations with entropy gaps.
problem Counting and equidistribution of cusped Hitchin representations.
method Renewal theorem of Kesseböhmer and Kombrink applied to count and equidistribute.
result Entropy gaps at infinity allow for counting and equidistribution results.
The paper tackles matching a desired mean in causal systems through shift interventions.
problem Matching a desired mean in causal systems.
method Defining Markov equivalence classes, proposing active learning strategies, deriving lower bounds.
result Proposed active learning strategies require fewer interventions than previous approaches, especially for certain graph classes.
Robust OPE framework uses human inputs to improve policy evaluation in changing environments.
problem Inaccurate policy evaluations due to shifts in environment properties.
method Adapts OPE methods to shifts on user-inputted covariates, providing more realistic utility estimates.
result Robust OPE framework yields less pessimistic policy evaluations and captures realistic dataset shifts.
Study optimal offline RL with uncertainty sets and distribution shifts.
problem Optimal offline reinforcement learning with limited data.
method Construct uncertainty sets and distribution shifts, solve robust Markov decision process.
result Least conservative estimator for unknown true distribution.
Study OOD generalization in meta-reinforcement learning using information theory.
problem Understanding how meta-reinforcement learning handles distribution shifts.
method Information-theoretic analysis of Markov Decision Processes and gradient-based algorithms.
result Established fine-grained generalization bounds for meta-reinforcement learning.
Researchers adaptively analyze market regimes to reveal investor behavior shifts.
problem Market relationships shift across different regimes, affecting investor behavior.
method Combining Kalman filtering, Markov-switching, and asymmetric response estimation.
result Foreign investors' predictive power increases during crises, while individual investors react more strongly to positive shocks.
Proposes an EM algorithm for high-dimensional Markov-switching VAR models.
problem Estimating regime shifts in high-dimensional time series data.
method Approximate EM algorithm for Markov-switching VAR models.
result Established consistency of the proposed EM algorithm in high dimensions.
New algorithms learn robust policies from shifted distributions.
problem Learning robust policies in environments with distributional shifts.
method Two novel model-free algorithms: distributionally robust Q-learning and variance-reduced distributionally robust Q-learning.
result Achieves minimax sample complexity upper bound of ildeO(∣S∣∣A∣(1−γ)−4ε−2). Bayesian model explains and improves black-box estimators for class distribution.
problem Calibrating probabilistic classifiers and uncertainty quantification for unlabeled data.
method Introduced a Bayesian model approximating the ground-truth generative process, using efficient MCMC sampling.
result The Bayesian model is competitive and sometimes superior to established point estimators.
Proposes a new approach for domain adaptation using latent representations.
problem Handling distribution shifts between source and target domains in high-dimensional data.
method Learn compact latent representations based on the label's Markov blanket, partitioning into parents, children, and spouses.
result General domain adaptation can be achieved by learning representations of the label's parents, children, and spouses.
Unified framework for error quantification in off-policy evaluation with distributional shift.
problem Establishing high-confidence CI for target policy value from offline data.
method Unified error analysis quantifying misspecification and sampling errors.
result Achieves tightest possible CI and robustness against distributional shifts.
Optimizes mobile notifications for multiple objectives using reinforcement learning.
problem Optimizing mobile notification systems for multiple objectives.
method End-to-end offline reinforcement learning with Double Deep Q-network and Conservative Q-learning.
result Demonstrates improved performance and benefits of the proposed approach.
Neural networks parameterize time-varying Markov dynamics in financial time series.
problem Estimating Markov transition matrices in high-resolution, high-noise financial data.
method Introduces a neural network framework to generate explicit, time-varying Markov transition matrices, constraining neural outputs to formal stochastic operators.
result Learned operators capture regime shifts, with high-volatility regimes homogenizing transition dynamics.
Paper proposes methods to improve graph domain adaptation by decorrelating node features.
problem Challenges in transferring knowledge from one graph to another.
method Proposes decorrelating node features using GCN and graph transformer layers.
result Significant performance enhancements and clear visualizations of learned representations.
We develop efficient and sharp bounds on policy value under perturbations in MDPs.
problem Evaluating policies under best- and worst-case perturbations in MDPs with transition observations.
method Proposed a perturbation model for MDPs, developed semiparametrically efficient estimator with asymptotic normality.
result Semiparametrically efficient and asymptotically normal estimator for policy value bounds.
Algorithm learns from offline data to improve performance in target environment.
problem Learning from offline data in a target environment with unknown shifts.
method Adaptive algorithm that uses offline data to improve performance when informative.
result Algorithm provably improves performance over purely online learning when offline data are informative.
Algorithm learns robust equilibrium in online Markov games with interactive data.
problem Sim-to-real gap in reinforcement learning.
method Distributionally robust RL with minimum value assumption, least square value iteration.
result Sample-efficient algorithm for robust equilibrium in online Markov games.
SCTL scales causal domain adaptation without prior knowledge.
problem Domain adaptation with covariate shift and invariances across domains.
method SCTL: scalable causal discovery algorithm based on Markov blanket.
result SCTL achieves scalable and robust domain adaptation.
Novel CMG framework improves financial sentiment forecasting.
problem Challenges in short-term sentiment forecasting of financial OHLC data.
method Integrates chaos theory, Markov chains, and Gaussian processes with transformer models.
result Consistently outperforms traditional models in accuracy and efficiency.
CtrlNS learns latent factors and distribution shifts from sparse transitions without prior knowledge.
problem Lack of prior knowledge of domain variables limits causal temporal representation learning.
method Sparse transition assumption and identifiability results from theoretical perspective.
result Effective in identifying distribution shifts and latent factors without prior knowledge.
Pessimistic model-based algorithm finds Nash equilibria in zero-sum Markov games from offline data.
problem Learning Nash equilibria in two-player zero-sum Markov games from limited data.
method Pessimistic model-based algorithm with Bernstein-style lower confidence bounds (VI-LCB-Game).
result Proves sample complexity no larger than (1−γ)3ε2Cclipped⋆S(A+B), achieving minimax optimality. CapOptix uses options theory to price capacity in electricity markets.
problem Traditional capacity market designs fail to account for risk and price shocks.
method Interprets capacity commitments as reliability options and uses Markov Regime Switching Process.
result CapOptix provides more accurate pricing of capacity premia compared to existing mechanisms.
Improves robust transfer learning with side information.
problem Addressing environmental shift in MDPs with side information.
method Estimate-centered uncertainty sets with side information integration.
result Improved robust policy with reduced sub-optimality gap.
Pessimistic Minimax Value Iteration finds efficient NE policies from offline data.
problem Finding an approximate Nash equilibrium in offline Markov games with non-uniform coverage.
method Pessimistic Minimax Value Iteration (PMVI) constructs pessimistic value function estimates and solves NEs.
result Established a nearly minimax optimal result for offline Markov games with function approximation.
New criterion improves predictive evaluation in weighted inference scenarios.
problem Improving predictive evaluation in scenarios with different likelihoods for estimation and evaluation.
method Developed the posterior covariance information criterion (PCIC) to handle weighted likelihood inference.
result PCIC is asymptotically unbiased for quasi-Bayesian generalization error in weighted inference.
SwISS improves scalability of Bayesian inference for large datasets.
problem Scalability issues in Bayesian inference for large datasets.
method Divide-and-conquer approach with SwISS for recombining sub-posterior samples.
result SwISS accurately approximates the original posterior distribution.
VA-OPE improves OPE by incorporating variance information, achieving tighter error bounds.
problem Estimating value function of a target policy from offline data collected by a behavior policy.
method Proposes VA-OPE, an algorithm that reweights Bellman residual using estimated variance of the value function.
result Achieves a tighter error bound than the best-known result.
Polynomial mixing times for simulated tempering in mixture sampling problems.
problem Sampling from mixtures of log-concave distributions with location shifts.
method Conductance decomposition applied to an auxiliary Markov chain on an augmented space.
result First polynomial-time guarantee for simulated tempering with MALA.
Improved reinforcement learning for environments with distributional shifts.
problem Learning optimal policies in environments with distributional shifts.
method Distributionally robust Q-learning with multi-level Monte Carlo estimator.
result Proved upper bound on sample complexity for robust RL.
Bayesian framework improves trading robustness against market shifts.
problem Insufficient robustness and overfitting in trading models.
method Bayesian Robust Framework integrating macro-conditioned GAN and adversarial learning.
result Framework outperforms state-of-the-art models in diverse financial instruments.
DAC-SSM learns domain-agnostic states for better imitation learning.
problem Domain shifts hinder imitation learning in partially observable tasks.
method DAC-SSM uses adversarial training to remove domain-dependent information from states.
result DAC-SSM achieves comparable performance to experts in sparse reward tasks.
Learning the undirected graph structure of a Markov network from data is a problem that has received a lot of attention during the last few decades. As a result of the general applicability of the model class, a myriad of methods have been developed in parallel in several research fields. Recently, as the size of the c…
A scalable Bayesian additive model for stellar flare detection using Gaussian process inference and hidden Markov models.
problem Bayesian time-series modeling for astronomical datasets
method Generative surrogate framework with Variational Autoencoder and neural network forward pass
result Significant reduction in computational time for stellar flare detection
A HMM for intraday momentum trading reduces lagging and incorporates side information.
problem Time-lagging in existing momentum trading models leads to incorrect momentum signals.
method State space formulation with latent momentum states, cross-validation for state estimation, and Bayesian inference for prediction.
result The model reduces lagging and accurately predicts market changes.
Proposes PRMs for interpreting financial risk concept drift.
problem Concept drift in high-stakes predictions like credit risk.
method Probabilistic Rule Models (PRMs) using Markov Logic Networks.
result Interpretable rules explain borrower risk changes.
Control charts have traditionally been used in industrial statistics, but are constantly seeing new areas of application, especially in the age of Industry 4.0. This paper introduces a new method, which is suitable for applications in the healthcare sector, especially for monitoring a health-characteristic of a patient…
Paper introduces a new method for improving reinforcement learning performance using transfer learning.
problem Improving reinforcement learning performance with limited sample sizes in dynamic decision-making scenarios.
method Developed a novel ``re-weighted targeting procedure'' and ``transfer deep Q∗-learning'' approach. result Demonstrated improved reinforcement learning performance through strategic sample construction.
We investigate the problem of optimal dividend distribution for a company in the presence of regime shifts. We consider a company whose cumulative net revenues evolve as a Brownian motion with positive drift that is modulated by a finite state Markov chain, and model the discount rate as a deterministic function of the…
DR-RPO optimizes robust policies in RL with limited interaction, achieving sublinear regret.
problem Policy optimization in RL under distribution shift and adversarial dynamics.
method DR-RPO algorithm incorporating reference-policy regularization and upper confidence bonus for exploration.
result DR-RPO achieves sublinear regret and polynomial suboptimality bounds in robust RL.
Policy gradient methods are among the most effective methods in challenging reinforcement learning problems with large state and/or action spaces. However, little is known about even their most basic theoretical convergence properties, including: if and how fast they converge to a globally optimal solution or how they …
Symbolic dynamics for flows in high dimensions, extending previous work.
problem Coding flows with positive speed in high dimensions.
method Construct symbolic dynamics for flows with positive speed in any dimension.
result Extended symbolic dynamics to flows in high dimensions, including homoclinic classes.
In this paper, we formulate a new local move on virtual knot diagram, called arc shift move. Further, we extend it to another local move called region arc shift defined on a region of a virtual knot diagram. We establish that these arc shift and region arc shift moves are unknotting operations by showing that any virtu…
Proposes DAM for optimizing discrete generative models.
problem Challenges in optimizing discrete generative models.
method Discrete Adjoint Matching (DAM) for discrete state spaces.
result Demonstrates effectiveness on synthetic and mathematical reasoning tasks.
New RL method tackles sim-to-real gap using interactive data collection.
problem Sim-to-real gap in reinforcement learning.
method Distributionally robust reinforcement learning with interactive data collection.
result Proves sample-efficient learning is impossible without additional assumptions.