Continuous-time optimal stopping solved with deep reinforcement learning
problem Optimal stopping problems in continuous time
method CARLOS (Continuous-time Adaptive Reinforcement Learning for Optimal Stopping)
result Higher prices than existing Bermudan solvers, approaching American upper bound
Existence of strong randomized equilibria in mean-field games with common noise.
problem Existence of strong solutions in mean-field games of optimal stopping.
method Connection with Bank-El Karoui's representation problem and continuity assumptions.
result Existence of strong randomized mean-field equilibrium under certain conditions.
This paper solves the best arm identification problem with both quick commitment and reward maximization.
problem Simultaneously identifying the best arm and minimizing regret in a stochastic Multi-Armed Bandit problem.
method Introduces Regret Optimal Best Arm Identification (ROBAI) and presents algorithms EOCP and its variants.
result Achieves asymptotic optimal regret and quick commitment to the optimal arm in both pre-determined and adaptive stopping times.
In this paper, we present a discrete-type approximation scheme to solve continuous-time optimal stopping problems based on fully non-Markovian continuous processes adapted to the Brownian motion filtration. The approximations satisfy suitable variational inequalities which allow us to construct ε-optimal stopping tim…
In this paper, we propose an adaptive stopping rule for kernel-based gradient descent (KGD) algorithms. We introduce the empirical effective dimension to quantify the increments of iterations in KGD and derive an implementable early stopping strategy. We analyze the performance of the adaptive stopping rule in the fram…
Optimal timing strategy for mean-reverting price spreads.
problem Trading price spreads with mean-reverting characteristics.
method Sequential optimal stopping framework with refined signature method.
result Precise entry and exit timings that maximize gains.
Study proposes a stopping criterion for active learning based on error stability.
problem Improving predictive performance in active learning by adaptively annotating samples.
method Proposes a stopping criterion based on error stability for Bayesian active learning.
result Demonstrates the proposed criterion stops active learning at the appropriate timing for various models and datasets.
This study analyzes AdaGrad's stability and convergence in non-convex optimization.
problem Lack of theoretical analysis for AdaGrad in non-convex optimization.
method Novel stopping time-based techniques from probability theory.
result Established stability and derived convergence rates for AdaGrad.
Study examines how slight model changes affect multi-period optimization outcomes.
problem Effect of small probabilistic model changes on multi-period optimization problems.
method Adapted Wasserstein distance for measuring changes, explicit first-order approximations proved.
result Explicit first-order approximations for multi-period stochastic optimization and optimal stopping problems.
New algorithm solves phase retrieval with adaptive stopping criteria.
problem Robust phase retrieval problem as nonsmooth, nonconvex optimization.
method Inexact proximal linear algorithm with adaptive stopping criteria.
result Proposed methods are more efficient than existing methods.
This paper studies the problem of optimal investment in incomplete markets, robust with respect to stopping times. We work on a Brownian motion framework and the stopping times are adapted to the Brownian filtration. Robustness can only be achieved for logartihmic utility, otherwise a cashflow should be added to the in…
The paper studies early stopping methods in linear contextual bandits.
problem Minimizing in-experiment regret and conducting robust post-experiment inferences in contextual bandits.
method The study proposes early stopping rules based on the Opportunity Cost and Threshold Method, using variances of estimators to quantify upper regret bounds.
result The proposed method provides a systematic approach to minimize in-experiment regret and conduct robust post-experiment inferences.
Early-stopped aggregation improves computational efficiency in adaptive statistical inference.
problem Computational inefficiency in model selection and aggregation for adaptive inference.
method Early-stopped aggregation (ESA) framework: compute only a few simpler models using early-stopping criteria.
result ESA achieves optimal adaptive contraction rates in variational Bayes and frequentist settings.
We propose a new approach to solve optimal stopping problems via simulation. Working within the backward dynamic programming/Snell envelope framework, we augment the methodology of Longstaff-Schwartz that focuses on approximating the stopping strategy. Namely, we introduce adaptive generation of the stochastic grids an…
A new sequential test for unnormalized densities.
problem Testing unnormalized densities with adaptive stopping.
method Sequential kernelized Stein discrepancy test, using non-uniform Stein kernels.
result Valid test with asymptotic lower bound for growth.
We propose a new algorithmic framework for sequential hypothesis testing with i.i.d. data, which includes A/B testing, nonparametric two-sample testing, and independence testing as special cases. It is novel in several ways: (a) it takes linear time and constant space to compute on the fly, (b) it has the same power gu…
RG-TTA adapts neural forecasters to streaming time series shifts by modulating adaptation intensity.
problem Adapting neural forecasters to distribution shifts in streaming time series data.
method RG-TTA uses a meta-controller that continuously modulates adaptation intensity based on distributional similarity.
result RG-TTA achieves the lowest MSE in 156 of 224 seed-averaged experiments, reducing MSE by 5.7% vs TTA.
Develops an anytime-valid framework for optimal policy identification from logged contextual bandit data.
problem Selecting the optimal policy from a candidate policy class while monitoring evidence continuously.
method Constructs a time-indexed set that retains the true optimal policy set uniformly over time.
result The procedure allows the analyst to monitor policy values, eliminate clearly suboptimal policies, and stop at data-dependent times without invalidating inference.
Solves optimal stopping problem with Poisson constraints using jumps.
problem Optimal stopping with Poisson constraints and jumps.
method Penalized backward stochastic differential equation (PBSDE) with jumps, decomposition method based on Jacod-Pham, comparison theorem of BSDEs with jumps.
result Solves American option pricing in nonlinear markets with Poisson constraints.
Adaptive stopping in MCMC using classifier-based dynamics
problem Sampling from complex, unnormalized probability densities
method Training state-dependent neural classifiers
result Significant reduction in average trajectory lengths
New algorithms improve stopping time for best arm identification.
problem Efficiently identifying the best alternative in experiments.
method Proposed algorithms with exponential-tailed stopping time.
result Proved that some algorithms never stop, leading to new methods.
We consider the optimal double stopping time problem defined for each stopping time S by $v(S)=\esssup\{E[ψ(τ_1, τ_2) | \F_S], τ_1, τ_2 \geq S \}$. Following the optimal one stopping time problem, we study the existence of optimal stopping times and give a method to compute them. The key point is the construction of …
Study optimal stopping problems with finite-time horizon and proves continuity and strict monotonicity of the boundary.
problem Optimal stopping problems with finite-time horizon and state-dependent discounting.
method Linear diffusion process, time-homogeneous gain function, fine regularity properties, continuity and strict monotonicity proof.
result Proves continuity and strict monotonicity of the optimal stopping boundary under mild assumptions.
Proposes a new model for predicting chronic conditions over time.
problem Predicting complex relationships between multiple chronic conditions.
method Continuous time Bayesian network with adaptive regularization for structure and parameter learning.
result Proposed model provides sparse, intuitive representation of chronic condition relationships.
We consider a zero-sum continuous time stopping game in which the pay-off is revealed in the maximum of the two stopping times instead of the minimum, which is the case in Dynkin games.
Solves optimal stopping for Gauss-Markov bridges using time-space transformation.
problem Optimal stopping problem of a Gauss-Markov bridge.
method Time-space transformation approach, Picard iteration algorithm.
result Lipschitz continuity of the optimal stopping boundary and its characterization.
We use probabilistic methods to characterise time dependent optimal stopping boundaries in a problem of multiple optimal stopping on a finite time horizon. Motivated by financial applications we consider a payoff of immediate stopping of "put" type and the underlying dynamics follows a geometric Brownian motion. The op…
In this paper, we propose several "measurements" of the "non-stopping timeness" of ends g of previsible sets, such that g avoids stopping times, in an ambiant filtration. We then study several explicit examples, involving last passage times of some remarkable martingales.
In this work we consider optimal stopping problems with conditional convex risk measures called optimised certainty equivalents. Without assuming any kind of time-consistency for the underlying family of risk measures, we derive a novel representation for the solution of the optimal stopping problem. In particular, we …
The paper tackles optimal stopping problems using reinforcement learning and singular control.
problem Continuous-time and state-space optimal stopping problems.
method Formulated as a singular control problem with randomized stopping times and penalized cumulative residual entropy.
result Identified unique optimal exploratory strategy through dynamic programming.
Early stopping method saves up to 75% computation time in policy search tasks.
problem Lengthy evaluation times in optimization problems, especially in robotics.
method A generalized early stopping criterion that only uses objective value at each time step.
result The method saves up to 75% computation time compared to no stopping.
Paper solves a complex stopping problem using regularization and HJB equations.
problem Time-inconsistent mean-variance optimal stopping problem
method Vanishing regularization method to derive HJB equations and prove existence of solutions
result Formally recovers variational inequalities for original problem
Method calculates Parisian stopping times and option prices using Markov chains.
problem Computing distribution and pricing of Parisian stopping times under Markov processes.
method Continuous-time Markov chain approximation to solve for distribution and convergence analysis.
result Sharp convergence rate and efficient method for diffusion and jump models.
Study optimal stopping times under regime-switching models with constraints.
problem Optimal stopping times for discounted payoffs on a regime-switching geometric Brownian motion.
method Solve variational inequality to find value functions and optimal thresholds.
result Existence and expressions of optimal stopping times under specific conditions.
In this paper, we study the dual representation for generalized multiple stopping problems, hence the pricing problem of general multiple exercise options. We derive a dual representation which allows for cashflows which are subject to volume constraints modeled by integer valued adapted processes and refraction period…
Inspired by Strotz's consistent planning strategy, we formulate the infinite horizon mean-variance stopping problem as a subgame perfect Nash equilibrium in order to determine time consistent strategies with no regret. Equilibria among stopping times or randomized stopping times may not exist. This motivates us to cons…
Study robustness of early-stopping GD for linear regression attacks.
problem Robustness of gradient-descent methods to adversarial attacks.
method Early-stopping strategies, gradient-descent, Mahalanobis attacks, feature-dependent learning rates, data transformations.
result Early-stopped GD is optimally robust to Euclidean-norm attacks but sub-optimal for Mahalanobis attacks.
Adaptive speculative decoding framework for LLMs using bandit algorithms.
problem Adaptive speculative decoding for LLMs to balance speed and quality.
method Formulated as a Multi-Armed Bandit problem, proposed UCBSpec and EXP3Spec algorithms.
result UCBSpec algorithm achieves optimal regret performance up to universal constants.
New method models stopping times that can be equal with non-zero probability.
problem Standard stopping time models assume conditional independence, limiting flexibility.
method Modified Cox construction with bivariate exponential distribution.
result Created a family of stopping times that can be equal with positive probability.
M-FISHER detects and adapts to streaming data shifts with statistical validity and stability.
problem Detecting and adapting to distributional shifts in streaming data.
method Constructs an exponential martingale from non-conformity scores and applies Ville's inequality for detection. Fisher-preconditioned updates for adaptation.
result Establishes M-FISHER as a principled approach for robust, anytime-valid detection and geometrically stable adaptation.
Paper proposes a sequential statistical test for comparing imitation learning policies with near-optimal stopping.
problem Challenges in rigorously comparing imitation learning policies due to small sample sizes and potential p-hacking.
method Sequential statistical test that adapts the number of trials based on intermediate results, achieving near-optimal stopping.
result Reduces the number of evaluation trials by up to 32% compared to state-of-the-art baselines, saving significant time and effort.
Paper offers anytime-valid inference for causal parameters using DML.
problem Classic DML is only valid asymptotically for a fixed sample size.
method Time-uniform DML results for anytime-valid inference.
result Valid inference at any arbitrary stopping time.
We show, under weaker assumptions than in the previous literature, that a perpetual optimal stopping game always has a value. We also show that there exists an optimal stopping time for the seller, but not necessarily for the buyer. Moreover, conditions are provided under which the existence of an optimal stopping time…
Unified framework for adaptive learning systems using consolidation and expansion operations.
problem Managing the balance between consolidating known knowledge and expanding into new evidence in adaptive learning systems.
method Introduces Consolidation-Expansion Operator Mechanics (OpMech) with the order-gap metric to control the balance.
result The order-gap signal provides real-time control and termination guarantees for adaptive learning systems.
This paper considers a time-inconsistent stopping problem in which the inconsistency arises from non-constant time preference rates. We show that the smooth pasting principle, the main approach that has been used to construct explicit solutions for conventional time-consistent optimal stopping problems, may fail under …
We consider two-player non-zero-sum stopping games in discrete time. Unlike Dynkin games, in our games the payoff of each player is revealed after both players stop. Moreover, each player can adjust her own stopping strategy according to the other player's action. In the first part of the paper, we consider the game wh…
New test detects independence in streaming data, adapting to data complexity.
problem Independence testing in streaming data with adaptive stopping.
method Sequential kernelized independence tests using betting principles.
result Valid inference in streaming data with improved power.
This paper extends results of Mortimer and Williams (1991) about changes of probability measure up to a random time under the assumptions that all martingales are continuous and that the random time avoids stopping times. We consider locally absolutely continuous measure changes up to a random time, changes of probabil…