A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
We develop a normative framework for hierarchical model-based policy optimization based on applying second-order methods in the space of all possible state-action paths. The resulting natural path gradient performs policy updates in a manner which is sensitive to the long-range correlational structure of the induced st…
New algorithms reduce regret in online MDPs by adapting to data and variance.
problem Adapting to both adversarial and stochastic environments in online MDPs.
method Develops algorithms based on global optimization and policy optimization, using optimistic follow-the-regularized-leader with log-barrier regularization.
result Achieves refined data-dependent and variance-dependent regret bounds.
We present the symmetric thermal optimal path (TOPS) method to determine the time-dependent lead-lag relationship between two stochastic time series. This novel version of the previously introduced TOP method alleviates some inconsistencies by imposing that the lead-lag relationship should be invariant with respect to …
In this paper, we prove some convergence results of a special case of optimistic policy iteration algorithm for stochastic shortest path problem. We consider both Monte Carlo and TD(λ) methods for the policy evaluation step under the condition that the termination state will eventually be reached almost surely.
We propose an algorithm for deterministic continuous Markov Decision Processes with sparse rewards that computes the optimal policy exactly with no dependency on the size of the state space. The algorithm has time complexity of O(∣R∣3×∣A∣2) and memory complexity of O(∣R∣×∣A∣), where ∣R∣ is the…
We consider model-free reinforcement learning for infinite-horizon discounted Markov Decision Processes (MDPs) with a continuous state space and unknown transition kernel, when only a single sample path under an arbitrary policy of the system is available. We consider the Nearest Neighbor Q-Learning (NNQL) algorithm to…
This paper improves tail dependence analysis by introducing a path-based approach.
problem The classical tail dependence coefficient fails to capture non-exchangeable features of tail dependence.
method The paper introduces a path-based maximal tail dependence approach to capture the most pronounced feature of dependence over all possible paths.
result The paper proves the existence and provides an explicit characterization of the path-based maximal TDC, improving analytical and computational tractability.
The present work extends the randomized shortest-paths framework (RSP), interpolating between shortest-path and random-walk routing in a network, in three directions. First, it shows how to deal with equality constraints on a subset of transition probabilities and develops a generic algorithm for solving this constrain…
Dupire's functional Itô calculus provides an alternative approach to the classical Malliavin calculus for the computation of sensitivities, also called Greeks, of path-dependent derivatives prices. In this paper, we introduce a measure of path-dependence of functionals within the functional Itô calculus framework. Name…
We develop asymptotically optimal policies for the multi armed bandit (MAB), problem, under a cost constraint. This model is applicable in situations where each sample (or activation) from a population (bandit) incurs a known bandit dependent cost. Successive samples from each population are iid random variables with u…
Recent years have witnessed a tremendous improvement of deep reinforcement learning. However, a challenging problem is that an agent may suffer from inefficient exploration, particularly for on-policy methods. Previous exploration methods either rely on complex structure to estimate the novelty of states, or incur sens…
In this paper, we introduce and develop the theory of semimartingale optimal transport in a path dependent setting. Instead of the classical constraints on marginal distributions, we consider a general framework of path dependent constraints. Duality results are established, representing the solution in terms of path d…
In this paper, we extend the first-order asymptotics analysis of Fouque et al. to general path-dependent financial derivatives using Dupire's functional Ito calculus. The main conclusion is that the market group parameters calibrated to vanilla options can be used to price to the same order exotic, path-dependent deriv…
The recently developed bag-of-paths (BoP) framework consists in setting a Gibbs-Boltzmann distribution on all feasible paths of a graph. This probability distribution favors short paths over long ones, with a free parameter (the temperature T) controlling the entropic level of the distribution. This formalism enables…
Collective classification has been intensively studied due to its impact in many important applications, such as web mining, bioinformatics and citation analysis. Collective classification approaches exploit the dependencies of a group of linked objects whose class labels are correlated and need to be predicted simulta…