arXiv research

Smooth distributions on subcartesian spaces can be globally finitely generated.

Kernel-UCBVI algorithm balances exploration and exploitation in metric state-action spaces.

This paper develops a Hoeffding inequality for the partial sums $\sum_{k=1}^n f (X_k)$, where $\{X_k\}_{k \in \mathbb{Z}_{> 0}}$ is an irreducible Markov chain on a finite state space $S$, and $f : S \to [a, b]$ is a real-valued function. Our bound is simple, general, since it only assumes irreducibility and finiteness…

Algorithm estimates human decision-making in high-dimensional states with finite-time guarantees.

The paper explores geometric calculations on probability manifolds derived from master equations.

We investigate the statistical complexity of estimating the parameters of a discrete-state Markov chain kernel from a single long sequence of state observations. In the finite case, we characterize (modulo logarithmic factors) the minimax sample complexity of estimation with respect to the operator infinity norm, while…

Method infers causal structure from system behaviors using RKHS and kernel $ε$-machines.

In his 2011 work, Maas has shown that the law of any time-reversible continuous-time Markov chain with finite state space evolves like a gradient flow of the relative entropy with respect to its stationary distribution. In this work we show the converse to the above by showing that if the relative law of a Markov chain…

Study on natural actor-critic for POMDPs with finite memory.

We study online reinforcement learning for finite-horizon deterministic control systems with {\it arbitrary} state and action spaces. Suppose that the transition dynamics and reward function is unknown, but the state and action space is endowed with a metric that characterizes the proximity between different states and…

New method predicts state evolution for non-first-order algorithms on nonconvex problems.

This paper generalizes neural transport learning for free energy estimation in arbitrary state spaces.

The paper explores harmonic maps and their properties in symmetric spaces.

We introduce the notion of a "state function" for framed tangles in a disk. After choosing a finite set of states for each marked disk, a state function is a projection from the vector space spanned by all tangles to the vector space spanned by the states, that is local, and topologically invariant. Given the states fo…

Develops a regression approach for solving MDPs with general state and action spaces.

Study provides convergence guarantees for discrete diffusion models on finite and infinite state spaces.

Adler had shown in 1979 that the Toda system can be given a coad- joint orbit description. We quantize the Toda system by viewing it as a single orbit of a multiplicative group of lower triangular matrices of determinant one with pos- itive diagonal entries. We get a unitary representation of the group with square inte…

New model for insurance states using Markov jump processes with non-countable state space.

Estimates quantum cohomology complexity for Fano varieties and homogeneous spaces.

The method approximates stationary distributions of Markov models by truncating irrelevant states.

RANDPOL uses randomized networks for efficient reinforcement learning in continuous state and action MDPs.

Abstract: Geometrically reformulates estimation theory for finite-dimensional C*-algebras.

We address several problems concerning the geometry of the space of Hermitian operators on a finite-dimensional Hilbert space, in particular the geometry of the space of density states and canonical group actions on it. For quantum composite systems we discuss and give examples of measures of entanglement.

Alexander's conjecture extended to infinite simplicial complexes.

The goal of this note is to prove a compact embedding result for spaces of forward rate curves. As a consequence of this result, we show that any forward rate evolution can be approximated by a sequence of finite dimensional processes in the larger state space.

In this paper we show how to approximate a Heath-Jarrow-Morton dynamics for the forward prices in commodity markets with arbitrage-free models which have a finite dimensional state space. Moreover, we recover a closed form representation of the forward price dynamics in the approximation models and derive the rate of c…

New method uses Cantor embeddings and Wasserstein distances to analyze predictive states in time series data.

We study an open problem of risk-sensitive portfolio allocation in a regime-switching credit market with default contagion. The state space of the Markovian regime-switching process is assumed to be a countably infinite set. To characterize the value function, we investigate the corresponding recursive infinite-dimensi…

Proves homological inequality for cycles in Hadamard spaces of asymptotic rank 2.

Main theorem of this paper states that Floer cohomology groups in a Hilbert space are isomorphic to the cohomological Conley Index. It is also shown that calculating cohomological Conley Index does not require finite dimensional approximations of the vector field. Further directions are discussed.

The paper introduces Causal Neural Operators to approximate operators in stochastic analysis.

Study learns state representations from observations for control, proving guarantees.

New empirical PAC-Bayes bound for Markov chains with finite state space.

We extend the coherent state transform (CST) of Hall to the context of the moduli spaces of semistable holomorphic vector bundles with fixed determinant over elliptic curves. We show that by applying the CST to appropriate distributions, we obtain the space of level k, rank n and genus one non-abelian theta functions w…

A quantum system can be entirely described by the Kähler structure of the projective space P(H) associated to the Hilbert space H of possible states; this is the so-called geometrical formulation of quantum mechanics. In this paper, we give an explicit link between the geometrical formulation (of finite dimensional qua…

Develops EFT for ResNets, revealing limitations of kernel-only approach.

The paper tackles finding optimal treatment sequences in continuous state spaces.

These introductory lectures show how to define finite type invariants of links and 3-manifolds by counting graph configurations in 3-manifolds, following ideas of Witten and Kontsevich. The linking number is the simplest finite type invariant for 2-component links. It is defined in many equivalent ways in the first sec…

Develops a dynamic mean field theory for reinforcement learning.

SRMC framework reduces Monte Carlo variance by history-based sampling in high-dimensional spaces.

Study cost-driven state representation learning for control from partial observations.

A Budgeted Markov Decision Process (BMDP) is an extension of a Markov Decision Process to critical applications requiring safety constraints. It relies on a notion of risk implemented in the shape of a cost signal constrained to lie below an - adjustable - threshold. So far, BMDPs could only be solved in the case of fi…

Reinforcement learning (RL) in Markov decision processes (MDPs) with large state spaces is a challenging problem. The performance of standard RL algorithms degrades drastically with the dimensionality of state space. However, in practice, these large MDPs typically incorporate a latent or hidden low-dimensional structu…

We consider the problem of estimating from sample paths the absolute spectral gap $γ_*$ of a reversible, irreducible and aperiodic Markov chain $(X_t)_{t \in \mathbb{N}}$ over a finite state space $Ω$. We propose the ${\tt UCPI}$ (Upper Confidence Power Iteration) algorithm for this problem, a low-complexity algorithm …

New RL method reduces sample complexity for large state-action spaces.

We consider nonparametric estimation of the state price density encapsulated in option prices. Unlike usual density estimation problems, we only observe option prices and their corresponding strike prices rather than samples from the state price density. We propose to model the state price density directly with a nonpa…

We consider finite groups which admit a faithful, smooth action on an acyclic manifold of dimension three, four or five (e.g. euclidean space). Our first main result states that a finite group acting on an acyclic 3- or 4-manifold is isomorphic to a subgroup of the orthogonal group O(3) or O(4), respectively. The analo…

Predictive State Representations (PSRs) are an expressive class of models for controlled stochastic processes. PSRs represent state as a set of predictions of future observable events. Because PSRs are defined entirely in terms of observable data, statistically consistent estimates of PSR parameters can be learned effi…