A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Optimizes control of hybrid systems with multiple switching processes.
Derives optimal control conditions using calculus of variations.
We present for the first time an asymptotic convergence analysis of two time-scale stochastic approximation driven by `controlled' Markov noise. In particular, both the faster and slower recursions have non-additive controlled Markov noise components in addition to martingale difference noise. We analyze the asymptotic…
Abstract: Surveying connections between ML and Control Theory.
Study nonparametric estimator for Markov chain transition matrices in offline setting.
Schrödinger bridge solved with Weyl calculus for quadratic state cost.
Study uses DRL with Lagrangian relaxation to solve temporal control tasks with STL constraints.
This paper tackles adaptive control of unknown Markov jump systems with sample complexity and regret bounds.
We are interested in understanding stability (almost sure boundedness) of stochastic approximation algorithms (SAs) driven by a `controlled Markov' process. Analyzing this class of algorithms is important, since many reinforcement learning (RL) algorithms can be cast as SAs driven by a `controlled Markov' process. In t…
SHADOWCAST generates graphs with user-specified attributes.
We consider an investor faced with the utility maximization problem in which the risky asset price process has pure-jump dynamics affected by an unobservable continuous-time finite-state Markov chain, the intensity of which can also be controlled by actions of the investor. Using the classical filtering theory, we redu…
Optimizes costs in uncertain Markov systems using risk filters.
Paper proposes a DRL-based controller for networked AP systems that reduces communication frequency.
We consider the inverse reinforcement learning problem, that is, the problem of learning from, and then predicting or mimicking a controller based on state/action data. We propose a statistical model for such data, derived from the structure of a Markov decision process. Adopting a Bayesian approach to inference, we sh…
RL approach for target tracking with unknown dynamics and sensor control.
This paper investigates methods for estimating the optimal stochastic control policy for a Markov Decision Process with unknown transition dynamics and an unknown reward function. This form of model-free reinforcement learning comprises many real world systems such as playing video games, simulated control tasks, and r…
Optimal investment strategy with expert opinions in uncertain conditions.
New algorithm solves uncertain Markov decision processes using Wasserstein uncertainty.
Framework for robust control in cooperative systems with uncertain common noise.
This paper improves MARL for networked systems through new protocols and discount factors.
Develops a model for bid and ask prices using stochastic control.
Bootstrap method for Markov chains in reinforcement learning.
We introduce a general framework for measuring risk in the context of Markov control processes with risk maps on general Borel spaces that generalize known concepts of risk measures in mathematical finance, operations research and behavioral economics. Within the framework, applying weighted norm spaces to incorporate …
This paper considers a non-Markov control problem arising in a financial market where asset returns depend on hidden factors. The problem is non-Markov because nonlinear filtering is required to make inference on these factors, and hence the associated dynamic program effectively takes the filtering distribution as one…
Investor selects portfolios based on news attention in a hidden Markov model.
We study a an optimal high frequency trading problem within a market microstructure model designed to be a good compromise between accuracy and tractability. The stock price is driven by a Markov Renewal Process (MRP), while market orders arrive in the limit order book via a point process correlated with the stock pric…
In this work, we consider the optimal portfolio selection problem under hard constraints on trading volume amounts when the dynamics of the risky asset returns are governed by a discrete-time approximation of the Markov-modulated geometric Brownian motion. The states of Markov chain are interpreted as the states of an …
We study the problem of online learning in a class of Markov decision processes known as linearly solvable MDPs. In the stationary version of this problem, a learner interacts with its environment by directly controlling the state transitions, attempting to balance a fixed state-dependent cost and a certain smooth cost…
Controller-Augmented Hidden Markov Models (CHMMs) are a framework for constrained sequential inference.
There are over 15 distinct communities that work in the general area of sequential decisions and information, often referred to as decisions under uncertainty or stochastic optimization. We focus on two of the most important fields: stochastic optimal control, with its roots in deterministic optimal control, and reinfo…
Develops CLTs for Markov chain transition probabilities and policies.
Unified approach for data-driven control of stochastic processes.
Model reduction of Markov processes is a basic problem in modeling state-transition systems. Motivated by the state aggregation approach rooted in control theory, we study the statistical state compression of a discrete-state Markov chain from empirical trajectories. Through the lens of spectral decomposition, we study…
We solve a continuous-time game-theoretic problem for Kihlstrom-Mirman preferences.
Market makers optimize bid/ask quotes under hidden Markov chain uncertainty.
New method controls gradient error for sparse MRFs.
We study the problem of regret minimization in partially observable linear quadratic control systems when the model dynamics are unknown a priori. We propose ExpCommit, an explore-then-commit algorithm that learns the model Markov parameters and then follows the principle of optimism in the face of uncertainty to desig…
Optimizes MCMC chains with neural control variates.
New method infers human sensorimotor costs from behavior.
In this paper we consider long-run risk sensitive average cost impulse control applied to a continuous-time Feller-Markov process. Using the probabilistic approach, we show how to get a solution to a suitable continuous-time Bellman equation and link it with the impulse control problem. The optimal strategy for the und…
Study optimal liquidation strategies under partial information in high-frequency trading.
Territorial control is a key aspect shaping the dynamics of civil war. Despite its importance, we lack data on territorial control that are fine-grained enough to account for subnational spatio-temporal variation and that cover a large set of conflicts. To resolve this issue, we propose a theoretical model of the relat…
This paper develops numerical methods for finding optimal dividend pay-out and reinsurance policies. A generalized singular control formulation of surplus and discounted payoff function are introduced, where the surplus is modeled by a regime-switching process subject to both regular and singular controls. To approxima…
This paper solves a Bayes sequential impulse control problem for a diffusion, whose drift has an unobservable parameter with a change point. The partially-observed problem is reformulated into one with full observations, via a change of probability measure which removes the drift. The optimal impulse controls can be ex…
Study optimal portfolios in a non-Markovian regime-switching model with random time horizon.
Study on natural actor-critic for POMDPs with finite memory.
We describe an abstract control-theoretic framework in which the validity of the dynamic programming principle can be established in continuous time by a verification of a small number of structural properties. As an application we treat several cases of interest, most notably the lower-hedging and utility-maximization…
We analyze stochastic approximation with Markov noise for reinforcement learning.