We aim to construct the optimal solutions to the undiscounted continuous-time infinite horizon optimization problems, the objective functionals of which may be unbounded. We identify the condition under which the limit of the solutions to the finite horizon problems is optimal for the infinite horizon problems under th…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “infinite horizon”
Study optimal liquidation strategies with infinite horizon and regime switching.
problem Optimal liquidation with semimartingale strategies in a stochastic environment.
method Characterization of value function and optimal strategy via BSDEs with infinite horizon.
result Existence and uniqueness of optimal control problem solutions.
Flexible forward improvement iteration for infinite time horizon Markovian optimal stopping problemsmath.PR
Improved algorithm for optimal stopping problems reduces runtime.
problem Optimal stopping problems with infinite time horizon and random discounting.
method Flexible forward improvement iteration with a variable look-ahead distance.
result The new algorithm converges and can significantly reduce runtime.
In an incomplete market, with incompleteness stemming from stochastic factors imperfectly correlated with the underlying stocks, we derive representations of homothetic (power, exponential and logarithmic) forward performance processes in factor-form using ergodic BSDE. We also develop a connection between the forward …
Paper proposes an efficient online learning method using an offline dataset for infinite horizon MDPs.
problem Efficient online reinforcement learning in infinite horizon MDPs with an unknown expert policy.
method Bayesian approach to model the expert's policy and minimize cumulative regret.
result Upper bound on regret of for the Informed PSRL algorithm.
Extends utility maximization theory for infinite horizons without strong no-arbitrage assumptions.
problem Maximizing lifetime utility from wealth over an infinite horizon.
method Develops a duality theory using deflators and supermartingale properties, extending previous work.
result Establishes a strong duality theorem for infinite horizon utility maximization under minimal no-arbitrage assumptions.
We aim to generalize the results of Cai and Nitta (2007) by allowing both the utility and production function to depend on time. We also consider an additional intertemporal optimality criterion. We clarify the conditions under which the limit of the solutions for the finite horizon problems is optimal among all attain…
The Infinite Horizon Investment-Consumption Problem for Epstein-Zin Stochastic Differential Utilityq-fin.MF
Investment and consumption strategy for risk-averse agents with Epstein-Zin utility.
problem Optimal investment and consumption strategy for Epstein-Zin utility.
method Detailed introduction to Epstein-Zin utility, existence and uniqueness proof, verification argument.
result Existence and uniqueness of optimal solution for Epstein-Zin utility under certain parameter restrictions.
Infinite-Dimensional LQ Mean Field Games with Common Noise: Small and Arbitrary Finite Time Horizonsmath.OC
Developed LQ MFG theory with common noise, proving existence and uniqueness.
problem Linear-quadratic mean field games with common noise.
method Coupled forward-backward stochastic evolution equations (FBSEEs) in Hilbert spaces.
result Existence and uniqueness of solutions for small and arbitrary finite time horizons.
Deep neural nets approximate random dynamical system trajectories uniformly in time.
problem Approximating trajectories of random dynamical systems over infinite time horizons.
method Recurrent neural networks with simple feedback structures.
result Certain random trajectories can be approximated uniformly in time to any desired accuracy.
Infinite horizon off-policy policy evaluation is a highly challenging task due to the excessively large variance of typical importance sampling (IS) estimators. Recently, Liu et al. (2018a) proposed an approach that significantly reduces the variance of infinite-horizon off-policy evaluation by estimating the stationar…
We consider an infinite horizon portfolio problem with borrowing constraints, in which an agent receives labor income which adjusts to financial market shocks in a path dependent way. This path-dependency is the novelty of the model, and leads to an infinite dimensional stochastic optimal control problem. We solve the …
Study tackles OPE in confounded settings, estimating policy value from proxies.
problem Difficulty in OPE due to unobserved confounders in infinite-horizon RL.
method Two-stage approach: estimating stationary distribution ratios and combining optimal balancing.
result Policy value can be identified from off-policy data with proxies and latent variable model.
UCRL2-VTR achieves nearly optimal regret for learning MDPs with linear function approximation.
problem Learning infinite-horizon average-reward MDPs with linear function approximation.
method UCRL2-VTR algorithm with Bernstein-type bonus.
result Achieves a regret of with matching lower bound.
Paper proves existence of ambient manifolds for null hypersurfaces solving Einstein equations.
problem Analyzing transverse expansion of metrics at null hypersurfaces.
method Covariant approach proving existence of ambient manifolds given asymptotic expansion and constraint equations.
result Existence of ambient manifolds solving Einstein equations to infinite order at null hypersurfaces.
Model-free reinforcement learning is known to be memory and computation efficient and more amendable to large scale problems. In this paper, two model-free algorithms are introduced for learning infinite-horizon average-reward Markov Decision Processes (MDPs). The first algorithm reduces the problem to the discounted-r…
Derives time-averaged active inference from control principles.
problem Finite-horizon or discounted-surprise problems in active inference.
method Derives infinite-horizon, average-surprise active inference from optimal control principles.
result Unified objective functional for sensorimotor control.
A Model-free Learning Algorithm for Infinite-horizon Average-reward MDPs with Near-optimal Regretcs.LG
A new model-free algorithm achieves near-optimal regret for infinite-horizon MDPs.
problem Model-free reinforcement learning for infinite-horizon average-reward MDPs.
method Exploration Enhanced Q-learning (EE-QL) for weakly communicating MDPs.
result Achieves regret bound for general weakly communicating MDPs.
We present a new infinite class of near-horizon geometries of degenerate horizons, satisfying Einstein's equations for all odd dimensions greater than five. The symmetry and topology of these solutions is compatible with those of black holes. The simplest examples give horizons of spatial topology S^3xS^2 or the non-tr…
We consider the off-policy estimation problem of estimating the expected reward of a target policy using samples collected by a different behavior policy. Importance sampling (IS) has been a key technique to derive (nearly) unbiased estimators, but is known to suffer from an excessively high variance in long-horizon pr…
Firms miscount their customers who stop buying without saying goodbye.
problem Counting non-contractual customers accurately.
method Estimating repeat purchase probabilities and extrapolating to infinite time.
result The count of alive customers is only partially identified, with a wide range of estimates.
New algorithms for learning MDPs with linear approximations in infinite-horizon settings.
problem Learning infinite-horizon average-reward MDPs with linear function approximation.
method Optimism principle, adversarial linear bandits, Natural Policy Gradient.
result Efficient algorithms with optimal or near-optimal regret bounds.
New insights into black hole horizons from asymptotic expansions.
problem Understanding the geometry of black hole horizons.
method Proving the asymptotic expansion of spacetime metrics at non-degenerate Killing horizons.
result The full asymptotic expansion of smooth vacuum metrics at non-degenerate Killing horizons is determined by the horizon geometry.
A fundamental question in reinforcement learning is whether model-free algorithms are sample efficient. Recently, Jin et al. \cite{jin2018q} proposed a Q-learning algorithm with UCB exploration policy, and proved it has nearly optimal regret bound for finite-horizon episodic MDP. In this paper, we adapt Q-learning with…
Reinforcement learning algorithms such as the deep deterministic policy gradient algorithm (DDPG) has been widely used in continuous control tasks. However, the model-free DDPG algorithm suffers from high sample complexity. In this paper we consider the deterministic value gradients to improve the sample efficiency of …
We demonstrate the existence of spherically-symmetric truly naked black holes (TNBH) for which the Kretschmann scalar is finite on the horizon but some curvature components including those responsible for tidal forces as well as the energy density measured by a free-falling observer are infinite. We choose a ra…
This paper improves Thompson Sampling for complex decision-making problems.
problem Learning in infinite-horizon discounted decision processes with unknown parameters.
method Developed a general canonical probability space and new metrics for analyzing adaptive learning algorithms.
result Thompson Sampling achieves complete learning in complex decision-making problems.
The application of existing methods for constructing optimal dynamic treatment regimes is limited to cases where investigators are interested in optimizing a utility function over a fixed period of time (finite horizon). In this manuscript, we develop an inferential procedure based on temporal difference residuals for …
New method estimates off-policy data without needing known behavior policy.
problem Long-horizon reinforcement learning with limited on-policy data.
method Formulates problem as fixed point of operator, uses RKHS for estimation.
result Asymptotic consistency and finite-sample generalization proven.
This paper is concerned with offline reinforcement learning (RL), which learns using pre-collected data without further exploration. Effective offline RL would be able to accommodate distribution shift and limited data coverage. However, prior algorithms or analyses either suffer from suboptimal sample complexities or …
New algorithm reduces reinforcement learning regret to sqrt(T) without strong dynamics assumptions.
problem Infinite-horizon average-reward reinforcement learning with linear MDPs.
method Approximate by discounted-reward MDPs and apply optimistic value iteration.
result Achieves O(sqrt(T)) regret with polynomial complexity.
In this paper, we provide an elementary, unified treatment of two distinct blue-shift instabilities for the scalar wave equation on a fixed Kerr black hole background: the celebrated blue-shift at the Cauchy horizon (familiar from the strong cosmic censorship conjecture) and the time-reversed red-shift at the event hor…
In this paper, we study optimal switching problems under ambiguity. To characterize the optimal switching under ambiguity in the finite horizon, we use multidimensional reflected backward stochastic differential equations (multidimensional RBSDEs) and show that a value function of the optimal switching under ambiguity …
New algorithm reduces regret in infinite MDPs with optimal variance-dependent bounds.
problem Infinite horizon MDPs lack optimal algorithms with low regret.
method Developed a UCB-style algorithm for average-reward and γ-regret.
result Achieved optimal variance-dependent regret bounds for both objectives.
Study optimal portfolio management with periodic evaluations in stochastic models, considering convex constraints.
problem Optimal portfolio management under ratio-type periodic evaluations in stochastic factor models with convex trading constraints.
method Transformed infinite horizon optimal control problem into an auxiliary terminal wealth optimization problem. Introduced an auxiliary unconstrained optimization problem in a modified market model. Used martingale duality approach to establish dual minimizer and optimal unconstrained wealth process.
result Derived and verified the optimal constrained portfolio process for the original problem over an infinite horizon.
New algorithms learn MDPs with better regret bounds using generative sampling.
problem Learning MDPs with optimal policies under uncertainty.
method Hybrid exploration-generative RL model, classical and quantum algorithms.
result Quantum algorithms achieve regret for infinite-horizon MDPs.
Study optimal consumption and investment for investors with Epstein-Zin preferences.
problem Optimal consumption and investment for investors with Epstein-Zin preferences in an incomplete market.
method Variational characterisation and direct method to prove existence of optimal policies.
result Existence and uniqueness of optimal consumption and investment policies.
We estimate risk measures in Markov cost processes with lower and upper bounds.
problem Estimating risk measures in infinite-horizon discounted costs within Markov processes.
method Truncation scheme and lower/upper bounds for CVaR and variance estimation.
result Upper and lower bounds for CVaR and variance estimation match up to logarithmic factors.
We prove that a large class of smooth solutions to the linear wave equation on subextremal rotating Kerr spacetimes which are regular and decaying along the event horizon become singular at the Cauchy horizon. More precisely, we show that assuming appropriate upper and lower bounds on the energy along t…
Proposes pT-Learning for optimal dynamic treatment regimes in mHealth.
problem Challenges in learning optimal dynamic treatment regimes with large intervention options and infinite time horizon.
method Proximal Temporal consistency Learning (pT-Learning) framework for adaptively adjusting between deterministic and stochastic policies.
result Minimax estimator avoids double sampling issue and can incorporate off-policy data.
We consider off-policy policy evaluation when the trajectory data are generated by multiple behavior policies. Recent work has shown the key role played by the state or state-action stationary distribution corrections in the infinite horizon context for off-policy policy evaluation. We propose estimated mixture policy …
The paper analyzes the sample complexity of offline RL with linear approximations, identifying a hard regime and providing an algorithm.
problem Sample complexity of policy evaluation in infinite-horizon offline reinforcement learning with linear function approximation.
method Identification of a hard regime and construction of hard instances; algorithm with sample complexity bound.
result An algorithm that guarantees approximation to the value function up to an additive error of ε with high probability.
New algorithms reduce dynamic regret in online MDPs with changing losses.
problem Online MDPs with adversarial loss changes and known transitions.
method Dynamic regret measure, novel ensemble algorithms for three models.
result Provably optimal dynamic regret bounds for episodic SSP, improved bounds for predictable environments.
The infinite Viterbi alignment is the limiting maximum a-posteriori estimate of the unobserved path in a hidden Markov model as the length of the time horizon grows. For models on state-space satisfying a new ``decay-convexity'' condition, we develop an approach to existence of the infinite Viterbi ali…
Overview of risk-sensitive Markov decision processes with Optimized Certainty Equivalent.
problem Optimizing decision-making under risk in Markov processes.
method Analyzes risk-sensitive criteria using Optimized Certainty Equivalent, including entropic risk and Conditional Value-at-Risk.
result Conditions for the existence of optimal policies and solution procedures are provided.
Symmetries of vacuum spacetimes with a compact Cauchy horizon of constant non-zero surface gravitymath.DG
We prove that any smooth vacuum spacetime containing a compact Cauchy horizon with surface gravity that can be normalised to a non-zero constant admits a Killing vector field. This proves a conjecture by Moncrief and Isenberg from 1983 under the assumption on the surface gravity and generalises previous results due to …
Ruin Probabilities for Risk Processes with Non-Stationary Arrivals and Subexponential Claimsq-fin.RM
In this paper, we obtain the finite-horizon and infinite-horizon ruin probability asymptotics for risk processes with claims of subexponential tails for non-stationary arrival processes that satisfy a large deviation principle. As a result, the arrival process can be dependent, non-stationary and non-renewal. We give t…
Study long-term asset liquidation behavior with external flows.
problem Investigate optimal liquidation in presence of external flows.
method Convergence analysis of BSDEs for value function and strategy.
result Long-term liquidation may not occur due to external flows.