arXiv research

UCRL-WVTR tackles long-term reinforcement learning with general approximations, achieving horizon-free and instance-dependent regret bounds.

New algorithm learns optimal policies with just 1 episode, settling horizon-dependence in RL.

New algorithms for efficient learning with long-term rewards in contextual bandits.

A new Bayesian method optimizes time-dependent expensive functions with lookahead.

Solves infinite horizon portfolio problem with path-dependent labor income.

New algorithm achieves asymptotically optimal regret without horizon dependence.

This paper studies the utility maximization problem with changing time horizons in the incomplete Brownian setting. We first show that the primal value function and the optimal terminal wealth are continuous with respect to the time horizon $T$. Secondly, we exemplify that the expected utility stemming from applying th…

Market dynamic is quantified in terms of the entropy $S(τ,n)$ of the clusters formed by the intersections between the series of the prices $p_t$ and the moving average $\widetilde{p}_{t,n}$. The entropy $S(τ,n)$ is defined according to Shannon as $\sum P(τ,n)\log P(τ,n),$ with $P(τ,n)$ the probability for the cluster t…

Despite its potential to improve sample complexity versus model-free approaches, model-based reinforcement learning can fail catastrophically if the model is inaccurate. An algorithm should ideally be able to trust an imperfect model over a reasonably long planning horizon, and only rely on model-free updates when the …

We review a resent {\em time-dependent} performance measure for economical time series -- the (optimal) investment horizon approach. For stock indices, the approach shows a pronounced gain-loss asymmetry that is {\em not} observed for the individual stocks that comprise the index. This difference may hint towards an sy…

Behavior cloning can achieve horizon-independent sample complexity in offline imitation learning.

The article discusses new horizons in black hole physics.

This paper examines the volatility and covariance dynamics of cash and futures contracts that underlie the Optimal Hedge Ratio (OHR) across different hedging time horizons. We examine whether hedge ratios calculated over a short term hedging horizon can be scaled and successfully applied to longer term horizons. We als…

Extends RL to random stopping times, improving optimization.

New algorithms learn MDPs with better regret bounds using generative sampling.

Recently, there has been significant progress in understanding reinforcement learning in discounted infinite-horizon Markov decision processes (MDPs) by deriving tight sample complexity bounds. However, in many real-world applications, an interactive learning agent operates for a fixed or bounded period of time, for ex…

Heterotic horizons preserving 4 supersymmetries have sections which are T^2 fibrations over 6-dimensional conformally balanced Hermitian manifolds. We give new examples of horizons with sections S^3 X S^3 X T^2 and SU(3). We then examine the heterotic horizons which are T^4 fibrations over a Kahler 4-dimensional manifo…

We aim to generalize the results of Cai and Nitta (2007) by allowing both the utility and production function to depend on time. We also consider an additional intertemporal optimality criterion. We clarify the conditions under which the limit of the solutions for the finite horizon problems is optimal among all attain…

New algorithm for RL with horizon-free reward-free exploration for linear MDPs.

Long horizon reinforcement learning is as hard as short horizon learning.

New algorithm reduces reinforcement learning complexity, approaching contextual bandits.

Develops a regression approach for solving MDPs with general state and action spaces.

Reservoir computing predicts chaotic systems for long horizons with sparse updates.

We show that the supersymmetric near horizon black hole geometries of 6-dimensional supergravity coupled to any number of scalar and tensor multiplets are either locally $AdS_3\times Σ^3$, where Σ^3 is a homology 3-sphere, or $\bR^{1,1}\times {\cal S}^4$, where ${\cal S}^4$ is a 4-manifold whose geometry depends on the…

MQF$^2$ forecasts multivariate quantiles globally.

Study optimal consumption with drawdown limits over a fixed time frame.

New optimizer SF-NorMuon matches tuned AdamW across various horizons.

Optimizes investment under uncertain time horizons with non-concave utility.

Algorithm reduces episode count for CMDPs with constraints.

Study on cryptocurrency market correlations at various time scales.

Temporal aggregation reveals latent default correlation from monthly data.

New algorithm reduces regret in infinite MDPs with optimal variance-dependent bounds.

Temporal coarse-graining of latent default paths explains effective correlation in corporate defaults.

Many robotic applications require the agent to perform long-horizon tasks in partially observable environments. In such applications, decision making at any step can depend on observations received far in the past. Hence, being able to properly memorize and utilize the long-term history is crucial. In this work, we pro…

For an investor with constant absolute risk aversion and a long horizon, who trades in a market with constant investment opportunities and small proportional transaction costs, we obtain explicitly the optimal investment policy, its implied welfare, liquidity premium, and trading volume. We identify these quantities as…

The paper extends spacetime topology results using codimension 2 null cut locus properties.

We demonstrate the existence of spherically-symmetric truly naked black holes (TNBH) for which the Kretschmann scalar is finite on the horizon but some curvature components including those responsible for tidal forces as well as the energy density $\barρ$ measured by a free-falling observer are infinite. We choose a ra…

We establish that an optimistic variant of Q-learning applied to a fixed-horizon episodic Markov decision process with an aggregated state representation incurs regret $\tilde{\mathcal{O}}(\sqrt{H^5 M K} + εHK)$, where $H$ is the horizon, $M$ is the number of aggregate states, $K$ is the number of episodes, and $ε$ is …

A defining feature of non-stationary systems is the time dependence of their statistical parameters. Measured time series may exhibit Gaussian statistics on short time horizons, due to the central limit theorem. The sample statistics for long time horizons, however, averages over the time-dependent parameters. To model…

The paper examines how loss aversion impacts multi-armed bandit decisions over long periods.

Forecastability measures predictive information across horizons.

In this paper, we obtain the finite-horizon and infinite-horizon ruin probability asymptotics for risk processes with claims of subexponential tails for non-stationary arrival processes that satisfy a large deviation principle. As a result, the arrival process can be dependent, non-stationary and non-renewal. We give t…

Study optimal stopping problems with finite-time horizon and proves continuity and strict monotonicity of the boundary.

We characterise the value function of the optimal dividend problem with a finite time horizon as the unique classical solution of a suitable Hamilton-Jacobi-Bellman equation. The optimal dividend strategy is realised by a Skorokhod reflection of the fund's value at a time-dependent optimal boundary. Our results are obt…

Study forecasts sub-city real estate prices weekly using radar and news sentiment.

Study optimal policy regret in partially observable Markov games with adaptive opponents.

Optimal reinsurance and dividend strategy for insurance companies in a finite time.

Study on black hole interiors with matter fields, showing oscillation condition impacts blow-up.