arXiv research

This work bridges hyperbolic discounting in RL with exponential discounting.

Lower discount factors act as a regularizer in RL, improving performance.

A new method separates long-term value functions into components for better reinforcement learning.

Q-Learning overestimation bias influenced by learning rate, discount factor, and reward signal.

There is a consensus that human and non-human subjects experience temporal distortions in many stages of their perceptual and decision-making systems. Similarly, intertemporal choice research has shown that decision-makers undervalue future outcomes relative to immediate ones. Here we combine techniques from informatio…

Study uses FDA to analyze discount functions of different temperaments.

New methods improve temporal difference learning for policy evaluation in Markov decision processes.

This paper introduces a new reward shaping method for average-reward reinforcement learning.

This study prioritizes temporal resolution over spatial in energy systems models due to higher influence.

Temporal difference learning explained through gradient splitting, improving convergence times.

Paper improves TD learning algorithm bounds with linear approx.

Improved Q-learning with hindsight factor reduces overestimation errors.

A new correction term improves sample efficiency in deep reinforcement learning.

Accelerates TD learning for long-horizon reinforcement learning problems.

SA-BCP combines long-term and local evidence for efficient, adaptive online prediction.

Recently, \citet{SuttonMW15} introduced the emphatic temporal differences (ETD) algorithm for off-policy evaluation in Markov decision processes. In this short note, we show that the projected fixed-point equation that underlies ETD involves a contraction operator, with a $\sqrtγ$-contraction modulus (where $γ$ is the …

Paper analyzes TD($λ$) convergence rates for arbitrary features.

Introduces a new RL formalism focusing on event desirability.

A new method for reinforcement learning scales errors without tuning.

New algorithms improve distributional TD learning with linear approximations.

A method to estimate functions of the return using its moments in reinforcement learning.

In this work, we propose the kernel Pitman-Yor process (KPYP) for nonparametric clustering of data with general spatial or temporal interdependencies. The KPYP is constructed by first introducing an infinite sequence of random locations. Then, based on the stick-breaking construction of the Pitman-Yor process, we defin…

Paper develops a discounted algorithm for online convex optimization that adapts to unknown discount factors.

Repo rates are explained as a convexity effect from bond and derivative discount rates.

Paper introduces non-linear discounting models for default compensation and climate valuation.

We propose and analyze an alternate approach to off-policy multi-step temporal difference learning, in which off-policy returns are corrected with the current Q-function in terms of rewards, rather than with the target policy in terms of transition probabilities. We prove that such approximate corrections are sufficien…

Study on reinforcement learning dynamics using statistical physics.

Study analyzes how discounts affect train ticket purchases and rescheduling in Switzerland.

We demonstrate a limitation of discounted expected utility, a standard approach for representing the preference to risk when future cost is discounted. Specifically, we provide an example of the preference of a decision maker that appears to be rational but cannot be represented with any discounted expected utility. A …

Proves lower discount rates are needed for future losses.

Proposes a new framework for discount models.

Revises derivative pricing post financial crisis by defining a discount rate.

Enhances xVA valuation with initial margin and CSA discounting.

This paper shows how forward rate interpolations are equivalent to discount factor interpolations in yield curve construction.

Lazy training and mean field regimes studied for TD learning with nonlinear function approximation.

NewsNet-SDF uses deep learning to integrate financial news with financial data for better asset pricing.

Paper extends option-critic architecture to estimate natural gradient for reinforcement learning.

New RL approach handles non-exponential discounting for sequential decisions.

New scalable MARL framework for dynamic networked systems.

The valuation process that economic agents undergo for investments with uncertain payoff typically depends on their statistical views on possible future outcomes, their attitudes toward risk, and, of course, the payoff structure itself. Yields vary across different investment opportunities and their interrelations are …

We optimize discounts to maximize influence spread in social networks.

Study optimal portfolio strategies with time-varying discount rates.

The study uses reproducing kernels to model bond discount curves.

Study non-parametric value function estimation from a single path.

Policy gradient methods do not optimize the discounted objective, leading to suboptimal results.

The paper analyzes an actor-critic algorithm with target networks for deep reinforcement learning.

We show that different rates should be used for borrowing and discount rates, and that the risk-free rate should be used for discounting when assessing and comparing the cost of energy accross diffferent producers and technologies, on the example of photovoltaics. Recent quantitative models using the same rate for borr…

New method uses entropy to improve policy gradient exploration.