A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New issue found in value-based reinforcement learning for stochastic environments.
Overparameterized models generalize well in offline contextual bandits, but policy-based algorithms struggle.
This work explores representation complexity in RL paradigms, revealing model-based RL as the easiest task.
Modern deep learning methods provide effective means to learn good representations. However, is a good representation itself sufficient for sample efficient reinforcement learning? This question has largely been studied only with respect to (worst-case) approximation error, in the more classical approximate dynamic pro…
The basic financial purpose of a firm is to maximize its value. An inventory management system should also contribute to realization of this basic aim. Many current asset management models currently found in financial management literature were constructed with the assumption of book profit maximization as basic aim. H…
This paper improves reinforcement learning policies in a scalable way.
Proposes a value-based method for continuous control without an actor.
Value functions struggle to represent transition dynamics, impacting statistical efficiency.
TF-Coder simplifies tensor manipulation programming in TensorFlow.
Value-based methods constitute a fundamental methodology in planning and deep reinforcement learning (RL). In this paper, we propose to exploit the underlying structures of the state-action value function, i.e., Q function, for both planning and deep RL. In particular, if the underlying system dynamics lead to some glo…
Unified framework for CO problems using RL, providing optimal solutions and convergence guarantees.
Study evaluates valuation models for UK companies using case studies.
Stochastic Q-learning tackles large action spaces with reduced computation.
Unified view on selective credit assignment for reinforcement learning.
New RL algorithms adapt to time limits, improving task performance.
Pretraining reinforcement learning methods with demonstrations has been an important concept in the study of reinforcement learning since a large amount of computing power is spent on online simulations with existing reinforcement learning algorithms. Pretraining reinforcement learning remains a significant challenge i…
New RL algorithm handles delayed feedback with posterior sampling.
Centralised training with decentralised execution is an important setting for cooperative deep multi-agent reinforcement learning due to communication constraints during execution and computational tractability in training. In this paper, we analyse value-based methods that are known to have superior performance in com…
New methods estimate transport-growth pairs in unbalanced optimal transport.
This paper aims at theoretically and empirically comparing two standard optimization criteria for Reinforcement Learning: i) maximization of the mean value and ii) minimization of the Bellman residual. For that purpose, we place ourselves in the framework of policy search algorithms, that are usually designed to maximi…
The paper explores how different loss functions impact reinforcement learning algorithms.
A new multi-agent learning method improves performance in complex games.
In many real-world settings, a team of agents must coordinate their behaviour while acting in a decentralised way. At the same time, it is often possible to train the agents in a centralised fashion in a simulated or laboratory setting, where global state information is available and communication constraints are lifte…
While biomanufacturing plays a significant role in supporting the economy and ensuring public health, it faces critical challenges, including complexity, high variability, lengthy lead time, and very limited process data, especially for personalized new cell and gene biotherapeutics. Driven by these challenges, we prop…
The goal of imitation learning is for an apprentice to learn how to behave in a stochastic environment by observing a mentor demonstrating the correct behavior. Accurate prior knowledge about the correct behavior can reduce the need for demonstrations from the mentor. We present a novel approach to encoding prior knowl…
New method improves stability of soft FQI for offline RL.
RL teaches agents to make decisions through trial and error.
Analyzes premium data of Indian non-life insurers, finding GEV distribution best fits Lognormal and GEV extremes.
This paper analyzes errors in Shapley value-based model explanations.
This paper analyzes the sample complexity of two timescale reinforcement learning algorithms.
This paper shows using classification instead of regression improves deep RL scalability.
Tutorial on optimizing diffusion model samples for specific metrics.
Obtaining more accurate equity value estimates is the starting point for stock selection, value-based indexing in a noisy market, and beating benchmark indices through tactical style rotation. Unfortunately, discounted cash flow, method of comparables, and fundamental analysis typically yield discrepant valuation estim…
For feature selection and related problems, we introduce the notion of classification game, a cooperative game, with features as players and hinge loss based characteristic function and relate a feature's contribution to Shapley value based error apportioning (SVEA) of total training error. Our major contribution is ($…
Enhances data valuation by integrating global and local statistical properties.
Reincarnating RL reuses prior work to accelerate RL progress.
Paper proposes using tree-based surrogate models for efficient Shapley computation.
In this paper, we present a discrete-type approximation scheme to solve continuous-time optimal stopping problems based on fully non-Markovian continuous processes adapted to the Brownian motion filtration. The approximations satisfy suitable variational inequalities which allow us to construct -optimal stopping tim…
In this paper we present a novel quasi-Newton algorithm for use in stochastic optimisation. Quasi-Newton methods have had an enormous impact on deterministic optimisation problems because they afford rapid convergence and computationally attractive algorithms. In essence, this is achieved by learning the second-order (…
Distributional approaches to value-based reinforcement learning model the entire distribution of returns, rather than just their expected values, and have recently been shown to yield state-of-the-art empirical performance. This was demonstrated by the recently proposed C51 algorithm, based on categorical distributiona…
In this work, we consider the problem of model selection for deep reinforcement learning (RL) in real-world environments. Typically, the performance of deep RL algorithms is evaluated via on-policy interactions with the target environment. However, comparing models in a real-world environment for the purposes of early …
Most learning algorithms are not invariant to the scale of the function that is being approximated. We propose to adaptively normalize the targets used in learning. This is useful in value-based reinforcement learning, where the magnitude of appropriate value approximations can change over time when we update the polic…
This paper proposes a new optimization objective for value-based deep reinforcement learning. We extend conventional Deep Q-Networks (DQNs) by adding a model-learning component yielding a transcoder network. The prediction errors for the model are included in the basic DQN loss as additional regularizers. This augmente…
In value-based reinforcement learning methods such as deep Q-learning, function approximation errors are known to lead to overestimated value estimates and suboptimal policies. We show that this problem persists in an actor-critic setting and propose novel mechanisms to minimize its effects on both the actor and the cr…
Pref-SHAP explains preferences using Shapley values.
Valid p-value for bounded random variables without distributional assumptions.
Privileged Information Dropout improves RL performance without distillation.
Deep networks have enabled reinforcement learning to scale to more complex and challenging domains, but these methods typically require large quantities of training data. An alternative is to use sample-efficient episodic control methods: neuro-inspired algorithms which use non-/semi-parametric models that predict valu…