A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Paper tackles policy selection in offline RL without hyperparameters.
RO-TD learns sparse value functions efficiently.
New algorithms for model selection in off-policy evaluation of reinforcement learning.
Bayesian approach improves Shapley value estimation efficiency.
Despite recent successes in Reinforcement Learning, value-based methods often suffer from high variance hindering performance. In this paper, we illustrate this in a continuous control setting where state of the art methods perform poorly whenever sensor noise is introduced. To overcome this issue, we introduce Recurre…
PBVFs generalize across policies using learned value functions.
We introduce Bayesian least-squares policy iteration (BLSPI), an off-policy, model-free, policy iteration algorithm that uses the Bayesian least-squares temporal-difference (BLSTD) learning algorithm to evaluate policies. An online variant of BLSPI has been also proposed, called randomised BLSPI (RBLSPI), that improves…
We consider the problem of reinforcement learning over episodes of a finite-horizon deterministic system and as a solution propose optimistic constraint propagation (OCP), an algorithm designed to synthesize efficient exploration and value function generalization. We establish that when the true value function lies wit…
This paper investigates the problem of determining a binary-valued function through a sequence of strategically selected queries. The focus is an algorithm called Generalized Binary Search (GBS). GBS is a well-known greedy algorithm for determining a binary-valued function through a sequence of strategically selected q…
Estimates and infers multi-stage stationary treatment policies with variable selection.
Recently, Petrik et al. demonstrated that L1Regularized Approximate Linear Programming (RALP) could produce value functions and policies which compared favorably to established linear value function approximation techniques like LSPI. RALP's success primarily stems from the ability to solve the feature selection and va…
Under a Bayesian framework, we formulate the fully sequential sampling and selection decision in statistical ranking and selection as a stochastic control problem, and derive the associated Bellman equation. Using value function approximation, we derive an approximately optimal allocation policy. We show that this poli…
Paper explores two methods for optimal portfolio selection in financial markets.
We consider a stochastic optimal control problem in a market model with temporary and permanent price impact, which is related to an expected utility maximization problem under finite fuel constraint. We establish the initial condition fulfilled by the corresponding value function and show its first regularity property…
In this paper, we propose a generic framework for devising an adaptive approximation scheme for value function approximation in reinforcement learning, which introduces multiscale approximation. The two basic ingredients are multiresolution analysis as well as tree approximation. Starting from simple refinable function…
UA-LQE improves value function learning by selectively erasing uncertain entries in Q-matrix.
We propose a flexible method for estimating value functions in reinforcement learning without parametric assumptions.
A novel approach uses an ensemble of Gaussian processes for robust and adaptive reinforcement learning.
Reinforcement learning encounters major challenges in multi-agent settings, such as scalability and non-stationarity. Recently, value function factorization learning emerges as a promising way to address these challenges in collaborative multi-agent systems. However, existing methods have been focusing on learning full…
We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning. Optimistic approaches presented in the literature apply an optimistic boost to the value estimate at each state-action pair and select actions that are greedy with respect to the resulting optimistic value f…
Dynamic abstention improves LLM accuracy by selectively terminating unpromising reasoning.
N-discount optimality was introduced as a hierarchical form of policy- and value-function optimality, with Blackwell optimality lying at the top level of the hierarchy Veinott (1969); Blackwell (1962). We formalize notions of myopic discount factors, value functions and policies in terms of Blackwell optimality in MDPs…
We present an optimal investment theorem for a currency exchange model with random and possibly discontinuous proportional transaction costs. The investor's preferences are represented by a multivariate utility function, allowing for simultaneous consumption of any prescribed selection of the currencies at a given term…
QPLEX learns efficient multi-agent Q-values by enforcing IGM principle.
The paper develops RL methods for optimal switching between multiple states.
A framework for reinforcement learning tackles CVRP with competitive results.
Investor aims to meet financial goals with deadlines and target amounts, considering stock trading costs.
Investigates portfolio selection with transaction costs and stochastic volatility, using deep learning for computation.
Breaks the hardness conjecture for batch RL with a novel tournament-based approach.
Partial domain adaptation aims to transfer knowledge from a label-rich source domain to a label-scarce target domain which relaxes the fully shared label space assumption across different domains. In this more general and practical scenario, a major challenge is how to select source instances in the shared classes acro…
We approximate derivatives of functions on manifolds by embedding them and applying vector-valued operators.
New method calculates Shapley values for uncertain functions.
This paper studies a recent proposal to use randomized value functions to drive exploration in reinforcement learning. These randomized value functions are generated by injecting random noise into the training data, making the approach compatible with many popular methods for estimating parameterized value functions. B…
The paper improves energy decay estimates for Dir-stationary Q-valued functions and applies them to Liouville-type theorems and continuity.
A core operation in reinforcement learning (RL) is finding an action that is optimal with respect to a learned value function. This operation is often challenging when the learned value function takes continuous actions as input. We introduce deep radial-basis value functions (RBVFs): value functions learned using a de…
New algorithm FLUTE achieves uniform-PAC convergence in RL with linear approx.
We study the portfolio selection problem of a long-run investor who is maximising the asymptotic growth rate of her expected utility. We show that, somewhat surprisingly, it is essentially not affected by introduction of a floor constraint which requires the wealth process to dominate a given benchmark at all times. We…
We propose a new perspective on representation learning in reinforcement learning based on geometric properties of the space of value functions. We leverage this perspective to provide formal evidence regarding the usefulness of value functions as auxiliary tasks. Our formulation considers adapting the representation t…
We seek to align agent policy with human expert behavior in a reinforcement learning (RL) setting, without any prior knowledge about dynamics, reward function, and unsafe states. There is a human expert knowing the rewards and unsafe states based on his preference and objective, but querying that human expert is expens…
A framework for goal-based investing with penalties for fund transfers.
In this work, we study value function approximation in reinforcement learning (RL) problems with high dimensional state or action spaces via a generalized version of representation policy iteration (RPI). We consider the limitations of proto-value functions (PVFs) at accurately approximating the value function in low d…
GPS model predicts subspace-valued functions efficiently.
New algorithms for collaborative reinforcement learning with limited communication.
We study the use of randomized value functions to guide deep exploration in reinforcement learning. This offers an elegant means for synthesizing statistically and computationally efficient exploration with common practical approaches to value function learning. We present several reinforcement learning algorithms that…
Selective state-adaptive regularization improves offline RL performance.
Develops hierarchical reinforcement learning value function approximators.
We consider the mean--variance portfolio optimization problem under the game theoretic framework and without risk-free assets. The problem is solved semi-explicitly by applying the extended Hamilton--Jacobi--Bellman equation. Although the coefficient of risk aversion in our model is a constant, the optimal amounts of m…
Boosts Q-learning by using value function bounds.