A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Reward tweaking optimizes behavior for long-term goals by adjusting the reward function.
New conditions show proxy data can improve policy learning in sparse expert data contexts.
Time series classification has received great attention over the past decade with a wide range of methods focusing on predictive performance by exploiting various types of temporal features. Nonetheless, little emphasis has been placed on interpretability and explainability. In this paper, we formulate the novel proble…
A contextual bandit method evaluates and improves inventory control policies.
A new method generates meta-tasks using generative models for unsupervised learning.
Eigenvalues of Steklov eigenproblems change predictably with boundary tweaks.
The classical trefoil is famous for having a three-colouring which distinguishes it from the unknot. The three-colouring is also notorious for not distinguishing the right handed from the left handed trefoil. However with a bit of tweaking the three colours can also be used for this task. What lies behind the method is…
This paper studies the performance of a recently proposed preconditioned stochastic gradient descent (PSGD) algorithm on recurrent neural network (RNN) training. PSGD adaptively estimates a preconditioner to accelerate gradient descent, and is designed to be simple, general and easy to use, as stochastic gradient desce…
In considering the mathematical problem of describing the geodesics on a torus or any other surface of revolution, there is a tremendous advantage in conceptual understanding that derives from taking the point of view of a physicist by interpreting parametrized geodesics as the paths traced out in time by the motion of…
Adversarial examples are maliciously tweaked images that can easily fool machine learning techniques, such as neural networks, but they are normally not visually distinguishable for human beings. One of the main approaches to solve this problem is to retrain the networks using those adversarial examples, namely adversa…
Efficient deep learning computing requires algorithm and hardware co-design to enable specialization: we usually need to change the algorithm to reduce memory footprint and improve energy efficiency. However, the extra degree of freedom from the algorithm makes the design space much larger: it's not only about designin…
We present SEBOOST, a technique for boosting the performance of existing stochastic optimization methods. SEBOOST applies a secondary optimization process in the subspace spanned by the last steps and descent directions. The method was inspired by the SESOP optimization method for large-scale problems, and has been ada…
Paper forecasts tax revenues in Bulgaria during pandemic.
LLMs produce volatile sentence-level sentiment classifications that affect financial decision-making.
Reward hacking exploits misspecified rewards, affecting agent capabilities and true performance.
Improved analysis shows momentum in SGD reduces batch size needs for non-convex objectives.
Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.
This work analyzes the value of future reward information in RL.
Self-supervised reward prediction improves RL in sparse reward settings.
The study categorizes reward errors in reinforcement learning, finding some can be beneficial.
Reward collapse occurs when ranking-based reward models yield uniform rewards for different prompts.
Reward models need more than just accuracy for effective RLHF.
Action guidance helps agents learn true objectives in games with sparse rewards.
The performance of Feedforward neural network (FNN) fully de-pends upon the selection of architecture and training algorithm. FNN architecture can be tweaked using several parameters, such as the number of hidden layers, number of hidden neurons at each hidden layer and number of connections between layers. There may b…
Quantum-enhanced metrology aims to estimate an unknown parameter such that the precision scales better than the shot-noise bound. Single-shot adaptive quantum-enhanced metrology (AQEM) is a promising approach that uses feedback to tweak the quantum process according to previous measurement outcomes. Techniques and form…
Paper proposes RRD to learn proxy rewards for sparse delayed rewards in episodic reinforcement learning.
Learning reward functions from data is a promising path towards achieving scalable Reinforcement Learning (RL) for robotics. However, a major challenge in training agents from learned reward models is that the agent can learn to exploit errors in the reward model to achieve high reward behaviors that do not correspond …
Enhances reward specification in RL with a novel language-based approach.
We propose a generic, Bayesian, information geometric approach to the exploration--exploitation trade-off in multi-armed bandit problems. Our approach, BelMan, uniformly supports pure exploration, exploration--exploitation, and two-phase bandit problems. The knowledge on bandit arms and their reward distributions is su…
Many reinforcement-learning researchers treat the reward function as a part of the environment, meaning that the agent can only know the reward of a state if it encounters that state in a trial run. However, we argue that this is an unnecessary limitation and instead, the reward function should be provided to the learn…
Extends reinforcement learning alignment to scalar rewards, improving math reasoning.
We introduce an algorithm for the pricing of finite expiry American options driven by Lévy processes. The idea is to tweak Carr's `Canadisation' method, cf. Carr [9] (see also Bouchard et al [5]), in such a way that the adjusted algorithm is viable for any Lévy process whose law at an independent, exponentially distrib…
This work characterizes reward function partial identifiability and its impact on policy optimization.
This paper introduces a new reward shaping method for average-reward reinforcement learning.
While recent progress in deep reinforcement learning has enabled robots to learn complex behaviors, tasks with long horizons and sparse rewards remain an ongoing challenge. In this work, we propose an effective reward shaping method through predictive coding to tackle sparse reward problems. By learning predictive repr…
Paper generalizes reward distribution in multi-armed bandits with temporally-partitioned rewards.
We present a novel method for learning a set of disentangled reward functions that sum to the original environment reward and are constrained to be independently obtainable. We define independent obtainability in terms of value functions with respect to obtaining one learned reward while pursuing another learned reward…
New reward function improves GAIL performance in task-based environments.
A new method resolves non-identifiability in reward modeling using anchor labels.
Reinforcement learning usually uses the feedback rewards of environmental to train agents. But the rewards in the actual environment are sparse, and even some environments will not rewards. Most of the current methods are difficult to get good performance in sparse reward or non-reward environments. Although using shap…
New concept: reward hacking, where optimizing a flawed reward function can hurt performance.
We propose RUDDER, a novel reinforcement learning approach for delayed rewards in finite Markov decision processes (MDPs). In MDPs the Q-values are equal to the expected immediate reward plus the expected future rewards. The latter are related to bias problems in temporal difference (TD) learning and to high variance p…
The paper tackles reinforcement learning with exogenous variables and rewards.
Study optimal reward schemes for inducing desired player performance in risky contests.
Reinforcement Learning (RL) agents require the specification of a reward signal for learning behaviours. However, introduction of corrupt or stochastic rewards can yield high variance in learning. Such corruption may be a direct result of goal misspecification, randomness in the reward signal, or correlation of the rew…
Develops HMRL for sparse reward RL problems, improving meta policy efficiency and transferability.
Study scaling laws of reward model overoptimization in reinforcement learning.
New RL algorithms improve average-reward performance.