A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Recent studies have shown that reinforcement learning (RL) models are vulnerable in various noisy scenarios. For instance, the observed reward channel is often subject to noise in practice (e.g., when rewards are collected through sensors), and is therefore not credible. In addition, for applications such as robotics, …
EBRM improves robustness and generalization of language model rewards.
problem Challenges in capturing complex human preferences and generalizing to unseen data in reward models.
method Energy-Based Reward Model (EBRM) that models reward distribution explicitly, using conflict-aware data filtering, label-noise-aware contrastive training, and hybrid initialization.
result Significant improvements in robustness and generalization, up to 5.97% improvement in safety-critical alignment tasks.
We consider a multi-armed bandit problem in a setting where each arm produces a noisy reward realization which depends on an observable random covariate. As opposed to the traditional static multi-armed bandit problem, this setting allows for dynamically changing rewards that better describe applications where side inf…
Deep reinforcement learning methods traditionally struggle with tasks where environment rewards are particularly sparse. One successful method of guiding exploration in these domains is to imitate trajectories provided by a human demonstrator. However, these demonstrations are typically collected under artificial condi…
We study reinforcement learning of chatbots with recurrent neural network architectures when the rewards are noisy and expensive to obtain. For instance, a chatbot used in automated customer service support can be scored by quality assurance agents, but this process can be expensive, time consuming and noisy. Previous …
Experience replay enables reinforcement learning agents to memorize and reuse past experiences, just as humans replay memories for the situation at hand. Contemporary off-policy algorithms either replay past experiences uniformly or utilize a rule-based replay strategy, which may be sub-optimal. In this work, we consid…
Novel framework learns policies from noisy expert demonstrations.
problem Learning effective policies with noisy expert demonstrations.
method Adaptive learning framework that jointly interacts with the environment and expert demonstrations, assigning weights to filter out noisy demonstrations.
result The proposed approach learns robustly with noisy demonstrations and achieves higher performance in fewer iterations.
Existing relation classification methods that rely on distant supervision assume that a bag of sentences mentioning an entity pair are all describing a relation for the entity pair. Such methods, performing classification at the bag level, cannot identify the mapping between a relation and a sentence, and largely suffe…
Graph representation learning, aiming to learn low-dimensional representations which capture the geometric dependencies between nodes in the original graph, has gained increasing popularity in a variety of graph analysis tasks, including node classification and link prediction. Existing representation learning methods …
We consider the problem of learning to play a repeated multi-agent game with an unknown reward function. Single player online learning algorithms attain strong regret bounds when provided with full information feedback, which unfortunately is unavailable in many real-world scenarios. Bandit feedback alone, i.e., observ…
Animals need to devise strategies to maximize returns while interacting with their environment based on incoming noisy sensory observations. Task-relevant states, such as the agent's location within an environment or the presence of a predator, are often not directly observable but must be inferred using available sens…
We describe a novel algorithm for noisy global optimisation and continuum-armed bandits, with good convergence properties over any continuous reward function having finitely many polynomial maxima. Over such functions, our algorithm achieves square-root regret in bandits, and inverse-square-root error in optimisation, …
New method uses hindsight to make exploration robust in stochastic environments.
problem Exploration in sparse-reward or reward-free environments, especially in stochastic settings.
method Learn representations of the future that capture unpredictable aspects, using them to predict and reward only the predictable parts of the world.
result Improves exploration in Atari games and Montezuma's Revenge, robust to stochasticity.
Deep RL algorithms struggle with noisy rewards in portfolio optimisation.
problem Evaluating deep reinforcement learning for portfolio optimisation with market impact.
method Simulated data with geometric Brownian motion and market impact model; Kelly criterion as upper bound; PPO and A2C with GAE; clipping; hidden Markov model for regime changes.
result PPO and A2C with GAE perform better with noisy rewards; PPO with HMM learns different policies for regime changes.