arXiv research

A new method recovers rewards from behavior policies using classification and regression.

Improved RCPs for MABs using normalized weight functions.

Thompson sampling used for linear bandits with normal-gamma priors.

INF-clip optimizes heavy-tailed MAB problems with improved performance.

The paper addresses statistical inference for online decision-making in a contextual bandit setting.

Most learning algorithms are not invariant to the scale of the function that is being approximated. We propose to adaptively normalize the targets used in learning. This is useful in value-based reinforcement learning, where the magnitude of appropriate value approximations can change over time when we update the polic…

This paper balances short-term and long-term rewards in policy learning.

A minimalist approach improves LLM reasoning by filtering incorrect responses.

Drawing an inspiration from behavioral studies of human decision making, we propose here a more general and flexible parametric framework for reinforcement learning that extends standard Q-learning to a two-stream model for processing positive and negative rewards, and allows to incorporate a wide range of reward-proce…

GRPO optimizes LLMs with verifiable rewards, amplifying policy success.

New framework recovers reward and rationality parameters from game behavior.

Proposes a new portfolio optimization method considering reward, dispersion, and asymmetry.

This paper studies semiparametric contextual bandits, a generalization of the linear stochastic bandit problem where the reward for an action is modeled as a linear function of known action features confounded by an non-linear action-independent term. We design new algorithms that achieve $\tilde{O}(d\sqrt{T})$ regret …

Deep reinforcement learning achieves superhuman performance in a range of video game environments, but requires that a designer manually specify a reward function. It is often easier to provide demonstrations of a target behavior than to design a reward function describing that behavior. Inverse reinforcement learning …

Study on optimizing task allocation for agents receiving proposals sequentially.

As bandit algorithms are increasingly utilized in scientific studies and industrial applications, there is an associated increasing need for reliable inference methods based on the resulting adaptively-collected data. In this work, we develop methods for inference on data collected in batches using a bandit algorithm. …

Proposes a real-time anomaly detection system using IRL.

Robust real-world learning should benefit from both demonstrations and interactions with the environment. Current approaches to learning from demonstration and reward perform supervised learning on expert demonstration data and use reinforcement learning to further improve performance based on the reward received from …

We study the regret minimization problem in the novel setting of generalized kernelized bandits (GKBs), where we optimize an unknown function $f^*$ belonging to a reproducing kernel Hilbert space (RKHS) having access to samples generated by an exponential family (EF) reward model whose mean is a non-linear function $μ(…

In this paper we investigate the Follow the Regularized Leader dynamics in sequential imperfect information games (IIG). We generalize existing results of Poincaré recurrence from normal-form games to zero-sum two-player imperfect information games and other sequential game settings. We then investigate how adapting th…

New algorithm uses control variates to improve multi-armed bandit performance.

The generalized linear bandit framework has attracted a lot of attention in recent years by extending the well-understood linear setting and allowing to model richer reward structures. It notably covers the logistic model, widely used when rewards are binary. For logistic bandits, the frequentist regret guarantees of e…

Bayes optimal algorithm under certain conditions doesn't achieve exponential simple regret.

A method for eliciting expert beliefs using preferential questions and normalizing flows.

New algorithm reduces regret for single-index bandits to nearly optimal.

Improved regret for single-index bandits with optimal algorithm.

At an early age, human infants are able to learn and build a model of the world very quickly by constantly observing and interacting with objects around them. One of the most fundamental intuitions human infants acquire is intuitive physics. Human infants learn and develop these models, which later serve as prior knowl…

Paper tackles outlier detection in multi-armed bandits, achieving high accuracy with reduced exploration costs.

AdaDEM decouples EM into two parts to improve class overlap and uncertainty.

New algorithm tackles heavy-tailed rewards in RL with instance-dependent regret bounds.

The paper analyzes RLVR's training dynamics, proving convergence depends on aligning update direction with Gradient Gap.

Recent work has identified that classification models implemented as neural networks are vulnerable to data-poisoning and Trojan attacks at training time. In this work, we show that these training-time vulnerabilities extend to deep reinforcement learning (DRL) agents and can be exploited by an adversary with access to…

In this paper, we present a two-stage stochastic international portfolio optimisation model to find an optimal allocation for the combination of both assets and currency hedging positions. Our optimisation model allows a "currency overlay", or a deviation of currency exposure from asset exposure, to provide flexibility…

Deep Reinforcement Learning (DRL) algorithms for continuous action spaces are known to be brittle toward hyperparameters as well as \cut{being}sample inefficient. Soft Actor Critic (SAC) proposes an off-policy deep actor critic algorithm within the maximum entropy RL framework which offers greater stability and empiric…

Develops a flexible batched experimentation framework for limited adaptivity.

New method decomposes Markov chain rewards into persistent and transient components.

Most real life systems have a random component: the multitude of endogenous and exogenous factors influencing them result in stochastic fluctuations of the parameters determining their dynamics. These empirical systems are in many cases subject to noise of multiplicative nature. The special properties of multiplicative…

New method identifies uncertainty shocks in financial markets using revised VIX.

Reward hacking exploits misspecified rewards, affecting agent capabilities and true performance.

DreamerV3 learns diverse tasks with a single configuration.

Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.

This work analyzes the value of future reward information in RL.

Self-supervised reward prediction improves RL in sparse reward settings.

The study categorizes reward errors in reinforcement learning, finding some can be beneficial.

Reward collapse occurs when ranking-based reward models yield uniform rewards for different prompts.

Reward models need more than just accuracy for effective RLHF.

Action guidance helps agents learn true objectives in games with sparse rewards.

Paper proposes RRD to learn proxy rewards for sparse delayed rewards in episodic reinforcement learning.