arXiv research

COCOA improves credit assignment in reinforcement learning by measuring contributions to rewards.

A fair reward system boosts participation in federated learning.

The paper proposes a model reward scheme for collaborative ML based on Shapley value and information gain.

We propose RUDDER, a novel reinforcement learning approach for delayed rewards in finite Markov decision processes (MDPs). In MDPs the Q-values are equal to the expected immediate reward plus the expected future rewards. The latter are related to bias problems in temporal difference (TD) learning and to high variance p…

New model shows most users on Q&A site Stack Overflow ignore badges.

CGAs estimate team performance from data, simplifying SV computation.

This paper balances short-term and long-term rewards in policy learning.

Adapts two algorithms for online learning with delayed rewards.

Reinforcement learning is concerned with identifying reward-maximizing behaviour policies in environments that are initially unknown. State-of-the-art reinforcement learning approaches, such as deep Q-networks, are model-free and learn to act effectively across a wide range of environments such as Atari games, but requ…

This paper formalizes reward training in RLHF and provides theoretical guarantees.

Top-k Combinatorial Bandits generalize multi-armed bandits, where at each round any subset of $k$ out of $n$ arms may be chosen and the sum of the rewards is gained. We address the full-bandit feedback, in which the agent observes only the sum of rewards, in contrast to the semi-bandit feedback, in which the agent obse…

Learning preferences implicit in the choices humans make is a well studied problem in both economics and computer science. However, most work makes the assumption that humans are acting (noisily) optimally with respect to their preferences. Such approaches can fail when people are themselves learning about what they wa…

We propose a mean field game model to study the question of how centralization of reward and computational power occur in Bitcoin-like cryptocurrencies. Miners compete against each other for mining rewards by increasing their computational power. This leads to a novel mean field game of jump intensity control, which we…

Game-theoretic analysis of mining gaps in blockchain systems.

CRB tackles rising rewards in combinatorial online learning.

Reinforcement Learning(RL) with sparse rewards is a major challenge. We propose \emph{Hindsight Trust Region Policy Optimization}(HTRPO), a new RL algorithm that extends the highly successful TRPO algorithm with \emph{hindsight} to tackle the challenge of sparse rewards. Hindsight refers to the algorithm's ability to l…

Privacy-preserving reinforcement learning from human feedback using decoupled reward modeling.

New algorithms reduce slate bandit regret for large slates, outperforming existing methods.

Neural algorithms optimize arm selection with human preference feedback for complex reward functions.

This paper proposes a definition of system health in the context of multiple agents optimizing a joint reward function. We use this definition as a credit assignment term in a policy gradient algorithm to distinguish the contributions of individual agents to the global reward. The health-informed credit assignment is t…

Bayesian optimization improved for biased data.

Paper optimizes learning MDPs with adversarial rewards using linear models.

Recent reinforcement learning algorithms, though achieving impressive results in various fields, suffer from brittle training effects such as regression in results and high sensitivity to initialization and parameters. We claim that some of the brittleness stems from variance differences, i.e. when different environmen…

We investigate and provide new insights on the sampling rule called Top-Two Thompson Sampling (TTTS). In particular, we justify its use for fixed-confidence best-arm identification. We further propose a variant of TTTS called Top-Two Transportation Cost (T3C), which disposes of the computational burden of TTTS. As our …

SWIFT learns intrinsic rewards from LLM hidden states for efficient best-of-N sampling.

A new algorithm improves stochastic linear bandit performance using residual bootstrap.

New algorithm learns optimal policy for average reward MDPs with sample complexity matching lower bound.

This paper presents a new neural architecture that combines a modulated Hebbian network (MOHN) with DQN, which we call modulated Hebbian plus Q network architecture (MOHQA). The hypothesis is that such a combination allows MOHQA to solve difficult partially observable Markov decision process (POMDP) problems which impa…

New approach to meta-learning with variational Bayes for unlabeled data.

A new algorithm tackles delayed combinatorial semi-bandit with causal relations.

Paper closes the gap in MP-MAB problems with novel adaptive communication and exploration.

In classical Q-learning, the objective is to maximize the sum of discounted rewards through iteratively using the Bellman equation as an update, in an attempt to estimate the action value function of the optimal policy. Conventionally, the loss function is defined as the temporal difference between the action value and…

Online learning has traditionally focused on the expected rewards. In this paper, a risk-averse online learning problem under the performance measure of the mean-variance of the rewards is studied. Both the bandit and full information settings are considered. The performance of several existing policies is analyzed, an…

This paper presents a novel nonmyopic adaptive Gaussian process planning (GPP) framework endowed with a general class of Lipschitz continuous reward functions that can unify some active learning/sensing and Bayesian optimization criteria and offer practitioners some flexibility to specify their desired choices for defi…

Investigates risk measures for DC pension decumulation.

The growing use of virtual autonomous agents in applications like games and entertainment demands better control policies for natural-looking movements and actions. Unlike the conventional approach of hard-coding motion routines, we propose a deep learning method for obtaining control policies by directly mimicking raw…

A new approach for specifying and synthesizing subroutines for optimizing metrics.

RLVR training dynamics reveal an implicit curriculum that shapes learning progression.

Resolves spurious correlations in causal models via intervention design.

BlockFLow ensures privacy and accountability in federated learning.

Prior work finds a diversity paradox: diversity breeds innovation, and yet, underrepresented groups that diversify organizations have less successful careers within them. Does the diversity paradox hold for scientists as well? We study this by utilizing a near-population of ~1.2 million US doctoral recipients from 1977…

AdaDEM decouples EM into two parts to improve class overlap and uncertainty.

Unified framework for multi-user bandits using Laplacian kernels.

Study designs steering rewards for MFGs with unknown dynamics and model uncertainty.

In environments with continuous state and action spaces, state-of-the-art actor-critic reinforcement learning algorithms can solve very complex problems, yet can also fail in environments that seem trivial, but the reason for such failures is still poorly understood. In this paper, we contribute a formal explanation of…

Nonparametric Thompson Sampling achieves optimal regret for risk-averse bandits with sub-Gaussian rewards.

Paper proposes incentives for federated learning to ensure truthful contributions.

The performance of imitation learning is typically upper-bounded by the performance of the demonstrator. While recent empirical results demonstrate that ranked demonstrations allow for better-than-demonstrator performance, preferences over demonstrations may be difficult to obtain, and little is known theoretically abo…