arXiv research

POTEC tackles off-policy learning in large action spaces, improving effectiveness.

In this work, we provide theoretical guarantees for reward decomposition in deterministic MDPs. Reward decomposition is a special case of Hierarchical Reinforcement Learning, that allows one to learn many policies in parallel and combine them into a composite solution. Our approach builds on mapping this problem into a…

The paper uncovers the mathematical structure enabling value decomposition in multi-agent systems.

This paper improves Q-learning bounds using reference-advantage decomposition.

The paper explains why estimating a history-dependent policy can reduce MSE in reinforcement learning.

New method combines value function decomposition and policy gradients for cooperative multi-agent reinforcement learning.

A new multi-agent learning method improves performance in complex games.

The paper proposes a method to decompose value functions in RL for better understanding and prediction.

SPEDER extracts state-action abstraction from dynamics for reinforcement learning.

The study examines how different interpolation methods affect the decomposition of life insurance surplus.

NOHD optimizes multi-agent systems by decomposing dynamics into irrotational and solenoidal components.

Paper introduces a new principle for fair redistribution of insurance surplus.

Many advanced Learning from Demonstration (LfD) methods consider the decomposition of complex, real-world tasks into simpler sub-tasks. By reusing the corresponding sub-policies within and between tasks, they provide training data for each policy from different high-level tasks and compose them to perform novel ones. E…

Evaluating AI investment strategies

Novel framework for risk-sensitive reinforcement learning using martingale decomposition.

Decomposes spillover effects under misspecified exposure mappings.

The paper introduces return parity for fairness in MDPs, addressing delayed and adverse effects.

In hierarchical reinforcement learning a major challenge is determining appropriate low-level policies. We propose an unsupervised learning scheme, based on asymmetric self-play from Sukhbaatar et al. (2018), that automatically learns a good representation of sub-goals in the environment and a low-level policy that can…

The paper studies reward concentration in MDPs, covering asymptotic and non-asymptotic settings.

The task of dialog management is commonly decomposed into two sequential subtasks: dialog state tracking and dialog policy learning. In an end-to-end dialog system, the aim of dialog state tracking is to accurately estimate the true dialog state from noisy observations produced by the speech recognition and the natural…

New method prevents RLHF alignment collapse by accounting for policy's influence on reward model updates.

Improved Q-learning for robot control with faster data-efficiency.

New method decomposes Markov chain rewards into persistent and transient components.

We present a novel method for learning a set of disentangled reward functions that sum to the original environment reward and are constrained to be independently obtainable. We define independent obtainability in terms of value functions with respect to obtaining one learned reward while pursuing another learned reward…

The paper introduces a novel method for stable off-policy learning using value function chaining.

Bayesian neural networks with latent variables are scalable and flexible probabilistic models: They account for uncertainty in the estimation of the network weights and, by making use of latent variables, can capture complex noise patterns in the data. We show how to extract and decompose uncertainty into epistemic and…

The paper develops a new model to evaluate policies in complex temporal/spatial experiments.

Enhances large language models' reasoning through simpler off-policy reinforcement learning.

Minimal assumptions analysis of Q-learning with time-varying policies.

New algorithm for risk-sensitive reinforcement learning with natural policy gradients.

A new reinforcement learning approach using competitive primitives that specialize and specialize based on information needs.

We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…

We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…

Second-order estimator improves continuous-time policy evaluation.

Paper proposes FedQ-Advantage for federated Q-learning with near-optimal regret and low communication cost.

Q($Δ$)-Learning improves Q-Learning by separating action-value functions into different time scales.

Graph ConvNet learns to solve Tree Decomposition problems efficiently.

New algorithm reduces decision switching in dynamic environments.

New estimator improves off-policy evaluation for large action spaces.

Paper proposes a new DRL algorithm optimizing Spectral Risk Measures for better risk management.

Perceptor Gradients learns symbolic representations from raw data.

DeepPPMNet forecasts EMS demand and performs causal analyses for policy-making.

We propose RUDDER, a novel reinforcement learning approach for delayed rewards in finite Markov decision processes (MDPs). In MDPs the Q-values are equal to the expected immediate reward plus the expected future rewards. The latter are related to bias problems in temporal difference (TD) learning and to high variance p…

Paper addresses online alignment of large language models under uncertain preference feedback.

Typical reinforcement learning (RL) agents learn to complete tasks specified by reward functions tailored to their domain. As such, the policies they learn do not generalize even to similar domains. To address this issue, we develop a framework through which a deep RL agent learns to generalize policies from smaller, s…

RODE learns roles to simplify multi-agent tasks.

The ability of a reinforcement learning (RL) agent to learn about many reward functions at the same time has many potential benefits, such as the decomposition of complex tasks into simpler ones, the exchange of information between tasks, and the reuse of skills. We focus on one aspect in particular, namely the ability…

Paper develops a new probabilistic method for American options using entropy regularization.