arXiv research

A new RL algorithm POWR learns world models to estimate action-values.

Q($Δ$)-Learning improves Q-Learning by separating action-value functions into different time scales.

We consider the problem of learning the optimal action-value function in the discounted-reward Markov decision processes (MDPs). We prove a new PAC bound on the sample-complexity of model-based value iteration algorithm in the presence of the generative model, which indicates that for an MDP with N state-action pairs a…

A new method learns action representations for reinforcement learning.

In classical Q-learning, the objective is to maximize the sum of discounted rewards through iteratively using the Bellman equation as an update, in an attempt to estimate the action value function of the optimal policy. Conventionally, the loss function is defined as the temporal difference between the action value and…

New research shows exponential lower bounds for planning in MDPs with linearly-realizable optimal action-value functions.

New RL algorithm tackles complex discrete action spaces.

In many real-world settings, a team of agents must coordinate their behaviour while acting in a decentralised way. At the same time, it is often possible to train the agents in a centralised fashion in a simulated or laboratory setting, where global state information is available and communication constraints are lifte…

Off-policy stochastic actor-critic methods rely on approximating the stochastic policy gradient in order to derive an optimal policy. One may also derive the optimal policy by approximating the action-value gradient. The use of action-value gradients is desirable as policy improvement occurs along the direction of stee…

New method QMLE performs well in complex action spaces without policy gradients.

Bayesian framework for learning optimal action-value function in MDPs.

New approach for open ad hoc teamwork using graph-based policy learning.

QR-MIX models joint state-action values as a distribution to handle randomness in MARL.

This paper investigates learning sparse representations and action-value functions simultaneously in deep reinforcement learning.

A new RL paradigm reduces state-action-value function approximation inefficiency.

Network slicing is a key technology in 5G communications system. Its purpose is to dynamically and efficiently allocate resources for diversified services with distinct requirements over a common underlying physical infrastructure. Therein, demand-aware resource allocation is of significant importance to network slicin…

Paper presents an efficient exploration method for reinforcement learning.

QMIX combines per-agent values to create decentralised policies.

Diffusion-QL uses diffusion models to improve offline RL performance.

A critical and challenging problem in reinforcement learning is how to learn the state-action value function from the experience replay buffer and simultaneously keep sample efficiency and faster convergence to a high quality solution. In prior works, transitions are uniformly sampled at random from the replay buffer o…

New algorithms speed up inverse reinforcement learning by solving MDPs once.

The paper tackles reward-relevance in offline RL with sparse decision dynamics.

Latent variable models improve RL by facilitating efficient learning and exploration.

We propose a flexible method for estimating value functions in reinforcement learning without parametric assumptions.

We explore value-based solutions for multi-agent reinforcement learning (MARL) tasks in the centralized training with decentralized execution (CTDE) regime popularized recently. However, VDN and QMIX are representative examples that use the idea of factorization of the joint action-value function into individual ones f…

New RL method handles large state-action spaces with complex models.

Deep Reinforcement Learning improves with Weighted Q-Learning to reduce bias and uncertainty.

USAC balances pessimism and optimism in actor-critic training for better exploration and performance.

QTRAN++ improves MARL performance in complex environments.

MAGE optimizes policies using action gradients from model-based learning.

Despite the great empirical success of deep reinforcement learning, its theoretical foundation is less well understood. In this work, we make the first attempt to theoretically understand the deep Q-network (DQN) algorithm (Mnih et al., 2015) from both algorithmic and statistical perspectives. In specific, we focus on …

UA-LQE improves value function learning by selectively erasing uncertain entries in Q-matrix.

We present an off-policy actor-critic algorithm for Reinforcement Learning (RL) that combines ideas from gradient-free optimization via stochastic search with learned action-value function. The result is a simple procedure consisting of three steps: i) policy evaluation by estimating a parametric action-value function;…

Band-limited SAC improves learning efficiency and stability in simulated environments.

Many policy gradient methods are variants of Actor-Critic (AC), where a value function (critic) is learned to facilitate updating the parameterized policy (actor). The update to the actor involves a log-likelihood update weighted by the action-values, with the addition of entropy regularization for soft variants. In th…

RLMM extends psychometric models to larger tasks.

We consider Markov Decision Problems defined over continuous state and action spaces, where an autonomous agent seeks to learn a map from its states to actions so as to maximize its long-term discounted accumulation of rewards. We address this problem by considering Bellman's optimality equation defined over action-val…

Adapting momentum from optimization to reinforcement learning.

Transformers learn to predict chess moves with surprising accuracy and strength.

Develops a dynamic mean field theory for reinforcement learning.

This paper investigates trajectory tracking problem for a class of underactuated autonomous underwater vehicles (AUVs) with unknown dynamics and constrained inputs. Different from existing policy gradient methods which employ single actor-critic but cannot realize satisfactory tracking control accuracy and stable learn…

DRIFT uses RL to automate functional software testing efficiently.

Deep RBVFs improve continuous control in RL.

Equivariant CNNs improve RL performance in symmetric environments.

Improves RL algorithms with two techniques.

The objective is to study an on-line Hidden Markov model (HMM) estimation-based Q-learning algorithm for partially observable Markov decision process (POMDP) on finite state and action sets. When the full state observation is available, Q-learning finds the optimal action-value function given the current action (Q func…

We study a budgeted hyper-parameter tuning problem, where we optimize the tuning result under a hard resource constraint. We propose to solve it as a sequential decision making problem, such that we can use the partial training progress of configurations to dynamically allocate the remaining budget. Our algorithm combi…

Paper analyzes Q-learning with neural networks, proving a fast convergence rate.