arXiv research

Develops a new method for optimizing policies in hierarchical models.

Inverse reinforcement learning (IRL) aims to explain observed strategic behavior by fitting reinforcement learning models to behavioral data. However, traditional IRL methods are only applicable when the observations are in the form of state-action paths. This assumption may not hold in many real-world modeling setting…

New method reduces regret for sparse adversarial SSP problems.

The present work extends the randomized shortest-paths framework (RSP), interpolating between shortest-path and random-walk routing in a network, in three directions. First, it shows how to deal with equality constraints on a subset of transition probabilities and develops a generic algorithm for solving this constrain…

New algorithms minimize regret in SSP with optimal sparse updates.

The paper uses a novel framework to learn option prices by imitating principal investor behavior.

Paper identifies reductive MDPs, solving them in polynomial time.

New RL method handles large state-action spaces with complex models.

HiDe learns hierarchical control for complex tasks by separating planning and control.

New method for natural policy gradients converges linearly.

New method recovers diverse policies from expert data using state-action pair weighting.

A new RL paradigm reduces state-action-value function approximation inefficiency.

New RL method reduces sample complexity for large state-action spaces.

This work tackles model-based RL by optimizing state-action queries to learn policies with minimal data.

Novel framework proves fast RL convergence in continuous spaces.

Study online learning in MDPs with aggregate bandit feedback, achieving low regret in both stochastic and adversarial settings.

Kernel-UCBVI algorithm balances exploration and exploitation in metric state-action spaces.

A new method, Count-MORL, improves offline reinforcement learning by using state-action frequency.

New findings reveal discount regularization can be seen as a strong prior, leading to poor performance in unevenly sampled data.

Study optimal offline RL with uncertainty sets and distribution shifts.

Hierarchical Modular Reinforcement Learning (HMRL), consists of 2 layered learning where Profit Sharing works to plan a prey position in the higher layer and Q-learning method trains the state-actions to the target in the lower layer. In this paper, we expanded HMRL to multi-target problem to take the distance between …

The paper introduces a new intrinsic reward method for exploration in reinforcement learning.

We present an efficient algorithm for model-free episodic reinforcement learning on large (potentially continuous) state-action spaces. Our algorithm is based on a novel $Q$-learning policy with adaptive data-driven discretization. The central idea is to maintain a finer partition of the state-action space in regions w…

This paper unifies three regularization methods in batch reinforcement learning.

BRPO optimizes batch RL policies to better exploit state-action differences.

Paper introduces SALE for better state-action learning in RL.

Paper tackles covariate shift in offline IL using less proficient behavior data.

SPEDER extracts state-action abstraction from dynamics for reinforcement learning.

GIM learns by analogy to improve reinforcement learning efficiency.

MO2 learns useful behaviours from past experience for new tasks.

New RL approach uses future state and action visitation measures for better exploration.

We consider the problem of learning a policy for a Markov decision process consistent with data captured on the state-actions pairs followed by the policy. We assume that the policy belongs to a class of parameterized policies which are defined using features associated with the state-action pairs. The features are kno…

The Bellman error is a poor proxy for value function accuracy, even with all state-action pairs.

Federated Q-learning achieves linear speedup with heterogeneity, improving sample complexity.

Paper proposes efficient sample collection strategy for RL.

We tackle the Multi-task Batch Reinforcement Learning problem. Given multiple datasets collected from different tasks, we train a multi-task policy to perform well in unseen tasks sampled from the same distribution. The task identities of the unseen tasks are not provided. To perform well, the policy must infer the tas…

ZoomRL learns efficient strategies for large state-action spaces using a metric.

A critical and challenging problem in reinforcement learning is how to learn the state-action value function from the experience replay buffer and simultaneously keep sample efficiency and faster convergence to a high quality solution. In prior works, transitions are uniformly sampled at random from the replay buffer o…

The paper proposes a principle for dynamically adjusting the granularity of reinforcement learning abstractions.

The paper improves Q-learning by incorporating pessimism for better sample efficiency.

QR-MIX models joint state-action values as a distribution to handle randomness in MARL.

Develops a dynamic mean field theory for reinforcement learning.

We consider off-policy policy evaluation when the trajectory data are generated by multiple behavior policies. Recent work has shown the key role played by the state or state-action stationary distribution corrections in the infinite horizon context for off-policy policy evaluation. We propose estimated mixture policy …

Estimation of importance sampling weights for off-policy evaluation of contextual bandits often results in imbalance - a mismatch between the desired and the actual distribution of state-action pairs after weighting. In this work we present balanced off-policy evaluation (B-OPE), a generic method for estimating weights…

KeRNS tackles non-stationary reinforcement learning in metric spaces.

RPO uses past and future state-action info for better policy optimization.

Model-free learning for multi-agent stochastic games is an active area of research. Existing reinforcement learning algorithms, however, are often restricted to zero-sum games, and are applicable only in small state-action spaces or other simplified settings. Here, we develop a new data efficient Deep-Q-learning method…

Model-free Reinforcement Learning (RL) algorithms such as Q-learning [Watkins, Dayan 92] have been widely used in practice and can achieve human level performance in applications such as video games [Mnih et al. 15]. Recently, equipped with the idea of optimism in the face of uncertainty, Q-learning algorithms [Jin, Al…