arXiv research

Paper presents an efficient exploration method for reinforcement learning.

We study the use of randomized value functions to guide deep exploration in reinforcement learning. This offers an elegant means for synthesizing statistically and computationally efficient exploration with common practical approaches to value function learning. We present several reinforcement learning algorithms that…

Value functions struggle to represent transition dynamics, impacting statistical efficiency.

We propose randomized least-squares value iteration (RLSVI) -- a new reinforcement learning algorithm designed to explore and generalize efficiently via linearly parameterized value functions. We explain why versions of least-squares value iteration that use Boltzmann or epsilon-greedy exploration can be highly ineffic…

Paper develops efficient RL algorithm for general value function approximation.

PPG separates policy and value function training phases for better reinforcement learning efficiency.

ZOSPI improves RL policies with global value function exploitation.

Efficient estimators for smooth Hilbert-valued parameters with theoretical guarantees.

Bayesian approach improves Shapley value estimation efficiency.

We consider the problem of reinforcement learning over episodes of a finite-horizon deterministic system and as a solution propose optimistic constraint propagation (OCP), an algorithm designed to synthesize efficient exploration and value function generalization. We establish that when the true value function lies wit…

Efficiently computes quasiconcave envelope with limited data.

Kernel smoothing improves LLM reasoning efficiency.

QPLEX learns efficient multi-agent Q-values by enforcing IGM principle.

This paper studies a recent proposal to use randomized value functions to drive exploration in reinforcement learning. These randomized value functions are generated by injecting random noise into the training data, making the approach compatible with many popular methods for estimating parameterized value functions. B…

Proposes EVE for efficient exploration in reinforcement learning.

New kernels capture both local and non-local interactions efficiently.

Randomized value functions offer a promising approach towards the challenge of efficient exploration in complex environments with high dimensional state and action spaces. Unlike traditional point estimate methods, randomized value functions maintain a posterior distribution over action-space values. This prevents the …

Paper analyzes distributional reinforcement learning with value function approximation, introducing Bellman unbiasedness and a new algorithm.

Imitation learning, followed by reinforcement learning algorithms, is a promising paradigm to solve complex control tasks sample-efficiently. However, learning from demonstrations often suffers from the covariate shift problem, which results in cascading errors of the learned policy. We introduce a notion of conservati…

Boosts Q-learning by using value function bounds.

Proposes a method to handle missing inputs in Bayesian optimization.

Efficiently plans large MDPs with weak function approximations.

Pessimistic Minimax Value Iteration finds efficient NE policies from offline data.

Polynomial-time RL algorithm for constant actions under linear Bellman completeness.

We propose a differentiable sigmoid function for efficient p-value calculation in clustering.

New method estimates causal effects with multi-valued, time-varying treatments.

New RL approach uses resets to learn complex MDPs efficiently.

We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning. Optimistic approaches presented in the literature apply an optimistic boost to the value estimate at each state-action pair and select actions that are greedy with respect to the resulting optimistic value f…

This work improves imitation learning and goal-conditioned RL by estimating value densities.

This paper provides a block coordinate descent algorithm to solve unconstrained optimization problems. In our algorithm, computation of function values or gradients is not required. Instead, pairwise comparison of function values is used. Our algorithm consists of two steps; one is the direction estimate step and the o…

MUMBO optimizes multiple tasks efficiently, even with low-cost related functions.

Efficient RL method uses SSL for small labeled data in health outcomes.

Offline RL struggles with sample efficiency due to fundamental barriers.

FFBO optimizes functions as inputs and outputs, improving on existing BO methods.

Improved MLMC method boosts risk estimation efficiency.

We give an algorithmically efficient version of the learner-to-compression scheme conversion in Moran and Yehudayoff (2016). In extending this technique to real-valued hypotheses, we also obtain an efficient regression-to-bounded sample compression converter. To our knowledge, this is the first general compressed regre…

Algorithm finds latent structure in value functions for improved reinforcement learning.

Discuss new policy learning objectives and methods.

Proposes a value-based method for continuous control without an actor.

EASE estimator improves probabilistic value estimation efficiency.

We study reinforcement learning (RL) in high dimensional episodic Markov decision processes (MDP). We consider value-based RL when the optimal Q-value is a linear function of d-dimensional state-action feature representation. For instance, in deep-Q networks (DQN), the Q-value is a linear function of the feature repres…

New lower bound shows RL with linear approximations is computationally hard.

A new RL paradigm reduces state-action-value function approximation inefficiency.

GPS model predicts subspace-valued functions efficiently.

We propose a plan online and learn offline (POLO) framework for the setting where an agent, with an internal model, needs to continually act and learn in the world. Our work builds on the synergistic relationship between local model-based control, global value function learning, and exploration. We study how local traj…

KANOP uses KANs to efficiently price American options.

New algorithm learns optimal policies without explicit rewards.

The aim of this paper is to study the fast computation of the lower and upper bounds on the value function for utility maximization under the Heston stochastic volatility model with general utility functions. It is well known there is a closed form solution of the HJB equation for power utility due to its homothetic pr…