arXiv research

Optimistic NPG improves policy optimization in online RL with efficient sample complexity.

New algorithm reduces regret in sequential decision-making problems.

Optimistic estimate predicts best fitting performance of nonlinear models.

We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning. Optimistic approaches presented in the literature apply an optimistic boost to the value estimate at each state-action pair and select actions that are greedy with respect to the resulting optimistic value f…

Unified framework for model-based RL with sample complexity guarantees.

It is well known that in stochastic multi-armed bandits (MAB), the sample mean of an arm is typically not an unbiased estimator of its true mean. In this paper, we decouple three different sources of this selection bias: adaptive \emph{sampling} of arms, adaptive \emph{stopping} of the experiment, and adaptively \emph{…

Bayesian classifier improves robustness with optimistic score ratio.

Paper introduces DOO models to outperform SAA out-of-sample.

Optimistic actor-critic tackles linear MDPs with parametric policies.

New learning rules achieve optimal sample complexity for weakly supervised classification.

The likelihood function is a fundamental component in Bayesian statistics. However, evaluating the likelihood of an observation is computationally intractable in many applications. In this paper, we propose a non-parametric approximation of the likelihood that identifies a probability measure which lies in the neighbor…

Optimizes algorithms for non-concave bandit problems.

A new framework for structured bandits using influence diagrams and variational Thompson sampling.

Optimistic Thompson Sampling reduces regret in unknown multi-player games.

We derive an alternative proof for the regret of Thompson sampling (\ts) in the stochastic linear bandit setting. While we obtain a regret bound of order $\widetilde{O}(d^{3/2}\sqrt{T})$ as in previous results, the proof sheds new light on the functioning of the \ts. We leverage on the structure of the problem to show …

StoSOO optimistically maximizes noisy, locally smooth functions.

Develops ODRPO to improve RL algorithms with better performance and stability.

We show that the optimistic limits of the colored Jones polynomials of the hyperbolic knots coincide with the optimistic limits of the Kashaev invariants modulo $4π^2$.

We consider the problem of online planning in a Markov Decision Process when given only access to a generative model, restricted to open-loop policies - i.e. sequences of actions - and under budget constraint. In this setting, the Open-Loop Optimistic Planning (OLOP) algorithm enjoys good theoretical guarantees but is …

Proposes Optimistic Pessimistically Initialised Q-Learning (OPIQ) for better exploration in RL.

Study shows DNNs can recover functions with fewer samples than model parameters at overparameterization.

Optimistic Q-learning reduces sample complexity for systems with known dynamics.

The optimistic limit is the mathematical formulation of the classical limit which is a physical method to expect the actual limit by using saddle point method of certain potential function. The original optimistic limit of the Kashaev invariant was formulated by Yokota, and a modified formulation was suggested by the a…

OPSRL algorithm reduces regret with few samples in reinforcement learning.

Model-based Bayesian Reinforcement Learning (BRL) allows a found formalization of the problem of acting optimally while facing an unknown environment, i.e., avoiding the exploration-exploitation dilemma. However, algorithms explicitly addressing BRL suffer from such a combinatorial explosion that a large body of work r…

Characterizes concept classes for optimistic online learning.

We show how to take any two parameter-free online learning algorithms with different regret guarantees and obtain a single algorithm whose regret is the minimum of the two base algorithms. Our method is embarrassingly simple: just add the iterates. This trick can generate efficient algorithms that adapt to many norms s…

New algorithms reduce forecasting errors by leveraging optimistic learning and hinting.

We propose a new variant of AMSGrad, a popular adaptive gradient based optimization algorithm widely used for training deep neural networks. Our algorithm adds prior knowledge about the sequence of consecutive mini-batch gradients and leverages its underlying structure making the gradients sequentially predictable. By …

Information extracted from electrohysterography recordings could potentially prove to be an interesting additional source of information to estimate the risk on preterm birth. Recently, a large number of studies have reported near-perfect results to distinguish between recordings of patients that will deliver term or p…

Bayesian optimisation is a powerful tool to solve expensive black-box problems, but fails when the stationary assumption made on the objective function is strongly violated, which is the case in particular for ill-conditioned or discontinuous objectives. We tackle this problem by proposing a new Bayesian optimisation f…

Omega method mitigates noise in stochastic game optimization.

New method minimizes regret in AMDP with high probability.

New algorithm reduces reinforcement learning regret to sqrt(T) without strong dynamics assumptions.

Unified framework for model- and value-optimistic reinforcement learning.

Efficiently solves exploration-exploitation in LQR using Lagrangian relaxation.

New RL algorithm handles delayed feedback with posterior sampling.

Proposes H-UCRL for efficient model-based RL with sublinear regret.

A fundamental problem arising in many areas of machine learning is the evaluation of the likelihood of a given observation under different nominal distributions. Frequently, these nominal distributions are themselves estimated from data, which makes them susceptible to estimation errors. We thus propose to replace each…

New method for optimistic planning in MDPs using regularization.

Optimistic Hedge achieves optimal regret bounds in two-player zero-sum games.

New framework improves learning across multiple distributions.

Optimistic method adapted for faster convex-concave min-max problems.

This paper advances sample-efficient learning for partially observable RL by introducing B-stability and new algorithms.

RandQL is a new model-free algorithm for MDPs with a novel learning rate randomization approach.

Actor-critic methods, a type of model-free Reinforcement Learning, have been successfully applied to challenging tasks in continuous control, often achieving state-of-the art performance. However, wide-scale adoption of these methods in real-world domains is made difficult by their poor sample efficiency. We address th…

New method optimizes complex models with minimal data, proving global optimality.

This paper develops a model of reference-dependent assessment of subjective beliefs in which loss-averse people optimally choose the expectation as the reference point to balance the current felicity from the optimistic anticipation and the future disappointment from the realisation. The choice of over-optimism or over…