arXiv research

We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning. Optimistic approaches presented in the literature apply an optimistic boost to the value estimate at each state-action pair and select actions that are greedy with respect to the resulting optimistic value f…

Optimistic estimate predicts best fitting performance of nonlinear models.

Unified framework for model- and value-optimistic reinforcement learning.

The optimistic limit is the mathematical formulation of the classical limit which is a physical method to expect the actual limit by using saddle point method of certain potential function. The original optimistic limit of the Kashaev invariant was formulated by Yokota, and a modified formulation was suggested by the a…

New method for optimistic planning in MDPs using regularization.

Kernel-based function approximation improves reinforcement learning performance.

Optimistic NPG improves policy optimization in online RL with efficient sample complexity.

StoSOO optimistically maximizes noisy, locally smooth functions.

Optimistic PPO variant solves linear MDPs with improved regret bound.

Optimistic bounds for multi-output learning using self-bounding Lipschitz condition.

New method minimizes regret in AMDP with high probability.

Improved regret bounds for online convex optimization under stochastic and adversarial settings.

A generalized optimistic method for saddle point problems with improved complexity.

Optimistic method adapted for faster convex-concave min-max problems.

Optimistic algorithm reduces regret and constraint violations in online convex optimization with adversarial constraints.

Paper proposes an optimistic likelihood approximation for nonparametric likelihoods.

Bayesian classifier improves robustness with optimistic score ratio.

Proposes Optimistic Pessimistically Initialised Q-Learning (OPIQ) for better exploration in RL.

We show that the optimistic limits of the colored Jones polynomials of the hyperbolic knots coincide with the optimistic limits of the Kashaev invariants modulo $4π^2$.

Novel approach to universal online learning for bounded losses, closing open problems.

Paper tackles constrained bandit problems with a new learning framework.

New algorithm reduces reinforcement learning regret to sqrt(d^3T).

Optimistic search speeds up change point detection in large datasets.

For the potential function of a link diagram induced by the optimistic limit of the colored Jones polynomial, we show the existence of a solution of the hyperbolicity equations by directly constructing it. This construction is based on the shadow-coloring of the conjugation quandle induced by a boundary-parabolic repre…

New algorithm reduces regret in sequential decision-making problems.

PROPO tackles non-stationary MDPs with efficient policy optimization.

Optimistic likelihoods improve classification accuracy by considering nearby distributions.

Characterizes concept classes for optimistic online learning.

Optimistic RL algorithms are simplified for deep RL with competitive performance.

We show how to take any two parameter-free online learning algorithms with different regret guarantees and obtain a single algorithm whose regret is the minimum of the two base algorithms. Our method is embarrassingly simple: just add the iterates. This trick can generate efficient algorithms that adapt to many norms s…

Optimistic algorithms achieve logarithmic regret bounds for MDPs without diameter dependence.

New algorithms reduce forecasting errors by leveraging optimistic learning and hinting.

New algorithm reduces reinforcement learning regret to sqrt(T) without strong dynamics assumptions.

Omega method mitigates noise in stochastic game optimization.

We consider the problem of minimizing a smooth convex function by reducing the optimization to computing the Nash equilibrium of a particular zero-sum convex-concave game. Zero-sum games can be solved using online learning dynamics, where a classical technique involves simulating two no-regret algorithms that play agai…

Model-based Bayesian Reinforcement Learning (BRL) allows a found formalization of the problem of acting optimally while facing an unknown environment, i.e., avoiding the exploration-exploitation dilemma. However, algorithms explicitly addressing BRL suffer from such a combinatorial explosion that a large body of work r…

New RL algorithm tackles nonstationary MDPs with linear approximations and varying rewards.

Efficiently solves exploration-exploitation in LQR using Lagrangian relaxation.

New algorithm achieves data-dependent regret bounds in MDPs with unknown transitions.

Study shows DNNs can recover functions with fewer samples than model parameters at overparameterization.

Proposes H-UCRL for efficient model-based RL with sublinear regret.

Bayesian optimisation (BO) is a well-known efficient algorithm for finding the global optimum of expensive, black-box functions. The current practical BO algorithms have regret bounds ranging from $\mathcal{O}(\frac{logN}{\sqrt{N}})$ to $\mathcal O(e^{-\sqrt{N}})$, where $N$ is the number of evaluations. This paper exp…

Optimistic Hedge achieves optimal regret bounds in two-player zero-sum games.

Optimizes algorithms for non-concave bandit problems.

New algorithm reduces regret for logistic bandits without $κ$ dependency.

The paper analyzes bias in sample means of multi-armed bandits.

We derive an alternative proof for the regret of Thompson sampling (\ts) in the stochastic linear bandit setting. While we obtain a regret bound of order $\widetilde{O}(d^{3/2}\sqrt{T})$ as in previous results, the proof sheds new light on the functioning of the \ts. We leverage on the structure of the problem to show …

Bayesian optimization is a powerful global optimization technique for expensive black-box functions. One of its shortcomings is that it requires auxiliary optimization of an acquisition function at each iteration. This auxiliary optimization can be costly and very hard to carry out in practice. Moreover, it creates ser…