arXiv research

Solves open problem on universally consistent online learning with unbounded losses.

Novel approach to universal online learning for bounded losses, closing open problems.

New findings on universal learning in contextual bandits with adversarial rewards.

We address the issue of limit cycling behavior in training Generative Adversarial Networks and propose the use of Optimistic Mirror Decent (OMD) for training Wasserstein GANs. Recent theoretical results have shown that optimistic mirror decent (OMD) can enjoy faster regret rates in the context of zero-sum games. WGANs …

This work initiates a general study of learning and generalization without the i.i.d. assumption, starting from first principles. While the traditional approach to statistical learning theory typically relies on standard assumptions from probability theory (e.g., i.i.d. or stationary ergodic), in this work we are inter…

New algorithm achieves consistent learning from context in bandit problems.

Bayesian classifier improves robustness with optimistic score ratio.

We show that the optimistic limits of the colored Jones polynomials of the hyperbolic knots coincide with the optimistic limits of the Kashaev invariants modulo $4π^2$.

Proposes Optimistic Pessimistically Initialised Q-Learning (OPIQ) for better exploration in RL.

We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning. Optimistic approaches presented in the literature apply an optimistic boost to the value estimate at each state-action pair and select actions that are greedy with respect to the resulting optimistic value f…

The optimistic limit is the mathematical formulation of the classical limit which is a physical method to expect the actual limit by using saddle point method of certain potential function. The original optimistic limit of the Kashaev invariant was formulated by Yokota, and a modified formulation was suggested by the a…

Optimistic estimate predicts best fitting performance of nonlinear models.

New algorithm reduces regret in sequential decision-making problems.

Optimistic NPG improves policy optimization in online RL with efficient sample complexity.

Characterizes concept classes for optimistic online learning.

We show how to take any two parameter-free online learning algorithms with different regret guarantees and obtain a single algorithm whose regret is the minimum of the two base algorithms. Our method is embarrassingly simple: just add the iterates. This trick can generate efficient algorithms that adapt to many norms s…

New algorithms reduce forecasting errors by leveraging optimistic learning and hinting.

Omega method mitigates noise in stochastic game optimization.

New method minimizes regret in AMDP with high probability.

Unified framework for model- and value-optimistic reinforcement learning.

New framework tackles multi-source domain adaptation with optimism and consistency.

It is well known that in stochastic multi-armed bandits (MAB), the sample mean of an arm is typically not an unbiased estimator of its true mean. In this paper, we decouple three different sources of this selection bias: adaptive \emph{sampling} of arms, adaptive \emph{stopping} of the experiment, and adaptively \emph{…

Efficiently solves exploration-exploitation in LQR using Lagrangian relaxation.

New framework uses tempered optimism to handle imperfect experts in online learning.

We consider model-based reinforcement learning in finite Markov De- cision Processes (MDPs), focussing on so-called optimistic strategies. In MDPs, optimism can be implemented by carrying out extended value it- erations under a constraint of consistency with the estimated model tran- sition probabilities. The UCRL2 alg…

Proposes H-UCRL for efficient model-based RL with sublinear regret.

We extend a recently proposed 1-nearest-neighbor based multiclass learning algorithm and prove that our modification is universally strongly Bayes-consistent in all metric spaces admitting any such learner, making it an "optimistically universal" Bayes-consistent learner. This is the first learning algorithm known to e…

A fundamental problem arising in many areas of machine learning is the evaluation of the likelihood of a given observation under different nominal distributions. Frequently, these nominal distributions are themselves estimated from data, which makes them susceptible to estimation errors. We thus propose to replace each…

In the present paper a model of a market consisting of real and financial interacting sectors is studied. Agents populating the stock market are assumed to be not able to observe the true underlying fundamental, and their beliefs are biased by either optimism or pessimism. Depending on the relevance they give to belief…

New method for optimistic planning in MDPs using regularization.

Off-policy reinforcement learning (RL) using a fixed offline dataset of logged interactions is an important consideration in real world applications. This paper studies offline RL using the DQN replay dataset comprising the entire replay experience of a DQN agent on 60 Atari 2600 games. We demonstrate that recent off-p…

Optimistic Hedge achieves optimal regret bounds in two-player zero-sum games.

Optimistic method adapted for faster convex-concave min-max problems.

This paper develops a model of reference-dependent assessment of subjective beliefs in which loss-averse people optimally choose the expectation as the reference point to balance the current felicity from the optimistic anticipation and the future disappointment from the realisation. The choice of over-optimism or over…

Optimistic PPO variant solves linear MDPs with improved regret bound.

Study optimizes linear regression analysis for high-dimensional settings.

Optimistic bounds for multi-output learning using self-bounding Lipschitz condition.

In this paper, we prove some convergence results of a special case of optimistic policy iteration algorithm for stochastic shortest path problem. We consider both Monte Carlo and $TD(λ)$ methods for the policy evaluation step under the condition that the termination state will eventually be reached almost surely.

Accelerates policy optimization in RL with optimistic and adaptive updates.

Unified framework for model-based RL with sample complexity guarantees.

Optimistic pricing algorithm handles online dynamic pricing with censored demand.

Yokota suggested an optimistic limit method of the Kashaev invariants of hyperbolic knots and showed it determines the complex volumes of the knots. His method is very effective and gives almost combinatorial method of calculating the complex volumes. However, to describe the triangulation of the knot complement, he re…

New algorithms solve linear bandits in high dimensions efficiently.

Safe linear bandits over unknown polytopes avoid safety violations and suboptimal actions.

Optimistic RL algorithms are simplified for deep RL with competitive performance.

New algorithms for regression with adversarial responses on various metric spaces.

The likelihood function is a fundamental component in Bayesian statistics. However, evaluating the likelihood of an observation is computationally intractable in many applications. In this paper, we propose a non-parametric approximation of the likelihood that identifies a probability measure which lies in the neighbor…

A generalized optimistic method for saddle point problems with improved complexity.