arXiv research

Study optimizes linear regression analysis for high-dimensional settings.

Optimistic Hedge achieves optimal regret bounds in two-player zero-sum games.

New learning rules achieve optimal sample complexity for weakly supervised classification.

Develops new algorithms for solving root-finding problems in large-scale settings.

Optimistic method adapted for faster convex-concave min-max problems.

This paper establishes that optimistic algorithms attain gap-dependent and non-asymptotic logarithmic regret for episodic MDPs. In contrast to prior work, our bounds do not suffer a dependence on diameter-like quantities or ergodicity, and smoothly interpolate between the gap dependent logarithmic-regret, and the $\wid…

New algorithm AG-OG optimizes separable convex-concave problems efficiently.

We address the issue of limit cycling behavior in training Generative Adversarial Networks and propose the use of Optimistic Mirror Decent (OMD) for training Wasserstein GANs. Recent theoretical results have shown that optimistic mirror decent (OMD) can enjoy faster regret rates in the context of zero-sum games. WGANs …

Optimistic search speeds up change point detection in large datasets.

We consider the problem of minimizing a smooth convex function by reducing the optimization to computing the Nash equilibrium of a particular zero-sum convex-concave game. Zero-sum games can be solved using online learning dynamics, where a classical technique involves simulating two no-regret algorithms that play agai…

RandQL is a new model-free algorithm for MDPs with a novel learning rate randomization approach.

Develops an accelerated algorithm for solving nonmonotone generalized equations.

Optimistic algorithm reduces regret and constraint violations in online convex optimization with adversarial constraints.

Bayesian classifier improves robustness with optimistic score ratio.

A generalized optimistic method for saddle point problems with improved complexity.

We show that the optimistic limits of the colored Jones polynomials of the hyperbolic knots coincide with the optimistic limits of the Kashaev invariants modulo $4π^2$.

Proposes Optimistic Pessimistically Initialised Q-Learning (OPIQ) for better exploration in RL.

We study the iteration complexity of the optimistic gradient descent-ascent (OGDA) method and the extra-gradient (EG) method for finding a saddle point of a convex-concave unconstrained min-max problem. To do so, we first show that both OGDA and EG can be interpreted as approximate variants of the proximal point method…

We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning. Optimistic approaches presented in the literature apply an optimistic boost to the value estimate at each state-action pair and select actions that are greedy with respect to the resulting optimistic value f…

Improved convergence rates for saddle-point optimization algorithms.

The optimistic limit is the mathematical formulation of the classical limit which is a physical method to expect the actual limit by using saddle point method of certain potential function. The original optimistic limit of the Kashaev invariant was formulated by Yokota, and a modified formulation was suggested by the a…

Optimistic estimate predicts best fitting performance of nonlinear models.

Algorithm learns to bid in auctions with shilling, masking real bids.

Bayesian optimization is a powerful global optimization technique for expensive black-box functions. One of its shortcomings is that it requires auxiliary optimization of an acquisition function at each iteration. This auxiliary optimization can be costly and very hard to carry out in practice. Moreover, it creates ser…

We consider the use of no-regret algorithms to compute equilibria for particular classes of convex-concave games. While standard regret bounds would lead to convergence rates on the order of $O(T^{-1/2})$, recent work \citep{RS13,SALS15} has established $O(1/T)$ rates by taking advantage of a particular class of optimi…

We study the problem of online learning with non-convex losses, where the learner has access to an offline optimization oracle. We show that the classical Follow the Perturbed Leader (FTPL) algorithm achieves optimal regret rate of $O(T^{-1/2})$ in this setting. This improves upon the previous best-known regret rate of…

New algorithm reduces regret in sequential decision-making problems.

Optimistic NPG improves policy optimization in online RL with efficient sample complexity.

Characterizes concept classes for optimistic online learning.

The CFR framework has been a powerful tool for solving large-scale extensive-form games in practice. However, the theoretical rate at which past CFR-based algorithms converge to the Nash equilibrium is on the order of $O(T^{-1/2})$, where $T$ is the number of iterations. In contrast, first-order methods can be used to …

We show how to take any two parameter-free online learning algorithms with different regret guarantees and obtain a single algorithm whose regret is the minimum of the two base algorithms. Our method is embarrassingly simple: just add the iterates. This trick can generate efficient algorithms that adapt to many norms s…

New algorithms reduce forecasting errors by leveraging optimistic learning and hinting.

Optimizes algorithms for non-concave bandit problems.

Omega method mitigates noise in stochastic game optimization.

New method minimizes regret in AMDP with high probability.

Unified framework for model- and value-optimistic reinforcement learning.

Growth rate of the world Growth Domestic Product (GDP) is analysed to determine possible pathways of the future economic growth. The analysis is based on using the latest data of the World Bank and it reveals that the growth rate between 1960 and 2014 was following a trajectory approaching asymptotically a constant val…

It is well known that in stochastic multi-armed bandits (MAB), the sample mean of an arm is typically not an unbiased estimator of its true mean. In this paper, we decouple three different sources of this selection bias: adaptive \emph{sampling} of arms, adaptive \emph{stopping} of the experiment, and adaptively \emph{…

Efficiently solves exploration-exploitation in LQR using Lagrangian relaxation.

Proposes H-UCRL for efficient model-based RL with sublinear regret.

A fundamental problem arising in many areas of machine learning is the evaluation of the likelihood of a given observation under different nominal distributions. Frequently, these nominal distributions are themselves estimated from data, which makes them susceptible to estimation errors. We thus propose to replace each…

New method for optimistic planning in MDPs using regularization.

Despite remarkable empirical success, the training dynamics of generative adversarial networks (GAN), which involves solving a minimax game using stochastic gradients, is still poorly understood. In this work, we analyze last-iterate convergence of simultaneous gradient descent (simGD) and its variants under the assump…

New learning dynamics achieve fast convergence in games without needing to know utility scales.

This paper develops a model of reference-dependent assessment of subjective beliefs in which loss-averse people optimally choose the expectation as the reference point to balance the current felicity from the optimistic anticipation and the future disappointment from the realisation. The choice of over-optimism or over…

Optimistic PPO variant solves linear MDPs with improved regret bound.

New framework for reinforcement learning with sporadic state observations.

New framework improves learning across multiple distributions.