arXiv research

New reinforcement learning algorithm achieves instance-optimal sample complexity.

Methods for learning to search for structured prediction typically imitate a reference policy, with existing theoretical guarantees demonstrating low regret compared to that reference. This is unsatisfactory in many applications where the reference policy is suboptimal and the goal of learning is to improve upon it. Ca…

We develop algorithms with low regret for learning episodic Markov decision processes based on kernel approximation techniques. The algorithms are based on both the Upper Confidence Bound (UCB) as well as Posterior or Thompson Sampling (PSRL) philosophies, and work in the general setting of continuous state and action …

New method constructs confidence sets for GLMs via game theory.

A new algorithm learns optimal personalized treatment plans online with low regret.

Efficient binary sampling method for global optimization of univariate functions with low regret.

Study online learning in MDPs with aggregate bandit feedback, achieving low regret in both stochastic and adversarial settings.

Robust estimation methods find global minima efficiently via quasi-gradients.

Study tests feasibility of linear programs with bandit feedback.

This thesis analyzes MACL systems with low-regret learning algorithms for sequential decision making.

New algorithms reduce forecasting errors by leveraging optimistic learning and hinting.

A major challenge in contextual bandits is to design general-purpose algorithms that are both practically useful and theoretically well-founded. We present a new technique that has the empirical and computational advantages of realizability-based approaches combined with the flexibility of agnostic methods. Our algorit…

Transforms offline algorithms to online with low regret in random order model.

Paper finds algorithms with both low regret and high exploitation.

Study of repeated games with unobserved agent rewards using MAB framework.

New algorithms improve on bandit feedback in matrix games with unknown payoff matrices.

Paper proposes ZO-SMD for MERO, achieving optimal convergence rates.

LAFF algorithm balances adaptability and non-exploitability in repeated games.

Recent work on follow the perturbed leader (FTPL) algorithms for the adversarial multi-armed bandit problem has highlighted the role of the hazard rate of the distribution generating the perturbations. Assuming that the hazard rate is bounded, it is possible to provide regret analyses for a variety of FTPL algorithms f…

A method to minimize regret in multi-agent control systems with adversarial disturbances.

The paper tackles robust policy learning from multiple data sources.

Contextual bandits have the same exploration-exploitation trade-off as standard multi-armed bandits. On adding positive externalities that decay with time, this problem becomes much more difficult as wrong decisions at the start are hard to recover from. We explore existing policies in this setting and highlight their …

New algorithm reduces online learning regret for bounded recall games.

Proposes MRO to achieve uniformly low regret in distributionally robust learning.

We study online prediction where regret of the algorithm is measured against a benchmark defined via evolving constraints. This framework captures online prediction on graphs, as well as other prediction problems with combinatorial structure. A key aspect here is that finding the optimal benchmark predictor (even in hi…

Reinforcement learning (RL) in Markov decision processes (MDPs) with large state spaces is a challenging problem. The performance of standard RL algorithms degrades drastically with the dimensionality of state space. However, in practice, these large MDPs typically incorporate a latent or hidden low-dimensional structu…

New algorithms for linear bandits avoid norm knowledge, reducing regret.

New algorithm reduces online decision-making regret with efficient LP re-solving and parallel first-order method.

We consider online algorithms under both the competitive ratio criteria and the regret minimization one. Our main goal is to build a unified methodology that would be able to guarantee both criteria simultaneously. For a general class of online algorithms, namely any Metrical Task System (MTS), we show that one can sim…

Estimates rewards from a demonstrator's learning process.

How can we efficiently gather information to optimize an unknown function, when presented with multiple, mutually dependent information sources with different costs? For example, when optimizing a robotic system, intelligently trading off computer simulations and real robot testings can lead to significant savings. Exi…

Algorithm identifies Pareto front using multiple context directions and reuses exploration samples.

New algorithm reduces regret in online learning for piecewise continuous functions.

Framework reduces contextual bandit learning to offline regression with near-optimal regret.

New algorithm reduces regret from sqrt(T) to polylog(T) in stochastic contextual linear bandits.

In many scientific tasks we are interested in discovering whether there exist any correlations in our data. This raises many questions, such as how to reliably and interpretably measure correlation between a multivariate set of attributes, how to do so without having to make assumptions on distribution of the data or t…

OE2D framework reduces contextual bandits to offline regression for near-optimal regret.

Algorithm minimizes regret in predictive models influenced by their own predictions.

Algorithm extsc{Pedel} learns near-optimal policies efficiently on specific problems.

New control methods for systems with adversarial perturbations.

Adaptive KL-UCB algorithm for Markov and i.i.d. rewards.

AES improves policy gradient performance by adaptively selecting experience.

The paper develops algorithms for competitive RL in partially observable MGs.

Optimal multiclass U-calibration error found to be Θ(√KT).

We study a strategic version of the multi-armed bandit problem, where each arm is an individual strategic agent and we, the principal, pull one arm each round. When pulled, the arm receives some private reward $v_a$ and can choose an amount $x_a$ to pass on to the principal (keeping $v_a-x_a$ for itself). All non-pulle…

Algorithm identifies the best arm in linear bandits with high probability.

Self-reinforcing feedback loops in personalization systems are typically caused by users choosing from a limited set of alternatives presented systematically based on previous choices. We propose a Bayesian choice model built on Luce axioms that explicitly accounts for users' limited exposure to alternatives. Our model…

The paper tackles online learning with two types of losses and shows it's impossible without certain assumptions.