arXiv research

New study shows exponential lower bound for RL even with constant suboptimality gap.

New algorithm reduces MDP regret by accounting for state suboptimality gaps and variance.

Fine-grained gap-dependent regret bounds for reinforcement learning.

Paper explores PG for MCR, finding suboptimal policies but providing bounds.

Study quantile multi-armed bandits for identifying the best arm with a specified quantile level.

GNA optimally identifies the best arm with small gaps.

New method reduces regret and communication costs in federated Q-learning.

Improved reinforcement learning for episodes with varying action sets.

Develops a Best-of-Both-Worlds algorithm for linear contextual bandits with Tsallis entropy.

Privacy-preserving reinforcement learning from human feedback using decoupled reward modeling.

New algorithm reduces regret by focusing on near-optimal actions.

This paper improves Q-learning bounds using reference-advantage decomposition.

AceIRL learns expert reward from active exploration.

In this paper, we propose several improvements on the block-coordinate Frank-Wolfe (BCFW) algorithm from Lacoste-Julien et al. (2013) recently used to optimize the structured support vector machine (SSVM) objective in the context of structured prediction, though it has wider applications. The key intuition behind our i…

Improved adaptive rates for Lipschitz bandit problem.

A stochastic combinatorial semi-bandit is an online learning problem where at each step a learning agent chooses a subset of ground items subject to constraints, and then observes stochastic weights of these items and receives their sum as a payoff. In this paper, we close the problem of computationally and sample effi…

Study best arm identification with contextual info, achieving optimal misidentification probability.

Improved online Q-learning for MDPs with concentration bounds.

Safe linear bandits over unknown polytopes avoid safety violations and suboptimal actions.

Boltzmann exploration is a classic strategy for sequential decision-making under uncertainty, and is one of the most standard tools in Reinforcement Learning (RL). Despite its widespread use, there is virtually no theoretical understanding about the limitations or the actual benefits of this exploration scheme. Does it…

This paper focuses on projection-free methods for solving smooth Online Convex Optimization (OCO) problems. Existing projection-free methods either achieve suboptimal regret bounds or have high per-iteration computational costs. To fill this gap, two efficient projection-free online methods called ORGFW and MORGFW are …

Unified framework for corruption-robust linear bandits with optimal gap-dependent misspecification bounds.

The recognition network in deep latent variable models such as variational autoencoders (VAEs) relies on amortized inference for efficient posterior approximation that can scale up to large datasets. However, this technique has also been demonstrated to select suboptimal variational parameters, often resulting in consi…

New algorithm optimizes multi-armed bandit performance in stochastic and adversarial settings.

GP-UCB performs suboptimally under certain conditions, as shown by a new regret lower bound.

Two entropy measures quantify suboptimal portfolio performance.

AdMRL improves meta-reinforcement learning by minimizing worst-case sub-optimality gap.

New algorithm optimizes best arm identification with minimal regret.

Policy gradients methods apply to complex, poorly understood, control problems by performing stochastic gradient descent over a parameterized class of polices. Unfortunately, even for simple control problems solvable by standard dynamic programming techniques, policy gradient algorithms face non-convex optimization pro…

Novel IRL method identifies suboptimal medical decisions in ICU data.

This paper closes the gap on matching pursuit's convergence rate.

New framework improves learning across multiple distributions.

We propose a faster and more accurate method for learning classification trees.

Recovering a low-rank tensor from incomplete information is a recurring problem in signal processing and machine learning. The most popular convex relaxation of this problem minimizes the sum of the nuclear norms of the unfoldings of the tensor. We show that this approach can be substantially suboptimal: reliably recov…

Algorithm reduces regret in multi-player bandits with unknown collision rewards.

We introduce a novel theoretical framework for Return On Investment (ROI) maximization in repeated decision-making. Our setting is motivated by the use case of companies that regularly receive proposals for technological innovations and want to quickly decide whether they are worth implementing. We design an algorithm …

New suboptimal algorithm for best subset selection in high-dimensional data.

Optimistic covariance-adaptive algorithms improve combinatorial semi-bandits regret.

The traditional sparse modeling approach, when applied to inverse problems with large data such as images, essentially assumes a sparse model for small overlapping data patches. While producing state-of-the-art results, this methodology is suboptimal, as it does not attempt to model the entire global signal in any mean…

We analyze the regret of combinatorial Thompson sampling (CTS) for the combinatorial multi-armed bandit with probabilistically triggered arms under the semi-bandit feedback setting. We assume that the learner has access to an exact optimization oracle but does not know the expected base arm outcomes beforehand. When th…

We study local complexity measures for stochastic convex optimization problems, providing a local minimax theory analogous to that of Hájek and Le Cam for classical statistical problems. We give complementary optimality results, developing fully online methods that adaptively achieve optimal convergence guarantees. Our…

New method for semiparametric bandits reduces regret to optimal levels.

The maximum a posteriori (MAP) configuration of binary variable models with submodular graph-structured energy functions can be found efficiently and exactly by graph cuts. Max-product belief propagation (MP) has been shown to be suboptimal on this class of energy functions by a canonical counterexample where MP conver…

New research shows Byzantine failures hurt generalization more than data poisoning in robust distributed learning.

The paper introduces a new measure of robustness for partially identifiable risks.

Proves minimax sample complexity for turn-based stochastic games.

Develops anytime-valid stopping rules for SGD based on observed trajectory.

Aims to create safe reinforcement learning policies by considering individual harm.