arXiv research

Efficient algorithms for planning in cooperative multi-agent reinforcement learning with combinatorial action spaces.

We consider the problem of online combinatorial optimization under semi-bandit feedback, where a learner has to repeatedly pick actions from a combinatorial decision set in order to minimize the total losses associated with its decisions. After making each decision, the learner observes the losses associated with its a…

Algorithm identifies best arm in combinatorial bandits with semi-bandit feedback.

Algorithm optimizes bandit decisions with changing action sets using Gaussian processes.

New algorithm identifies optimal actions in large reward spaces efficiently.

The paper tackles combinatorial pure exploration with various feedback structures and proposes efficient algorithms.

This paper extends combinatorial semi-bandits to graph feedback, improving regret bounds.

A framework for reinforcement learning tackles CVRP with competitive results.

We use the combinatorial harmonic map theory to study the isometric actions of discrete groups on Hadamard spaces. Given a finitely generated group acting by automorphisms, properly discontinuously and cofinitely on a simplicial complex and its isometric action on a Hadamard space, we formulate criterions for the actio…

Fixed point sets of certain group actions are contractible.

Study optimal arms in combinatorial bandits with semi-bandit feedback and finite budget.

The paper studies quaternionic structures on GKM graphs and their relation to torus actions on quaternionic projective spaces.

We present and study a partial-information model of online learning, where a decision maker repeatedly chooses from a finite set of actions, and observes some subset of the associated losses. This naturally models several situations where the losses of different actions are related, and knowing the loss of one action p…

New method for evaluating and learning in complex decision-making scenarios.

Simplifies large action space bandits by selecting representative actions.

Math verifies Aganagic's proposal for Khovanov homology.

In complex tasks, such as those with large combinatorial action spaces, random exploration may be too inefficient to achieve meaningful learning progress. In this work, we use a curriculum of progressively growing action spaces to accelerate learning. We assume the environment is out of our control, but that the agent …

Improved statistical efficiency of Thompson Sampling for combinatorial semi-bandits.

In many practical problems, a learning agent may want to learn the best action in hindsight without ever taking a bad action, which is significantly worse than the default production action. In general, this is impossible because the agent has to explore unknown actions, some of which can be bad, to learn better action…

We give examples of symplectic actions of a cyclic group, inducing a trivial action on homology, on four-manifolds that admit Hamiltonian circle actions, and show that they do not extend to Hamiltonian circle actions. Our work applies holomorphic methods to extend combinatorial tools developed for circle actions to stu…

SRL embeds combinatorial optimization into RL for better decision-making.

Paper solves no-swap regret minimization for combinatorial bandits with polylogarithmic dependence on N.

Paper tackles combinatorial reinforcement learning with preference feedback.

Deep RL learns to construct objects from 2D images by avoiding brick overlaps.

We propose a sample-efficient alternative for importance weighting for situations where one only has sample access to the probability distribution that generates the observations. Our new method, called Geometric Resampling (GR), is described and analyzed in the context of online combinatorial optimization under semi-b…

KG-A2C agent learns natural language IF games by reasoning and constraining action spaces.

We describe an algorithm for the enumeration of (candidates of) vertex-transitive combinatorial $d$-manifolds. With an implementation of our algorithm, we determine, up to combinatorial equivalence, all combinatorial manifolds with a vertex-transitive automorphism group on $n\leq 13$ vertices. With the exception of act…

CRB tackles rising rewards in combinatorial online learning.

New algorithm for combinatorial bandit problems reduces regret.

Researchers compute spin structures on hyperelliptic curves using braid groups.

We address online combinatorial optimization when the player has a prior over the adversary's sequence of losses. In this framework, Russo and Van Roy proposed an information-theoretic analysis of Thompson Sampling based on the information ratio, resulting in optimal worst-case regret bounds. In this paper we introduce…

We address online linear optimization problems when the possible actions of the decision maker are represented by binary vectors. The regret of the decision maker is the difference between her realized loss and the best loss she would have achieved by picking, in hindsight, the best possible action. Our goal is to unde…

New algorithms tackle adversarial combinatorial bandits with switching costs.

The paper surveys some new results and open problems connected with such fundamental combinatorial concepts as polytopes, simplicial complexes, cubical complexes, and subspace arrangements. Particular attention is paid to the case of simplicial and cubical subdivisions of manifolds and, especially, spheres. We describe…

A transitive smooth action of a connected Lie group G on a manifold M is called almost primitive (resp. primitive) if G doesn't contain any proper subgroup (resp. any proper normal subgroup) whose induced action on M is transitive as well. The aim of the present work is to investigate some combinatory properties of sym…

Optimistic covariance-adaptive algorithms improve combinatorial semi-bandits regret.

We propose a new framework for designing estimators for off-policy evaluation in contextual bandits. Our approach is based on the asymptotically optimal doubly robust estimator, but we shrink the importance weights to minimize a bound on the mean squared error, which results in a better bias-variance tradeoff in finite…

Study of group actions on CAT(0) cube complexes, focusing on marked length spectra.

Solves action selection for large spaces in RL, achieving near-optimal performance.

Improved sample complexity for contextual combinatorial semi-bandits with sparse rewards.

We study the problem of learning sequential decision-making policies in settings with multiple state-action representations. Such settings naturally arise in many domains, such as planning (e.g., multiple integer programming formulations) and various combinatorial optimization problems (e.g., those with both integer pr…

We make a few observations on the absence of geometric and topological rigidity for acylindrically hyperbolic and relatively hyperbolic groups. In particular, we demonstrate the lack of a well-defined limit set for acylindrical actions on hyperbolic spaces, even under the assumption of universality. We also prove a sta…

New framework tackles submodular welfare with multi-agent combinatorial bandits.

We introduce a new online learning framework where, at each trial, the learner is required to select a subset of actions from a given known action set. Each action is associated with an energy value, a reward and a cost. The sum of the energies of the actions selected cannot exceed a given energy budget. The goal is to…

New experimental design minimizes regret in bandits.

We give a complete classification of irreducible symmetric spaces for which there exist proper SL(2,R)-actions as isometries, using the criterion for proper actions by T. Kobayashi [Math. Ann. '89] and combinatorial techniques of nilpotent orbits. In particular, we classify irreducible symmetric spaces that admit surfa…

This paper has been withdrawn by the author. Improved versions (arXiv:1109.5548 and arXiv:0708.4190) are accepted.

A new method learns action representations for reinforcement learning.