arXiv research

This paper optimizes slate decision systems for large action spaces.

Let M be a compact manifold. We show the identity component $\mathrm{Homeo}_0(M)$ of the group of self-homeomorphisms of M has a well-defined quasi-isometry type, and study its large scale geometry. Through examples, we relate this large scale geometry to both the topology of M and the dynamics of group actions on M. T…

In complex tasks, such as those with large combinatorial action spaces, random exploration may be too inefficient to achieve meaningful learning progress. In this work, we use a curriculum of progressively growing action spaces to accelerate learning. We assume the environment is out of our control, but that the agent …

Transformers learn to predict chess moves with surprising accuracy and strength.

BanditLP optimizes personalized recommendations for large-scale systems.

New algorithm for context bandits with continuous actions.

Most model-free reinforcement learning methods leverage state representations (embeddings) for generalization, but either ignore structure in the space of actions or assume the structure is provided a priori. We show how a policy can be decomposed into a component that acts in a low-dimensional space of action represen…

Efficiently plans large MDPs with weak function approximations.

Interactive machine learning improves learning efficiency with user input.

Practical algorithm for contextual bandits with large action spaces.

Solves action selection for large spaces in RL, achieving near-optimal performance.

In the study manifolds of Ricci curvature bounded below, a stumbling obstruction is the lack of links between large-scale geometry and small-scale geometry at a fixed reference point. There have been few links (volume, dimension) when the unit ball at the point is not collapsed, that is, $\mathrm{vol}(B_1(p))\ge v>0$. …

Stochastic Q-learning tackles large action spaces with reduced computation.

Being able to reason in an environment with a large number of discrete actions is essential to bringing reinforcement learning to a larger class of problems. Recommender systems, industrial plants and language models are only some of the many real-world tasks involving large numbers of discrete actions for which curren…

A CNN-DRL model improves learning in finance with scalable actions.

Efficient RL in large POMDPs with latent determinism and embeddings.

New RL method reduces sample complexity for large state-action spaces.

For a discrete metric space (or more generally a large scale space) $X$ and an action of a group $G$ on $X$ by coarse equivalences, we define a type of coarse quotient space $X_G$, which agrees up to coarse equivalence with the orbit space $X/G$ when $G$ is finite. We then restrict our attention to what we call coarsel…

Efficient algorithms for contextual bandits with smooth regret in continuous action spaces.

We consider Markov Decision Processes (MDPs) where the rewards are unknown and may change in an adversarial manner. We provide an algorithm that achieves state-of-the-art regret bound of $O( \sqrt{τ(\ln|S|+\ln|A|)T}\ln(T))$, where $S$ is the state space, $A$ is the action space, $τ$ is the mixing time of the MDP, and $…

In many real-world sequential decision making problems, the number of available actions (decisions) can vary over time. While problems like catastrophic forgetting, changing transition dynamics, changing rewards functions, etc. have been well-studied in the lifelong learning literature, the setting where the action set…

HyperAgent improves RL exploration in large-scale problems.

Reinforcement learning (RL) has recently been introduced to interactive recommender systems (IRS) because of its nature of learning from dynamic interactions and planning for long-run performance. As IRS is always with thousands of items to recommend (i.e., thousands of actions), most existing RL-based methods, however…

Survey of group actions on hyperbolic spaces, focusing on mapping class groups and Out(F_n).

For large-scale industrial processes under closed-loop control, process dynamics directly resulting from control action are typical characteristics and may show different behaviors between real faults and normal changes of operating conditions. However, conventional distributed monitoring approaches do not consider the…

This paper improves reinforcement learning efficiency for large-scale MDPs.

FPGs use structure to improve policy learning in complex tasks.

Let $G$ be a group acting properly and by isometries on a metric space $X$; it follows that the quotient or orbit space $X/G$ is also a metric space. We study the Vietoris-Rips and Čech complexes of $X/G$. Whereas (co)homology theories for metric spaces let the scale parameter of a Vietoris-Rips or Čech complex go to z…

The sample inefficiency of standard deep reinforcement learning methods precludes their application to many real-world problems. Methods which leverage human demonstrations require fewer samples but have been researched less. As demonstrated in the computer vision and natural language processing communities, large-scal…

New algorithm allows IGL to work with action-inclusive feedback.

Study evaluates machine learning methods for large-scale network reliability, revealing ANN's and PR's performance.

We consider the Markov Decision Process (MDP) of selecting a subset of items at each step, termed the Select-MDP (S-MDP). The large state and action spaces of S-MDPs make them intractable to solve with typical reinforcement learning (RL) algorithms especially when the number of items is huge. In this paper, we present …

Collective phenomena with universal properties have been observed in many complex systems with a large number of components. Here we present a microscopic model of the emergence of scaling behavior in such systems, where the interaction dynamics between individual components is mediated by a global variable making the …

New method reduces bias in learning from large action spaces using selective importance sampling.

The use of Reinforcement Learning in real-world scenarios is strongly limited by issues of scale. Most RL learning algorithms are unable to deal with problems composed of hundreds or sometimes even dozens of possible actions, and therefore cannot be applied to many real-world problems. We consider the RL problem in the…

Social media plays a major role during and after major natural disasters (e.g., hurricanes, large-scale fires, etc.), as people ``on the ground'' post useful information on what is actually happening. Given the large amounts of posts, a major challenge is identifying the information that is useful and actionable. Emerg…

Large-scale public datasets have been shown to benefit research in multiple areas of modern artificial intelligence. For decision-making research that requires human data, high-quality datasets serve as important benchmarks to facilitate the development of new methods by providing a common reproducible standard. Many h…

LLM extracts actionable insights from customer reviews.

Machine learning finds a compact fixed point action for SU(3) gauge theory.

We adopt Deep Reinforcement Learning algorithms to design trading strategies for continuous futures contracts. Both discrete and continuous action spaces are considered and volatility scaling is incorporated to create reward functions which scale trade positions based on market volatility. We test our algorithms on the…

Method uses ANN to estimate incentive salience from large behavioral data.

Law explains how LLMs learn to predict next tokens.

Online retailers execute a very large number of price updates when compared to brick-and-mortar stores. Even a few mis-priced items can have a significant business impact and result in a loss of customer trust. Early detection of anomalies in an automated real-time fashion is an important part of such a pricing system.…

ABS dynamically adjusts batch size based on policy stability, improving RL performance.

New RL method reduces sample complexity for large policy spaces.

In decision making problems for continuous state and action spaces, linear dynamical models are widely employed. Specifically, policies for stochastic linear systems subject to quadratic cost functions capture a large number of applications in reinforcement learning. Selected randomized policies have been studied in th…

The paper tackles counterfactual learning for stochastic policies with continuous actions.

Q($Δ$)-Learning improves Q-Learning by separating action-value functions into different time scales.