A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New method learns near-optimal policies with polynomial samples in A and H.
Many interesting real world domains involve reinforcement learning (RL) in partially observable environments. Efficient learning in such domains is important, but existing sample complexity bounds for partially observable RL are at least exponential in the episode length. We give, to our knowledge, the first partially …
Algorithm learns near-optimal policies for reward-mixing MDPs with few latent contexts.
New memory allocation scheme improves image generation performance.
We discuss memory models which are based on tensor decompositions using latent representations of entities and events. We show how episodic memory and semantic memory can be realized and discuss how new memory traces can be generated from sensory input: Existing memories are the basis for perception and new memories ar…
We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…
A graph bandit algorithm learns optimal paths on unknown graphs.
Paper closes the gap in MP-MAB problems with novel adaptive communication and exploration.
We formulate the problem of sampling and recovering clustered graph signal as a multi-armed bandit (MAB) problem. This formulation lends naturally to learning sampling strategies using the well-known gradient MAB algorithm. In particular, the sampling strategy is represented as a probability distribution over the indiv…
We study the exploration problem in episodic MDPs with rich observations generated from a small number of latent states. Under certain identifiability assumptions, we demonstrate how to estimate a mapping from the observations to latent states inductively through a sequence of regression and clustering steps -- where p…
A new bandit problem where experiments can be interrupted if results are not promising.
We extend Bayesian multi-armed bandit (MAB) algorithms beyond their original setting by making use of sequential Monte Carlo (SMC) methods. A MAB is a sequential decision making problem where the goal is to learn a policy that maximizes long term payoff, where only the reward of the executed action is observed. In the …
In this work, we study recommendation systems modelled as contextual multi-armed bandit (MAB) problems. We propose a graph-based recommendation system that learns and exploits the geometry of the user space to create meaningful clusters in the user domain. This reduces the dimensionality of the recommendation problem w…
Transfer and adaptation to new unknown environmental dynamics is a key challenge for reinforcement learning (RL). An even greater challenge is performing near-optimally in a single attempt at test time, possibly without access to dense rewards, which is not addressed by current methods that require multiple experience …
New algorithms detect changes in non-stationary MABs for better performance.
LC-SAC tackles non-stationary dynamics in reinforcement learning.
Trust-aware MAB improves learning performance by accounting for human deviation.
A survey is performed of various Multi-Armed Bandit (MAB) strategies in order to examine their performance in circumstances exhibiting non-stationary stochastic reward functions in conjunction with delayed feedback. We run several MAB simulations to simulate an online eCommerce platform for grocery pick up, optimizing …
New algorithm for multi-fidelity bandits reduces costs and improves regret.
New algorithm ensures consistent results in constrained MAB problems.
Paper proposes a method to optimize policies for diverse individuals using heterogeneous data.
INF-clip optimizes heavy-tailed MAB problems with improved performance.
The Multi-Armed Bandits (MAB) framework highlights the tension between acquiring new knowledge (Exploration) and leveraging available knowledge (Exploitation). In the classical MAB problem, a decision maker must choose an arm at each time step, upon which she receives a reward. The decision maker's objective is to maxi…
We extend the classic multi-armed bandit (MAB) model to the setting of noncompliance, where the arm pull is a mere instrument and the treatment applied may differ from it, which gives rise to the instrument-armed bandit (IAB) problem. The IAB setting is relevant whenever the experimental units are human since free will…
New MAB model for online caching costs.
New method improves robustness in partially observable domains by training against latent distribution shifts.
Chronological Causal Bandits (CCB) tackles dynamic causal decision-making.
New model for multi-armed bandits with growing arms.
Develops Thompson Sampling algorithms for mean-variance bandits.
In several realistic situations, an interactive learning agent can practice and refine its strategy before going on to be evaluated. For instance, consider a student preparing for a series of tests. She would typically take a few practice tests to know which areas she needs to improve upon. Based of the scores she obta…
New TS algorithms improve performance in non-stationary multi-armed bandit problems.
We propose an alternative framework to existing setups for controlling false alarms when multiple A/B tests are run over time. This setup arises in many practical applications, e.g. when pharmaceutical companies test new treatment options against control pills for different diseases, or when internet companies test the…
This paper deals with bandit online learning problems involving feedback of unknown delay that can emerge in multi-armed bandit (MAB) and bandit convex optimization (BCO) settings. MAB and BCO require only values of the objective function involved that become available through feedback, and are used to estimate the gra…
Incentivized exploration improves MAB performance under reward drift.
The paper optimizes risk-sensitive RL with CVaR, achieving near-minimax-optimal results.
New MAB model incentivizes user arm-pulling with self-reinforcing preferences.
New L2 regularization improves softmax MAB performance.
Planning has been very successful for control tasks with known environment dynamics. To leverage planning in unknown environments, the agent needs to learn the dynamics from interactions with the world. However, learning dynamics models that are accurate enough for planning has been a long-standing challenge, especiall…
A scheme robust to action erasures improves MAB performance.
This paper analyzes the multi-armed bandit model using path-integral methods.
Enhanced feedback model improves sample-efficiency in POMDPs.
A method for self-supervised representation learning in partially observable environments.
Paper proposes CVaR-TS for risk-constrained MAB problems.
We study a policy gradient method with L2 regularization for MAB problems.
Contextual multi-armed bandit (MAB) algorithms have been shown promising for maximizing cumulative rewards in sequential decision tasks such as news article recommendation systems, web page ad placement algorithms, and mobile health. However, most of the proposed contextual MAB algorithms assume linear relationships be…
The paper studies MAB problems with LDP to balance privacy and service quality.
Study preference-based reinforcement learning in episodic kernel MDPs.
This paper proposes a general framework of multi-armed bandit (MAB) processes by introducing a type of restrictions on the switches among arms evolving in continuous time. The Gittins index process is constructed for any single arm subject to the restrictions on switches and then the optimality of the corresponding Git…