A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New AIM algorithm optimizes exploration-exploitation in bandits.
Lecture notes on reinforcement learning using statistical methods.
This paper applies Thompson Sampling to asymmetric -stable bandits for financial and wireless data.
The explore{exploit dilemma is one of the central challenges in Reinforcement Learning (RL). Bayesian RL solves the dilemma by providing the agent with information in the form of a prior distribution over environments; however, full Bayesian planning is intractable. Planning with the mean MDP is a common myopic approxi…
New algorithms for batch decision-making with high-dimensional user data.
ASD algorithm maximizes model estimates by adaptively labeling points.
Reinforcement learning addresses the dilemma between exploration to find profitable actions and exploitation to act according to the best observations already made. Bandit problems are one such class of problems in stateless environments that represent this explore/exploit situation. We propose a learning algorithm for…
Hyper addresses the hyperparameter tuning challenge in RL, improving exploration efficiency and robustness.
Kernel-UCBVI algorithm balances exploration and exploitation in metric state-action spaces.
The exploration/exploitation (E/E) dilemma arises naturally in many subfields of Science. Multi-armed bandit problems formalize this dilemma in its canonical form. Most current research in this field focuses on generic solutions that can be applied to a wide range of problems. However, in practice, it is often the case…
Efficiently solves exploration-exploitation in LQR using Lagrangian relaxation.
Model-based Bayesian Reinforcement Learning (BRL) allows a found formalization of the problem of acting optimally while facing an unknown environment, i.e., avoiding the exploration-exploitation dilemma. However, algorithms explicitly addressing BRL suffer from such a combinatorial explosion that a large body of work r…
The paper analyzes CMDPs, balancing exploration and exploitation to avoid constraint violations.
Meta-SAC automatically tunes SAC's entropy temperature for better exploration.
New algorithm reduces RL complexity with low switching costs.
Thompson sampling is an efficient algorithm for sequential decision making, which exploits the posterior uncertainty to address the exploration-exploitation dilemma. There has been significant recent interest in integrating Bayesian neural networks into Thompson sampling. Most of these methods rely on global variable u…
We consider a novel stochastic multi-armed bandit problem called {\em good arm identification} (GAI), where a good arm is defined as an arm with expected reward greater than or equal to a given threshold. GAI is a pure-exploration problem that a single agent repeats a process of outputting an arm as soon as it is ident…
This paper considers stochastic bandits with side observations, a model that accounts for both the exploration/exploitation dilemma and relationships between arms. In this setting, after pulling an arm i, the decision maker also observes the rewards for some other actions related to i. We will see that this model is su…
Multiple machine learning and prediction models are often used for the same prediction or recommendation task. In our recent work, where we develop and deploy airline ancillary pricing models in an online setting, we found that among multiple pricing models developed, no one model clearly dominates other models for all…
We introduce Bayesian least-squares policy iteration (BLSPI), an off-policy, model-free, policy iteration algorithm that uses the Bayesian least-squares temporal-difference (BLSTD) learning algorithm to evaluate policies. An online variant of BLSPI has been also proposed, called randomised BLSPI (RBLSPI), that improves…
Study shows certainty equivalent policy minimizes regret in continuous-time systems.
We consider the exploration-exploitation dilemma in finite-horizon reinforcement learning (RL). When the state space is large or continuous, traditional tabular approaches are unfeasible and some form of function approximation is mandatory. In this paper, we introduce an optimistically-initialized variant of the popula…
Multi-armed bandit algorithms have become a reference solution for handling the explore/exploit dilemma in recommender systems, and many other important real-world problems, such as display advertisement. However, such algorithms usually assume a stationary reward distribution, which hardly holds in practice as users' …
This paper extends the MAB problem to consider risk-reward tradeoffs.
This research tackles balancing exploration and exploitation in deep RL for partially observable systems.
A simple uncertainty measure improves deep bandit performance.
One of the fundamental assumptions in stochastic control of continuous time processes is that the dynamics of the underlying (diffusion) process is known. This is, however, usually obviously not fulfilled in practice. On the other hand, over the last decades, a rich theory for nonparametric estimation of the drift (and…
Heterogeneous SVO leads to diverse policies in sequential social dilemmas.
SLIM model tackles graph classification by resolving part-interaction dilemmas.
Study online learning with off-policy feedback in adversarial bandit problems.
Bayesian Reinforcement Learning (RL) is capable of not only incorporating domain knowledge, but also solving the exploration-exploitation dilemma in a natural way. As Bayesian RL is intractable except for special cases, previous work has proposed several approximation methods. However, these methods are usually too sen…
The multi-armed bandit (MAB) problem is a classic example of the exploration-exploitation dilemma. It is concerned with maximising the total rewards for a gambler by sequentially pulling an arm from a multi-armed slot machine where each arm is associated with a reward distribution. In static MABs, the reward distributi…
Multinomial logit bandit is a sequential subset selection problem which arises in many applications. In each round, the player selects a -cardinality subset from candidate items, and receives a reward which is governed by a {\it multinomial logit} (MNL) choice model considering both item utility and substitution…
Algorithm reduces long-term policy regret in ML decision-making.
Improved analysis of UCRL2 with empirical Bernstein inequality reduces exploration-exploitation regret.
NFTs raise concerns like scams, racism, and sexism; centralization vs decentralization debate.
Many popular reinforcement learning problems (e.g., navigation in a maze, some Atari games, mountain car) are instances of the episodic setting under its stochastic shortest path (SSP) formulation, where an agent has to achieve a goal state while minimizing the cumulative cost. Despite the popularity of this setting, t…
Adding data can sometimes hurt model performance in multi-source healthcare tasks.
Batch Thompson Sampling reduces exploration-exploitation trade-off in online decision making.
Paper develops a dynamic Bayesian approach for active learning that optimizes exploration-exploitation balance.
We address the cold start problem in recommendation systems assuming no contextual information is available neither about users, nor items. We consider the case in which we only have access to a set of ratings of items by users. Most of the existing works consider a batch setting, and use cross-validation to tune param…
Paper finds wide minima are better for generalization and proposes a new learning rate schedule.
The Data Clustering (DC) problem is of central importance for the area of Machine Learning (ML), given its usefulness to represent data structural similarities from input spaces. Differently from Supervised Machine Learning (SML), which relies on the theoretical frameworks of the Statistical Learning Theory (SLT) and t…
We develop a coherent framework for integrative simultaneous analysis of the exploration-exploitation and model order selection trade-offs. We improve over our preceding results on the same subject (Seldin et al., 2011) by combining PAC-Bayesian analysis with Bernstein-type inequality for martingales. Such a combinatio…
ICEE learns new RL tasks in less time with a Transformer model.
Study of multidimensional control problems with reflection controls.
Leo Breiman's Rashomon Effect and Occam Dilemma are re-evaluated in the context of modern machine learning.
Margin enlargement over training data has been an important strategy since perceptrons in machine learning for the purpose of boosting the robustness of classifiers toward a good generalization ability. Yet Breiman (1999) showed a dilemma that a uniform improvement on margin distribution does NOT necessarily reduces ge…