A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Greedy algorithm nearly outperforms exploration in contextual bandits.
New algorithm improves graph-based active learning by identifying unexplored regions.
Contextual multi-armed bandit problems arise frequently in important industrial applications. Existing solutions model the context either linearly, which enables uncertainty driven (principled) exploration, or non-linearly, by using epsilon-greedy exploration policies. Here we present a deep learning framework for cont…
PC-PG balances exploration and exploitation in reinforcement learning.
An online labor platform faces an online learning problem in matching workers with jobs and using the performance on these jobs to create better future matches. This learning problem is complicated by the rise of complex tasks on these platforms, such as web development and product design, that require a team of worker…
Contextual Bandits is one of the widely popular techniques used in applications such as personalization, recommendation systems, mobile health, causal marketing etc . As a dynamic approach, it can be more efficient than standard A/B testing in minimizing regret. We propose an end to end automated meta-learning pipeline…
A new reinforcement learning framework separates users into risk-tolerant and risk-averse groups for better performance.
Algorithm optimizes two objectives in bandits: minimizing regret and identifying best arm.
Contextual bandit algorithms are sensitive to the estimation method of the outcome model as well as the exploration method used, particularly in the presence of rich heterogeneity or complex outcome models, which can lead to difficult estimation problems along the path of learning. We study a consideration for the expl…
A scalable portfolio approach speeds up Bayesian optimization for noisy functions.
MADE improves exploration in RL by maximizing deviation from explored regions.
Thompson Sampling improves decision-making in partially observed contexts.
Many supervised learning tasks are emerged in dual forms, e.g., English-to-French translation vs. French-to-English translation, speech recognition vs. text to speech, and image classification vs. image generation. Two dual tasks have intrinsic connections with each other due to the probabilistic correlation between th…
We discuss deep reinforcement learning in an overview style. We draw a big picture, filled with details. We discuss six core elements, six important mechanisms, and twelve applications, focusing on contemporary work, and in historical contexts. We start with background of artificial intelligence, machine learning, deep…
New meta-RL method avoids exploration-exploitation trade-off.
We study minimax methods for off-policy evaluation (OPE) using value functions and marginalized importance weights. Despite that they hold promises of overcoming the exponential variance in traditional importance sampling, several key problems remain: (1) They require function approximation and are generally biased. Fo…
Paper develops a dynamic Bayesian approach for active learning that optimizes exploration-exploitation balance.
Pioneers a new network slicing solution for beyond-5G networks.
We consider recommendation systems that need to operate under wireless bandwidth constraints, measured as number of broadcast transmissions, and demonstrate a (tight for some instances) tradeoff between regret and bandwidth for two scenarios: the case of multi-armed bandit with context, and the case where there is a la…
The support vector machine (SVM) is a powerful and widely used classification algorithm. This paper uses the Karush-Kuhn-Tucker conditions to provide rigorous mathematical proof for new insights into the behavior of SVM. These insights provide perhaps unexpected relationships between SVM and two other linear classifier…
We show how to reduce the problem of computing VaR and CVaR with Student T return distributions to evaluation of analytical functions of the moments. This allows an analysis of the risk properties of systems to be carefully attributed between choices of risk function (e.g. VaR vs CVaR); choice of return distribution (p…
New AIM algorithm optimizes exploration-exploitation in bandits.
Clinical electroencephalographic (EEG) data varies significantly depending on a number of operational conditions (e.g., the type and placement of electrodes, the type of electrical grounding used). This investigation explores the statistical differences present in two different referential montages: Linked Ear (LE) and…
Algorithm achieves optimal pricing with minimal exploration for dynamic markets.
This research tackles balancing exploration and exploitation in deep RL for partially observable systems.
Improved Thompson Sampling for Bayesian Optimization.
Sparse reward is one of the biggest challenges in reinforcement learning (RL). In this paper, we propose a novel method called Generative Exploration and Exploitation (GENE) to overcome sparse reward. GENE automatically generates start states to encourage the agent to explore the environment and to exploit received rew…
Paper explores how knowledge distillation transfers inductive biases between models.
An agent explores indefinitely in an environment with unlimited rewards.
Improved analysis of UCRL2 with empirical Bernstein inequality reduces exploration-exploitation regret.
In graph-based active learning, algorithms based on expected error minimization (EEM) have been popular and yield good empirical performance. The exact computation of EEM optimally balances exploration and exploitation. In practice, however, EEM-based algorithms employ various approximations due to the computational ha…
Proposes EE-Net for neural exploration in contextual bandits.
The paper explores learning good policies from past data in large state spaces.
This paper applies Thompson Sampling to asymmetric -stable bandits for financial and wireless data.
XploVAE improves recommendation by balancing known and novel items.
Bayesian optimization offers the possibility of optimizing black-box operations not accessible through traditional techniques. The success of Bayesian optimization methods such as Expected Improvement (EI) are significantly affected by the degree of trade-off between exploration and exploitation. Too much exploration c…
Hyper addresses the hyperparameter tuning challenge in RL, improving exploration efficiency and robustness.
Neural Architecture Search (NAS) represents a class of methods to generate the optimal neural network architecture and typically iterate over candidate architectures till convergence over some particular metric like validation loss. They are constrained by the available computation resources, especially in enterprise e…
The aim of the current paper is to explore the implications on the group of the non-vanishing of the cohomology in degree one of one of its representation , given some mixing conditions on . In one direction, harmonic cocycles are used to show that the FC-centre should be finite (for mildly mixing unitary rep…
Batch Thompson Sampling reduces exploration-exploitation trade-off in online decision making.
We present a new model-based algorithm for reinforcement learning (RL) which consists of explicit exploration and exploitation phases, and is applicable in large or infinite state spaces. The algorithm maintains a set of dynamics models consistent with current experience and explores by finding policies which induce hi…
Finding a good compromise between the exploitation of known resources and the exploration of unknown, but potentially more profitable choices, is a general problem, which arises in many different scientific disciplines. We propose a stylized model for these exploration-exploitation situations, including population or e…
Kernel-UCBVI algorithm balances exploration and exploitation in metric state-action spaces.
Paper explores learning patterns in binary sequences, finding no method consistently outperforms others.
We propose a meta path planning algorithm named \emph{Neural Exploration-Exploitation Trees~(NEXT)} for learning from prior experience for solving new path planning problems in high dimensional continuous state and action spaces. Compared to more classical sampling-based methods like RRT, our approach achieves much bet…
The paper analyzes CMDPs, balancing exploration and exploitation to avoid constraint violations.
Exploration-exploitation of functions, that is learning and optimizing a mapping between inputs and expected outputs, is ubiquitous to many real world situations. These situations sometimes require us to avoid certain outcomes at all cost, for example because they are poisonous, harmful, or otherwise dangerous. We test…
Interactive user interfaces need to continuously evolve based on the interactions that a user has (or does not have) with the system. This may require constant exploration of various options that the system may have for the user and obtaining signals of user preferences on those. However, such an exploration, especiall…