A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
GPE algorithm optimizes nonparametric contextual bandits with efficient regret bounds.
Paper eliminates warm-up phase for PO in linear MDPs, achieving optimal regret.
A new policy for contextual bandits adapts to reward vector shifts.
In this paper, we consider the problem of online learning of Markov decision processes (MDPs) with very large state spaces. Under the assumptions of realizable function approximation and low Bellman ranks, we develop an online learning algorithm that learns the optimal value function while at the same time achieving ve…
We consider a multi-armed bandit problem in a setting where each arm produces a noisy reward realization which depends on an observable random covariate. As opposed to the traditional static multi-armed bandit problem, this setting allows for dynamically changing rewards that better describe applications where side inf…
Policy gradient methods are very attractive in reinforcement learning due to their model-free nature and convergence guarantees. These methods, however, suffer from high variance in gradient estimation, resulting in poor sample efficiency. To mitigate this issue, a number of variance-reduction approaches have been prop…
A highly influential ingredient of many techniques designed to exploit sparsity in numerical optimization is the so-called chordal extension of a graph representation of the optimization problem. The definitive relation between chordal extension and the performance of the optimization algorithm that uses the extension …
Coagent policy gradient algorithms (CPGAs) are reinforcement learning algorithms for training a class of stochastic neural networks called coagent networks. In this work, we prove that CPGAs converge to locally optimal policies. Additionally, we extend prior theory to encompass asynchronous and recurrent coagent networ…
New RL algorithm reduces policy switching cost to loglog(T) with similar regret.
SPO optimizes LLMs by eliminating group-based baselines and variance issues.
Maximum entropy deep reinforcement learning (RL) methods have been demonstrated on a range of challenging continuous tasks. However, existing methods either suffer from severe instability when training on large off-policy data or cannot scale to tasks with very high state and action dimensionality such as 3D humanoid l…
Analyzes how economic policies affect wealth distribution in Bitcoin token economy.
Optimization-based pruning eliminates backpropagation for large language models.
We establish a new connection between value and policy based reinforcement learning (RL) based on a relationship between softmax temporal value consistency and policy optimality under entropy regularization. Specifically, we show that softmax consistent action values correspond to optimal entropy regularized policy pro…
The paper analyzes the intrinsic exploration terms in policy-gradient algorithms.
New method estimates off-policy data without needing known behavior policy.
Develops an anytime-valid framework for optimal policy identification from logged contextual bandit data.
Generative Adversarial Regression (GAR) learns risk scenarios robustly across policies.
This paper tackles distribution shift in model-based offline RL, proposing a shifts-aware reward method.
In this paper, we study the multi-armed bandit problem in the batched setting where the employed policy must split data into a small number of batches. While the minimax regret for the two-armed stochastic bandits has been completely characterized in \cite{perchet2016batched}, the effect of the number of arms on the re…
Enhances large language models' reasoning through simpler off-policy reinforcement learning.
New RL approach infers optimal policies via variational inference.
New method tackles safe reinforcement learning from offline data.
Automates bias control in reinforcement learning algorithms.
This work provides guarantees for off-policy function estimation under realizability assumptions.
Develops a support-aware framework for reserve-policy selection in advertising markets.
The paper uses machine learning to optimize rework policies in semiconductor manufacturing.
The paper addresses treatment recommendation problems by optimizing distributional characteristics.
An optimal feedback controller for a given Markov decision process (MDP) can in principle be synthesized by value or policy iteration. However, if the system dynamics and the reward function are unknown, a learning agent must discover an optimal controller via direct interaction with the environment. Such interactive d…
Study aims to optimize financial investments by balancing risk and reward efficiently.
Safe RL-based vibration control using LQR guidance.
GradientDICE improves offline estimation for reinforcement learning policies.
We consider the dynamic assortment optimization problem under the multinomial logit model (MNL) with unknown utility parameters. The main question investigated in this paper is model mis-specification under the -contamination model, which is a fundamental model in robust statistics and machine learning. In…
Two algorithms address limited adaptivity in generalized linear contextual bandits.
We present an approach to make molecular optimization more efficient. We infer a hypergraph replacement grammar from the ChEMBL database, count the frequencies of particular rules being used to expand particular nonterminals in other rules, and use these as conditional priors for the policy model. Simulating random mol…
We study the problem of multi-agent reinforcement learning (MARL) with adaptivity constraints -- a new problem motivated by real-world applications where deployments of new policies are costly and the number of policy updates must be minimized. For two-player zero-sum Markov Games, we design a (policy) elimination base…
Policy gradient and actor-critic algorithms form the basis of many commonly used training techniques in deep reinforcement learning. Using these algorithms in multiagent environments poses problems such as nonstationarity and instability. In this paper, we first demonstrate that standard softmax-based policy gradient c…
Pessimistic RL algorithm improves offline RL performance.
We propose and study a new model for reinforcement learning with rich observations, generalizing contextual bandits to sequential decision making. These models require an agent to take actions based on observations (features) with the goal of achieving long-term performance competitive with a large set of policies. To …
Proposes a flexible tournament design combining knockout and round-robin.
Lapse-supported life insurance exacerbates adverse selection risks.
Improved exploration in adversarial MDPs via dilated bonuses.
Improved elimination strategies for adaptive bandit identification reduce sample complexity and computational burden.
Information systems experience an ever-growing volume of unstructured data, particularly in the form of textual materials. This represents a rich source of information from which one can create value for people, organizations and businesses. For instance, recommender systems can benefit from automatically understanding…
We present Placeto, a reinforcement learning (RL) approach to efficiently find device placements for distributed neural network training. Unlike prior approaches that only find a device placement for a specific computation graph, Placeto can learn generalizable device placement policies that can be applied to any graph…
As the Portable Document Format (PDF) file format increases in popularity, research in analysing its structure for text extraction and analysis is necessary. Detecting headings can be a crucial component of classifying and extracting meaningful data. This research involves training a supervised learning model to detect…
Classification may not be reliable for several reasons: noise in the data, insufficient input information, overlapping distributions and sharp definition of classes. Faced with several possibilities neural network may in such cases still be useful if instead of a classification elimination of improbable classes is done…
New BE dimension measure reveals rich RL problems with sample-efficient algorithms.