A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
TIR expands LLM capabilities by enabling problem-solving strategies.
This paper augments the reward received by a reinforcement learning agent with potential functions in order to help the agent learn (possibly stochastic) optimal policies. We show that a potential-based reward shaping scheme is able to preserve optimality of stochastic policies, and demonstrate that the ability of an a…
We present a new method of blackbox optimization via gradient approximation with the use of structured random orthogonal matrices, providing more accurate estimators than baselines and with provable theoretical guarantees. We show that this algorithm can be successfully applied to learn better quality compact policies …
This paper introduces a new reward shaping method for average-reward reinforcement learning.
Proposes Constrained Q-learning for reinforcement learning with constraints.
A new method shapes reinforcement learning environments by abstracting large state spaces.
Adapts GRPO for off-policy RL, improving reward.
Paper introduces DTAE to optimize RL algorithms, balancing exploration and exploitation.
Develops a method to efficiently use offline data for RL policy optimization.
A new method removes policy optimization in adversarial imitation learning.
Recent policy optimization approaches have achieved substantial empirical success by constructing surrogate optimization objectives. The Approximate Policy Iteration objective (Schulman et al., 2015a; Kakade and Langford, 2002) has become a standard optimization target for reinforcement learning problems. Using this ob…
Describing shapes by suitable measures in object segmentation, as proposed in [24], allows to combine the advantages of the representations as parametrized contours and indicator functions. The pseudo-Riemannian structure of optimal transport can be used to model shapes in ways similar as with contours, while the Kanto…
New approach optimizes policies in adversarial MDPs using adversarial learning.
NVMDP framework tackles non-stationary MDPs with varying discount rates.
The optimal policy of a reinforcement learning problem is often discontinuous and non-smooth. I.e., for two states with similar representations, their optimal policies can be significantly different. In this case, representing the entire policy with a function approximator (FA) with shared parameters for all states may…
WDAIL uses Wasserstein distance for more effective reward shaping in IL.
We present an efficient algorithm for model-free episodic reinforcement learning on large (potentially continuous) state-action spaces. Our algorithm is based on a novel -learning policy with adaptive data-driven discretization. The central idea is to maintain a finer partition of the state-action space in regions w…
Real-world tasks are often highly structured. Hierarchical reinforcement learning (HRL) has attracted research interest as an approach for leveraging the hierarchical structure of a given task in reinforcement learning (RL). However, identifying the hierarchical policy structure that enhances the performance of RL is n…
Policy optimization on high-dimensional continuous control tasks exhibits its difficulty caused by the large variance of the policy gradient estimators. We present the action subspace dependent gradient (ASDG) estimator which incorporates the Rao-Blackwell theorem (RB) and Control Variates (CV) into a unified framework…
New RL approach infers optimal policies via variational inference.
Physics-informed learning framework for pH systems and EB-PBC control.
This paper improves Q-learning bounds using reference-advantage decomposition.
Deriving and applying Proximal Policy Optimization to GFlowNets for efficient training of discrete sampling policies
Optimal trading strategy with predictor and costs, derived equations and shape.
In the NIPS 2017 Learning to Run challenge, participants were tasked with building a controller for a musculoskeletal model to make it run as fast as possible through an obstacle course. Top participants were invited to describe their algorithms. In this work, we present eight solutions that used deep reinforcement lea…
ADPO optimizes relative advantage in reinforcement learning from human feedback.
A novel efficient method for computing the Knowledge-Gradient policy for Continuous Parameters (KGCP) for deterministic optimization is derived. The differences with Expected Improvement (EI), a popular choice for Bayesian optimization of deterministic engineering simulations, are explored. Both policies and the Upper …
A3RL combines online and offline RL with active sampling to improve policy learning.
Self-supervised reward prediction improves RL in sparse reward settings.
Optimal policies in Markov decision processes (MDPs) are very sensitive to model misspecification. This raises serious concerns about deploying them in high-stake domains. Robust MDPs (RMDP) provide a promising framework to mitigate vulnerabilities by computing policies with worst-case guarantees in reinforcement learn…
DG improves policy gradients by weighting actions with a sigmoid of advantage and surprisal.
Paper proposes FedQ-Advantage for federated Q-learning with near-optimal regret and low communication cost.
Policy gradient based reinforcement learning algorithms coupled with neural networks have shown success in learning complex policies in the model free continuous action space control setting. However, explicitly parameterized policies are limited by the scope of the chosen parametric probability distribution. We show t…
In this work we present a technique to use natural language to help reinforcement learning generalize to unseen environments. This technique uses neural machine translation, specifically the use of encoder-decoder networks, to learn associations between natural language behavior descriptions and state-action informatio…
This paper analyzes optimal consumption strategies for loss-averse investors with multiplicative habit formation.
A novel approach for safe offline RL using latent safety constraints.
The growing prospect of deep reinforcement learning (DRL) being used in cyber-physical systems has raised concerns around safety and robustness of autonomous agents. Recent work on generating adversarial attacks have shown that it is computationally feasible for a bad actor to fool a DRL policy into behaving sub optima…
Study visualizes actor-critic loss landscapes for inventory optimization.
Traffic congestion in metropolitan areas is a world-wide problem that can be ameliorated by traffic lights that respond dynamically to real-time conditions. Recent studies applying deep reinforcement learning (RL) to optimize single traffic lights have shown significant improvement over conventional control. However, o…
As the most successful variant and improvement for Trust Region Policy Optimization (TRPO), proximal policy optimization (PPO) has been widely applied across various domains with several advantages: efficient data utilization, easy implementation, and good parallelism. In this paper, a first-order gradient reinforcemen…
Generative thermal design learns optimal shapes using multi-agent reinforcement learning.
Deep Reinforcement Learning (DRL) has become a powerful strategy to solve complex decision making problems based on Deep Neural Networks (DNNs). However, it is highly data demanding, so unfeasible in physical systems for most applications. In this work, we approach an alternative Interactive Machine Learning (IML) stra…
DNN policies improve stochastic AC OPF for power grid optimization.
Develops CLTs for Markov chain transition probabilities and policies.
SPO optimizes LLMs by eliminating group-based baselines and variance issues.
Study uses machine learning to estimate effective policies in settings with hidden individual actions.
Paper uses DRL to optimize portfolios, balancing risk and return.
SIBRE boosts reinforcement learning convergence by rewarding improvement over past performance.