Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

295886115 · Jun 202019922001200920172026
48 results for TD3 agents

This study examines deep hedging for S&P 500 options, revealing systematic delta corrections and fragility.

problem Understanding and validating deep hedging strategies for financial options.
method Compared TD3 agents with a Black-Scholes delta hedge, using walk-forward tests and symbolic regression.
result Deep hedging agents learn systematic delta corrections, which can improve performance but are regime-fragile.

Deep Q-learning agent outperforms traditional hedging in S&P 500 options.

problem Optimizing hedging strategies for at-the-money S&P 500 options.
method Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm trained on historical data.
result Deep reinforcement learning agent outperforms traditional delta-hedging in various market conditions.

Improved RL policies from offline data with relaxed BC constraints.

problem Overestimation bias in offline RL due to lack of interaction with environment.
method Introducing a policy constraint via behavioural cloning (BC) and adjusting the balance between RL and BC.
result Refined policies outperform baseline and match/exceed complex alternatives.

Paper tackles offline RL from mixed datasets with adaptive KL regularizer.

problem Challenges in optimizing RL and BC signals with varying action coverage and multiple action modes.
method Adaptively weighted reverse KL divergence regularizer based on TD3 algorithm.
result Empirically outperforms existing offline RL algorithms in MuJoCo locomotion tasks.

This paper proposes a trading strategy using TD3 for stock and cryptocurrency markets.

problem Predicting price movements in financial markets using historical data.
method Twin-Delayed DDPG (TD3) for continuous action space in algorithmic trading.
result The proposed strategy improves trading performance based on Return and Sharpe ratio metrics.

When learning behavior, training data is often generated by the learner itself; this can result in unstable training dynamics, and this problem has particularly important applications in safety-sensitive real-world control tasks such as robotics. In this work, we propose a principled and model-agnostic approach to miti…

2019-10-09abs ↗pdf ↗

AlphaGrad optimizes memory usage in RL algorithms by normalizing gradients.

problem Memory overhead and hyperparameter complexity in adaptive optimizers.
method Tensor-wise L2 normalization followed by a smooth hyperbolic tangent transformation controlled by a single parameter.
result AlphaGrad provides enhanced training stability and competitive performance in various RL algorithms.

SMEs provide a transparent testbed for RL evaluation.

problem Lack of precise, white-box diagnostics in RL environments.
method Synthetic Monitoring Environments (SMEs) with fully configurable task characteristics and known optimal policies.
result SMEs allow for precise evaluation of RL algorithms, revealing the impact of specific environmental properties.

Paper introduces SALE for better state-action learning in RL.

problem Challenges in representation learning for low-level states in RL.
method Introduces SALE, a novel approach for learning embeddings of state-action interactions.
result TD7 algorithm significantly outperforms existing continuous control algorithms.

SAVO actor improves reinforcement learning by avoiding local optima in complex Q-functions.

problem Gradient ascent in complex Q-functions leads to suboptimal solutions.
method SAVO actor generates multiple action proposals and truncates poor local optima.
result SAVO actor finds optimal actions more frequently and outperforms other architectures.

This paper analyzes DRL strategies in finance, revealing unique trading patterns and performance differences.

problem Limited research on DRL behavior in finance applications.
method Analysis of trading behaviors and purchase diversity of DRL algorithms (A2C, PPO, SAC, DDPG, TD3).
result DRL algorithms exhibit distinct trading patterns and performance differences, with A2C outperforming others in terms of cumulative rewards.

Offline RL tackles resource-constrained online deployment with improved policy transfer.

problem Training policies with limited online features using a rich offline dataset.
method Introduce a policy transfer algorithm that first trains a teacher agent with full offline features and then transfers knowledge to a student agent with limited online features.
result Consistent improvement in performance over baseline methods on resource-constrained datasets.

Deep reinforcement learning algorithms have been successfully applied to a range of challenging control tasks. However, these methods typically struggle with achieving effective exploration and are extremely sensitive to the choice of hyperparameters. One reason is that most approaches use a noisy version of their oper…

2019-05-02abs ↗pdf ↗

Automates RL with sample-efficient hyperparameter optimization.

problem Challenges in applying deep RL due to hyperparameter sensitivity and inefficiency.
method Population-based AutoRL framework for meta-optimizing RL algorithms and architectures.
result Reduces the number of environment interactions needed for meta-optimization by up to an order of magnitude.

FinRL simplifies deep RL for stock trading, making it accessible to beginners.

problem Lack of accessible tools for beginners in deep RL for stock trading.
method Developed a DRL library with reproducible tutorials and backtesting.
result FinRL streamlines development and comparison of trading strategies.

Bayesian VAR and Elliptical Black-Litterman models improve portfolio optimization during regime changes and heavy-tailed returns.

problem Portfolio optimization under market regime changes and heavy-tailed returns.
method BAVAR-BLED algorithm combining BAVAR and Black-Litterman models with Elliptical Distributions.
result Significant outperformance of state-of-the-art methods in Sharpe, Sortino ratios, and total returns.

Deep RL algorithms struggle with noisy rewards in portfolio optimisation.

problem Evaluating deep reinforcement learning for portfolio optimisation with market impact.
method Simulated data with geometric Brownian motion and market impact model; Kelly criterion as upper bound; PPO and A2C with GAE; clipping; hidden Markov model for regime changes.
result PPO and A2C with GAE perform better with noisy rewards; PPO with HMM learns different policies for regime changes.

Research integrates sentiment analysis with reinforcement learning for better trading strategies.

problem Improving trading performance by integrating sentiment data.
method Developed a sentiment-driven trading system using a large language model and reinforcement learning.
result Sentiment signals from FinGPT improve trading performance when combined with technical indicators.

In this paper, a new population-guided parallel learning scheme is proposed to enhance the performance of off-policy reinforcement learning (RL). In the proposed scheme, multiple identical learners with their own value-functions and policies share a common experience replay buffer, and search a good policy in collabora…

2020-01-09abs ↗pdf ↗

PBCS combines RL and motion planning for better exploration.

problem RL algorithms struggle with versatile exploration in complex environments.
method PBCS uses motion planning to find a good trajectory, then trains RL on a curriculum derived from it.
result PBCS outperforms state-of-the-art RL algorithms in 2D maze environments.

Model-based reinforcement learning (MBRL) with model-predictive control or online planning has shown great potential for locomotion control tasks in terms of both sample efficiency and asymptotic performance. Despite their initial successes, the existing planning methods search from candidate sequences randomly generat…

2019-06-20abs ↗pdf ↗

Method models other agents' behaviors without requiring direct observation.

problem Understanding and interacting effectively with other agents in reinforcement learning.
method Extracts representations from local observations of the controlled agent using encoder-decoder architectures.
result The method achieves higher returns than baseline methods in multi-agent environments.

Agent-to-agent finance aims to manage payments and trust for AI agents.

problem Managing financial interactions between autonomous AI agents.
method Develops agent-to-agent finance concept and explores blockchain solutions.
result Agent-to-agent finance can address coordination frictions in financial markets.

AI agents manage portfolios, improving on human oversight.

problem Improving strategic asset allocation for institutional investors.
method 50 specialized agents produce capital market assumptions, construct portfolios, critique, and vote on each other's output.
result Meta-agent compares forecasts with realized returns and improves agent performance.

New algorithm reduces learning regret in multi-agent systems with unknown dynamics.

problem Challenges in decentralized learning due to unknown dynamics and lack of communication.
method Proposed MARL algorithm for two-agent LQ systems with unknown dynamics and one-directional communication.
result Achieved O(T)O(\sqrt{T}) regret bound for multi-agent LQ systems with certain communication patterns.

Agents learn to give rewards to others in a shared learning environment.

problem How to encourage cooperation among RL agents in a shared environment.
method Each agent learns a reward function to influence others, optimizing for its own and others' extrinsic objectives.
result Agents significantly outperform standard RL in Markov games, often finding near-optimal division of labor.

New algorithm reduces regret in multi-agent bandits with malicious agents.

problem Collaboration between honest and malicious agents in multi-armed bandits.
method Dynamic reduction of communication with malicious agents, learning who is malicious.
result Algorithm reduces regret even with a single malicious agent, assuming mm is small compared to KK.

We formulate and analyze a multi-agent model for the evolution of individual and systemic risk in which the local agents interact with each other through a central agent who, in turn, is influenced by the mean field of the local agents. The central agent is stabilized by a bistable potential, the only stabilizing force…

2015-07-29abs ↗pdf ↗

Algorithm maximizes total reward in multi-agent bandits with adversarial corruptions.

problem Maximizing total reward in multi-agent bandits with adversarial corruptions.
method Proposes a cooperative learning algorithm robust to adversarial corruptions.
result Demonstrates an additive O((L/Lmin)C)O((L / L_{\min}) C) regret term for an adversary with unknown corruption budget.

I2C enables agents to learn efficient communication without redundancy.

problem Redundant broadcast communication in multi-agent cooperation.
method I2C learns a prior for agent-agent communication via causal inference and reinforcement learning.
result I2C reduces communication overhead and improves multi-agent cooperative performance.

PEAR dynamically reconfigures agent roles to prevent persistent biases in multi-agent debates.

problem Persistent positional biases and sensitivity to role assignments in fixed topologies.
method Dynamic reconfiguration of agent roles and sparse topologies based on evolving agent states.
result Significantly improves average accuracy over debate baselines across multiple reasoning benchmarks.
Agents Play Mix-gamephysics.soc-ph

In mix-game which is an extension of minority game, there are two groups of agents; group1 plays the majority game, but the group2 plays the minority game. This paper studies the change of the average winnings of agents and volatilities vs. the change of mixture of agents in mix-game model. It finds that the correlatio…

2005-05-17abs ↗pdf ↗

We propose a method for modeling and learning turn-taking behaviors for accessing a shared resource. We model the individual behavior for each agent in an interaction and then use a multi-agent fusion model to generate a summary over the expected actions of the group to render the model independent of the number of age…

2018-12-10abs ↗pdf ↗

Many learning agents impact a financial market model, showing complex dynamics.

problem Understanding the dynamics of financial markets with multiple learning agents.
method Agent-based model of financial market with multiple reinforcement learning agents interacting.
result Inclusion of learning agents changes market dynamics to match empirical data.

Reinforcement learning (RL) algorithms allow agents to learn skills and strategies to perform complex tasks without detailed instructions or expensive labelled training examples. That is, RL agents can learn, as we learn. Given the importance of learning in our intelligence, RL has been thought to be one of key compone…

2019-01-01abs ↗pdf ↗

Agents collaborate to reduce regret in a multi-agent linear bandit problem with side information.

problem Reducing regret in a multi-agent stochastic linear bandit with side information.
method A decentralized algorithm where agents communicate subspace indices and each plays a projected LinUCB on the corresponding low-dimensional subspace.
result Per-agent finite-time regret is much smaller when agents communicate compared to non-communicating case.

Adapts agent strategies on-the-fly for better cross-play in cooperative settings.

problem Cross-play issues between self-play agents and unseen partners.
method Adapts agent strategies using posterior belief updates via Gibbs sampling.
result Achieves strong cross-play in the Hanabi game without prior knowledge of partners' strategies.

A novel framework uses goal-conditioned reinforcement learning to generate diverse samples.

problem Generating high-quality, diverse samples from generative models.
method Two agents: GC-agent learns to reconstruct the training set, S-agent learns to imitate GC-agent without knowing the goals.
result Empirically, the method generates diverse and high-quality samples in image synthesis.