This paper proposes a trading strategy using TD3 for stock and cryptocurrency markets.
problem Predicting price movements in financial markets using historical data.
method Twin-Delayed DDPG (TD3) for continuous action space in algorithmic trading.
result The proposed strategy improves trading performance based on Return and Sharpe ratio metrics.
MVPI framework optimizes risk in reinforcement learning, improving performance in robot simulations.
problem Optimizing risk in reinforcement learning control problems.
method Mean-Variance Policy Iteration (MVPI) framework for risk-averse control in MDPs.
result Risk-averse TD3 outperforms previous methods in robot simulation tasks.
The paper examines various RL algorithms to address overestimation and noise issues.
problem Overestimation and noise in deep reinforcement learning algorithms.
method Analysis of DQN, double DQN, DDPG, TD3, and hill climbing algorithms.
result Optimal noise settings for TD3 in specific environments.
Improved Q-learning for robot control with faster data-efficiency.
problem Poor data-efficiency and stochasticity in deep Q-learning.
method Composite Q-learning: Decomposes Q-function into short- and long-term predictions.
result Significant learning speedup and improved data-efficiency in robot tasks.
AlphaGrad optimizes memory usage in RL algorithms by normalizing gradients.
problem Memory overhead and hyperparameter complexity in adaptive optimizers.
method Tensor-wise L2 normalization followed by a smooth hyperbolic tangent transformation controlled by a single parameter.
result AlphaGrad provides enhanced training stability and competitive performance in various RL algorithms.
Improved RL policies from offline data with relaxed BC constraints.
problem Overestimation bias in offline RL due to lack of interaction with environment.
method Introducing a policy constraint via behavioural cloning (BC) and adjusting the balance between RL and BC.
result Refined policies outperform baseline and match/exceed complex alternatives.
Paper tackles offline RL from mixed datasets with adaptive KL regularizer.
problem Challenges in optimizing RL and BC signals with varying action coverage and multiple action modes.
method Adaptively weighted reverse KL divergence regularizer based on TD3 algorithm.
result Empirically outperforms existing offline RL algorithms in MuJoCo locomotion tasks.
Paper introduces SALE for better state-action learning in RL.
problem Challenges in representation learning for low-level states in RL.
method Introduces SALE, a novel approach for learning embeddings of state-action interactions.
result TD7 algorithm significantly outperforms existing continuous control algorithms.
SAVO actor improves reinforcement learning by avoiding local optima in complex Q-functions.
problem Gradient ascent in complex Q-functions leads to suboptimal solutions.
method SAVO actor generates multiple action proposals and truncates poor local optima.
result SAVO actor finds optimal actions more frequently and outperforms other architectures.
This paper analyzes DRL strategies in finance, revealing unique trading patterns and performance differences.
problem Limited research on DRL behavior in finance applications.
method Analysis of trading behaviors and purchase diversity of DRL algorithms (A2C, PPO, SAC, DDPG, TD3).
result DRL algorithms exhibit distinct trading patterns and performance differences, with A2C outperforming others in terms of cumulative rewards.
This study examines deep hedging for S&P 500 options, revealing systematic delta corrections and fragility.
problem Understanding and validating deep hedging strategies for financial options.
method Compared TD3 agents with a Black-Scholes delta hedge, using walk-forward tests and symbolic regression.
result Deep hedging agents learn systematic delta corrections, which can improve performance but are regime-fragile.
DRL automates stock market trading with a 2.68 Sharpe Ratio.
problem Automating profitable trades in the stock market.
method Formulated as a POMDP, solved with TD3 algorithm.
result 2.68 Sharpe Ratio on unseen data.
CERL uses a portfolio of learners to explore diverse regions, outperforming individual learners.
problem Limited exploration and sensitivity to hyperparameters in reinforcement learning.
method CERL employs a portfolio of learners with varying time-horizons and a shared replay buffer, dynamically distributing computational resources.
result Emergent learner outperforms individual learners and is more sample-efficient.
Combines evolutionary and RL methods for better policy search.
problem Low sample efficiency in evolutionary methods and high sensitivity in RL methods.
method Combines CEM and td3 algorithms for policy search.
result cem-rl offers a better trade-off between performance and sample efficiency.
CARML uses meta-learning to avoid obstacles in 2D vehicle navigation.
problem Collision avoidance in 2D vehicle navigation.
method Model-Agnostic Meta-Learning for multi-objective reinforcement learning.
result CARML outperforms a baseline TD3 solution in obstacle avoidance.
Bayesian VAR and Elliptical Black-Litterman models improve portfolio optimization during regime changes and heavy-tailed returns.
problem Portfolio optimization under market regime changes and heavy-tailed returns.
method BAVAR-BLED algorithm combining BAVAR and Black-Litterman models with Elliptical Distributions.
result Significant outperformance of state-of-the-art methods in Sharpe, Sortino ratios, and total returns.
Deep RL algorithms struggle with noisy rewards in portfolio optimisation.
problem Evaluating deep reinforcement learning for portfolio optimisation with market impact.
method Simulated data with geometric Brownian motion and market impact model; Kelly criterion as upper bound; PPO and A2C with GAE; clipping; hidden Markov model for regime changes.
result PPO and A2C with GAE perform better with noisy rewards; PPO with HMM learns different policies for regime changes.
Deep Q-learning agent outperforms traditional hedging in S&P 500 options.
problem Optimizing hedging strategies for at-the-money S&P 500 options.
method Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm trained on historical data.
result Deep reinforcement learning agent outperforms traditional delta-hedging in various market conditions.
Research integrates sentiment analysis with reinforcement learning for better trading strategies.
problem Improving trading performance by integrating sentiment data.
method Developed a sentiment-driven trading system using a large language model and reinforcement learning.
result Sentiment signals from FinGPT improve trading performance when combined with technical indicators.
SMEs provide a transparent testbed for RL evaluation.
problem Lack of precise, white-box diagnostics in RL environments.
method Synthetic Monitoring Environments (SMEs) with fully configurable task characteristics and known optimal policies.
result SMEs allow for precise evaluation of RL algorithms, revealing the impact of specific environmental properties.
Mitigates instability in reinforcement learning for safer robotics.
problem Unstable training dynamics in reinforcement learning, especially for safety-sensitive tasks.
method Maintains a history of the agent and reverts to previous parameters when performance decreases.
result Improves performance and stability compared to state-of-the-art algorithms.
Enhances RL performance with a population-guided parallel learning scheme.
problem Improving off-policy reinforcement learning performance.
method Population-guided parallel learning scheme with shared experience replay buffer and soft policy update.
result Monotone improvement of the expected cumulative return proved theoretically and demonstrated in practice.
PBCS combines RL and motion planning for better exploration.
problem RL algorithms struggle with versatile exploration in complex environments.
method PBCS uses motion planning to find a good trajectory, then trains RL on a curriculum derived from it.
result PBCS outperforms state-of-the-art RL algorithms in 2D maze environments.
Paper introduces a meta-critic for accelerating off-policy actor-critic learning.
problem Improving sample efficiency in continuous control tasks.
method Meta-critic that meta-learns an additional loss for the actor.
result Online meta-critic learning leads to improved performance in various continuous control environments.
Automates RL with sample-efficient hyperparameter optimization.
problem Challenges in applying deep RL due to hyperparameter sensitivity and inefficiency.
method Population-based AutoRL framework for meta-optimizing RL algorithms and architectures.
result Reduces the number of environment interactions needed for meta-optimization by up to an order of magnitude.
TT-DAC-PS: A deterministic actor-critic approach for optimal trade execution
problem Optimal execution of large stock sell programs
method Twin-Target Deterministic Actor-Critic with Policy Smoothing
result Reduces mean implementation shortfall percentage
POPLIN improves model-based planning in complex environments.
problem Efficient planning in complex high-dimensional environments.
method Combines policy networks with online planning, optimizing parameters directly.
result POPLIN achieves state-of-the-art performance in MuJoCo benchmarks, 3x more sample efficient.
PDERL improves evolutionary reinforcement learning by using learning-based variation operators.
problem Scalability issue in Genetic Algorithms when combined with Deep Neural Networks.
method Integrates evolutionary and reinforcement learning through a hierarchical approach with learning-based variation operators.
result PDERL outperforms traditional evolutionary and reinforcement learning methods in robot locomotion tasks.
Offline RL tackles resource-constrained online deployment with improved policy transfer.
problem Training policies with limited online features using a rich offline dataset.
method Introduce a policy transfer algorithm that first trains a teacher agent with full offline features and then transfers knowledge to a student agent with limited online features.
result Consistent improvement in performance over baseline methods on resource-constrained datasets.
Improves BC policies by generating new plausible trajectories.
problem Sub-optimal data quality in BC leads to poor policy performance.
method Trajectory Stitching (TS) generates new plausible transitions.
result TS significantly improves behavioural policies over original data.
FinRL simplifies deep RL for stock trading, making it accessible to beginners.
problem Lack of accessible tools for beginners in deep RL for stock trading.
method Developed a DRL library with reproducible tutorials and backtesting.
result FinRL streamlines development and comparison of trading strategies.