Study higher order mean curvatures in SAC half-lightlike submanifolds.
problem Exploring new geometric properties of SAC submanifolds.
method Introduced and used higher order mean curvatures; derived integration formulae.
result Generalized known results and derived new integration formulae.
Meta-SAC automatically tunes SAC's entropy temperature for better exploration.
problem Exploration-exploitation dilemma in reinforcement learning.
method Meta-SAC uses metagradient and a novel meta objective to automatically adjust SAC's entropy temperature.
result Meta-SAC outperforms SAC-v2 by 10% on the humanoid-v2 task.
Enhances SAC for better sample efficiency in continuous-action tasks.
problem Improving sample efficiency in soft actor-critic algorithms.
method Integrating Emphasizing Recent Experience (ERE) with Soft Actor-Critic (SAC) and Priority Experience Replay (PER).
result ERE significantly improves sample efficiency compared to vanilla SAC, especially for continuous-action tasks.
PI-SAC agents learn predictive information to improve RL efficiency.
problem Improving sample efficiency in reinforcement learning.
method PI-SAC agents use a contrastive version of Conditional Entropy Bottleneck to learn predictive information from past and future states.
result PI-SAC agents significantly improve sample efficiency on challenging continuous control tasks.
Band-limited SAC improves learning efficiency and stability in simulated environments.
problem Improving sample efficiency and stability in SAC algorithms.
method Artificially bandlimiting the target critic's spatial resolution using a convolutional filter.
result Bandlimited SAC outperforms classic twin-critic SAC in various Gym environments and is more stable.
SAC-NF improves RL exploration by using normalizing flows to discover policies faster.
problem Discovering optimal policies in sparse reward domains.
method Extending SAC with normalizing flows to improve exploration efficiency.
result SAC-NF accelerates policy discovery with smaller parameter usage.
LC-SAC tackles non-stationary dynamics in reinforcement learning.
problem Degradation of deep RL methods in non-stationary environments.
method LC-SAC uses latent context encoders and contrastive loss for dynamic information capture.
result LC-SAC outperforms SAC on environments with drastic dynamics changes.
Paper uses SAC and DDPG to optimize cryptocurrency portfolios.
problem Adapting to volatile and nonlinear cryptocurrency markets.
method Reinforcement learning with SAC and DDPG algorithms.
result SAC and DDPG outperform traditional strategies in cryptocurrency markets.
Paper improves AIRL by enhancing policy imitation and addressing reward recovery issues.
problem Inadequate policy imitation and limited transferable reward recovery in AIRL.
method Substituted built-in algorithm with SAC for policy updating and proposed PPO-AIRL + SAC hybrid framework.
result SAC improves policy imitation but hinders reward recovery; PPO-AIRL + SAC achieves satisfactory transfer effect.
SAC improves deep RL by balancing reward and randomness.
problem High sample complexity and hyperparameter brittleness in RL.
method Maximum entropy framework, constrained optimization, temperature tuning.
result SAC achieves state-of-the-art performance and stability.
Hybrid SAC improves RL for video games with discrete, continuous actions.
problem Improving RL performance in video games with practical constraints.
method Extension of Soft Actor-Critic (SAC) for handling discrete, continuous, and parameterized actions.
result Hybrid SAC successfully solves a high-speed driving task and is competitive on parameterized actions benchmarks.
Paper uses SAC RL to optimize market-making strategies.
problem Optimizing market-making strategies with risk management.
method Applying SAC reinforcement learning to automate market-making decisions.
result Agent learns to optimize spreads and hedge trades.
Paper proposes multiscale self-attentive convolutions for vision and language.
problem Improving language and vision understanding models using self-attention.
method Developed 1D and 2D Self Attentive Convolutions (SAC), multiscale SAC (MSAC).
result MSAC enhances model performance for vision and language tasks.
TOPPO improves PPO for MTRL by balancing critic gradients, outperforming SAC.
problem Critic-side gradient ill-conditioning in PPO for MTRL.
method Critic Balancing modules to improve gradient conditioning and balance task updates.
result TOPPO achieves stronger mean and tail-task performance than SAC-family and ARS-family baselines.
SAC-X enables learning complex behaviors from sparse rewards.
problem Learning complex behaviors from sparse reward signals.
method Scheduled Auxiliary Control (SAC-X) with auxiliary tasks.
result SAC-X enables efficient exploration and complex behavior learning.
Improved SAC with AWMP for better control tasks.
problem Discontinuous and non-smooth optimal policies in reinforcement learning.
method Advantage Weighted Mixture Policy (AWMP) for SAC, learning state-specific weights.
result SAC with AWMP outperforms SAC in four control tasks.
Paper uses RL to optimize multi-asset portfolios in fluctuating markets.
problem Optimizing multi-asset portfolios in time-varying financial markets.
method Soft Actor-Critic (SAC) algorithm for policy learning, policy iteration process.
result SAC algorithm outperforms in various criteria in simulated and real financial markets.
Improved exploration in SAC using Normalizing Flows policies.
problem Brittleness and inefficiency of DRL algorithms in continuous action spaces.
method Introducing Normalizing Flow policies within the SAC framework to learn more expressive policies.
result Increased stability and better exploration in sparse reward settings.
Paper improves SVaR estimation for stress testing under macro scenarios using a hybrid GPR-HS framework.
problem Numerical instability in traditional SVaR estimation under extreme shocks.
method Extends GPR-HS framework to forward-looking stress scenarios with SACS for stable covariance.
result Stable SVaR ranges from -2.1020% to -2.2231%, preserving coherence property.
A streamlined DRL algorithm improves sample efficiency without entropy maximization.
problem Improving sample efficiency in off-policy DRL algorithms.
method Output normalization and non-uniform sampling.
result Proposed algorithm matches SAC's performance without entropy maximization and improves sample efficiency.
New algorithms improve deep RL with entropy-based action selection and environment exploration.
problem Improving deep reinforcement learning algorithms for better sample efficiency and effectiveness.
method Proposes Tsallis entropy Actor-Critic (TAC), Renyi entropy Actor-Critic (RAC), and Ensemble Actor-Critic (EAC) algorithms.
result Empirically, TAC, RAC, and EAC outperform SAC and other algorithms in benchmark control tasks.
New PAC-Bayesian approach stabilizes actor-critic learning.
problem Training instability in actor-critic algorithms.
method Employing PAC-Bayesian bound as the critic training objective.
result Significant improvement in online learning performance.
Researchers analyze how algorithmic and implementation choices affect RL performance.
problem Difficulty in separating algorithmic and implementation differences in RL performance.
method Unified derivations through a single control-as-inference objective, categorizing algorithms as EM or KL minimization.
result Implementation details are co-adapted with algorithmic choices, some transferable across algorithms.
Paper uses DRL to improve volatility fitting in equity derivatives.
problem Improving volatility fitting in equity derivatives markets.
method Apply Deep Reinforcement Learning (DRL) to solve the fitting problem.
result DRL algorithms achieve at least as good as standard fitting methods.
Simple image augmentation boosts deep RL from pixels.
problem Training deep RL models from raw pixel inputs is challenging.
method Applying standard computer vision data augmentation to reinforcement learning.
result Our method significantly improves SAC's performance on DeepMind control suite.
This paper analyzes DRL strategies in finance, revealing unique trading patterns and performance differences.
problem Limited research on DRL behavior in finance applications.
method Analysis of trading behaviors and purchase diversity of DRL algorithms (A2C, PPO, SAC, DDPG, TD3).
result DRL algorithms exhibit distinct trading patterns and performance differences, with A2C outperforming others in terms of cumulative rewards.
CausalCOMRL improves RL task representations by integrating causal relationships, enhancing generalizability.
problem Spurious correlations in context-based offline meta-reinforcement learning.
method CausalCOMRL integrates causal representation learning to uncover and incorporate causal relationships among task components.
result CausalCOMRL achieves better performance on meta-reinforcement learning benchmarks.
ESAC improves reinforcement learning by lookahead and intuition.
problem Designing efficient reinforcement learning architectures for intelligent agents.
method Integrates selector, tuner, model-learner, estimator, and lookahead into ESAC.
result ESAC outperforms other architectures in optimizing policies.
Safe-FinRL uses DRL for high-frequency stock trading, reducing bias and variance.
problem Challenges in applying DRL to high-frequency stock trading, especially bias and variance issues.
method Safe-FinRL separates financial time series into near-stationary short environments and uses Trace-SAC with a general retrace operator.
result Safe-FinRL reduces bias and variance significantly in near-stationary financial environments.
AG-RL uses action grammars to improve reinforcement learning efficiency.
problem Improving sample efficiency in reinforcement learning.
method Integrates action grammars into reinforcement learning algorithms to enhance performance.
result Significant improvement in performance across multiple Atari games.
Method trains vision and control policies on real robots quickly.
problem Training vision-based control policies on real robots efficiently.
method Multi-task Reinforcement Learning with auxiliary tasks.
result Significant learning speed-ups and task learning from-scratch.
TT-DAC-PS: A deterministic actor-critic approach for optimal trade execution
problem Optimal execution of large stock sell programs
method Twin-Target Deterministic Actor-Critic with Policy Smoothing
result Reduces mean implementation shortfall percentage
ESAC combines genetic methods with RL to improve scalability and efficiency.
problem Combining genetic scalability with RL's data efficiency and optimal control.
method Combines Evolution Strategies (ES) with Soft Actor-Critic (SAC) to enable skill transfer and reduce hyperparameter sensitivity.
result Demonstrates improved performance and sample efficiency in challenging tasks.
Develops RL for optimal market-making in non-Markov processes.
problem Optimal market-making in non-Markov price processes.
method Deep reinforcement learning with Soft Actor-Critic (SAC) algorithm.
result Optimal strategy for market-making in semi-Markov and Hawkes Jump-Diffusion dynamics.
Deep RL algorithms struggle with noisy rewards in portfolio optimisation.
problem Evaluating deep reinforcement learning for portfolio optimisation with market impact.
method Simulated data with geometric Brownian motion and market impact model; Kelly criterion as upper bound; PPO and A2C with GAE; clipping; hidden Markov model for regime changes.
result PPO and A2C with GAE perform better with noisy rewards; PPO with HMM learns different policies for regime changes.
Mutual-information regularization improves RL algorithms, especially in high-dimensional domains.
problem Value overestimation and exploration in reinforcement learning.
method Developed a novel mutual-information regularized actor-critic learning (MIRACLE) algorithm.
result MIRACLE outperforms state-of-the-art algorithms in continuous action spaces.
Mirror descent method improved RL algorithms.
problem Improving RL algorithms for better performance.
method Mirror descent method applied to RL, solving trust-region problems.
result MDPO outperforms or matches other RL algorithms in continuous control tasks.
ApolloRL offers a platform for RL research in autonomous driving.
problem Improving reinforcement learning for autonomous driving.
method Open platform with training, simulation, and evaluation components.
result Baseline agents perform well in the ApolloRL environment.
The paper solves a decades-old problem using complex bordism and Pontryagin duality.
problem A decades-old question raised by Iz Singer about complex bundles and their connections.
method Combining complex bordism, Pontryagin duality, and the APS Index Theorem to solve the problem.
result Stable complex bundles have complete numerical invariants computable by integrals of characteristic forms.
New reinforcement learning bound improves generalization for sequential data.
problem Challenges in obtaining generalization guarantees for reinforcement learning due to sequential data.
method PAC-Bayesian reinforcement learning with consideration of Markov dependencies and mixing time.
result Demonstrated practical utility through PB-SAC, providing meaningful confidence certificates.
A new RL framework handles autocorrelated actions for better learning and stability.
problem Improving reinforcement learning algorithms for better stability and efficiency.
method Introduces a new algorithm that optimizes policies with autocorrelated actions.
result The new algorithm outperforms existing methods in four simulated control problems.
A new RL method uses flows to mix policies for better exploration.
problem Learning in high-dimensional state spaces with deceptive rewards.
method Uses normalizing flows to attract and repel policies in a population.
result Empirically outperforms Soft-Actor Critic (SAC) on MuJoCo tasks.
This paper surveys parallel submanifolds in Riemannian and pseudo-Riemannian manifolds.
problem Understanding parallel submanifolds in Riemannian and pseudo-Riemannian manifolds.
method Comprehensive survey of parallel submanifolds.
result Extrinsic invariants of parallel submanifolds do not vary from point to point.
The study characterizes submanifolds in product spaces.
problem Limited studies on pseudo-umbilical submanifolds.
method Using projections from product structure, conditions for submanifolds to be invariant, anti-invariant, or semi-invariant are derived.
result Necessary and sufficient conditions for pseudo-umbilical submanifolds in locally product Riemannian manifolds.
Researchers describe a specific type of submanifolds in Euclidean space.
problem Understanding inhomogeneous almost symmetric submanifolds.
method Completely describing submanifolds as unions of parallel symmetric submanifolds.
result Described inhomogeneous properly embedded almost symmetric submanifolds as unions of symmetric submanifolds.
Study on hemi-slant submanifolds in nearly Kaehler manifolds.
problem Characterizing hemi-slant submanifolds in nearly Kaehler manifolds.
method Integrability of distributions and cohomology class analysis.
result Results on totally umbilical hemi-slant submanifolds.
The paper finds many infinite-dimensional weakly reflective PF submanifolds in Hilbert spaces.
problem Minimal submanifolds in Hilbert spaces with reflective properties.
method Introduced weakly reflective PF submanifolds into Hilbert spaces and showed their existence.
result Existence of infinite-dimensional weakly reflective PF submanifolds in Hilbert spaces.
We introduce the notions of pointwise almost h-slant submanifolds and pointwise almost h-semi-slant submanifolds as a generalization of slant submanifolds, pointwise slant submanifolds, semi-slant submanifolds, and pointwise semi-slant submanifolds. We have characterizations and investigate the integrability of distrib…