New SDE model for continuous-time reinforcement learning.
problem Modeling exploration in continuous-time reinforcement learning.
method Introduced grid-sampling SDE as a proxy model.
result Wellposedness of the SDE in the presence of jumps.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “continual reinforcement learning”
Continuous reinforcement learning such as DDPG and A3C are widely used in robot control and autonomous driving. However, both methods have theoretical weaknesses. While DDPG cannot control noises in the control process, A3C does not satisfy the continuity conditions under the Gaussian policy. To address these concerns,…
Logarithmic regret for continuous-time reinforcement learning.
problem Continuous-time Markov decision processes with unknown transition probabilities and holding times.
method Upper confidence reinforcement learning, mean holding time estimation, stochastic comparison of point processes.
result Logarithmic regret bound achieved in finite time.
Most artificial intelligence models have limiting ability to solve new tasks faster, without forgetting previously acquired knowledge. The recently emerging paradigm of continual learning aims to solve this issue, in which the model learns various tasks in a sequential fashion. In this work, a novel approach for contin…
Paper explores using LLMs for zero-shot reinforcement learning in continuous spaces.
problem Leveraging LLMs for continuous state spaces in reinforcement learning.
method Disentangled In-Context Learning (DICL) to handle multivariate data and control signal.
result DICL produces well-calibrated uncertainty estimates in reinforcement learning settings.
Develops DPG methods for continuous-time RL with deterministic policies.
problem High variance and slow convergence in stochastic policy RL methods.
method Derives continuous-time policy gradient formula and proposes CT-DDPG algorithm.
result CT-DDPG achieves superior stability and faster convergence in continuous-time RL.
Logarithmic regret achieved in continuous-time linear-quadratic reinforcement learning.
problem Optimizing control actions in unknown continuous-time systems over a finite time horizon.
method Least-squares algorithm based on continuous-time observations and controls, with perturbation analysis and parameter estimation error analysis.
result Logarithmic regret bound of order .
This paper introduces Dex, a reinforcement learning environment toolkit specialized for training and evaluation of continual learning methods as well as general reinforcement learning problems. We also present the novel continual learning method of incremental learning, where a challenging environment is solved using o…
Many real-world control problems involve both discrete decision variables - such as the choice of control modes, gear switching or digital outputs - as well as continuous decision variables - such as velocity setpoints, control gains or analogue outputs. However, when defining the corresponding optimal control or reinf…
Paper establishes baselines for offline RL from visual observations.
problem Challenges in offline reinforcement learning from visual observations with continuous action spaces.
method Simple baselines and benchmarking tasks for offline RL from visual observations.
result Simple modifications to existing online RL algorithms outperform existing offline RL methods.
New RL framework models continuous-time dynamics using neural ODEs.
problem Modeling continuous-time dynamics in semi-Markov decision processes.
method Model-based reinforcement learning with neural ODEs.
result High-performing policies developed with minimal data.
Continuous-time optimal stopping solved with deep reinforcement learning
problem Optimal stopping problems in continuous time
method CARLOS (Continuous-time Adaptive Reinforcement Learning for Optimal Stopping)
result Higher prices than existing Bermudan solvers, approaching American upper bound
Study of discrete-time mean-variance model using reinforcement learning.
problem Discrete-time model with more general return distribution assumptions.
method Entropy-based exploration cost, reinforcement learning algorithm design.
result Optimal investment strategy with Gaussian density function.
Continuous-time MBRL framework tackles control systems with Bayesian ODEs.
problem Discretization of continuous-time systems in MBRL.
method Novel actor-critic method with Bayesian ODEs for state inference.
result Model robust against irregular and noisy data, sample-efficient, solves challenging control problems.
Novel framework proves fast RL convergence in continuous spaces.
problem Analyzing stability in continuous state-action RL.
method Introduces a novel framework to analyze stability properties of RL.
result Highlights two key stability properties and demonstrates their satisfaction in RL.
A new trading model combines GARCH and PPO for better stock trading profits.
problem Limited performance of reinforcement learning in stock market trading.
method Parallel-network continuous trading model using GARCH and PPO.
result The model achieves more profit compared to traditional reinforcement learning methods.
Study shows certainty equivalent policy minimizes regret in continuous-time systems.
problem Minimizing regret in continuous-time stochastic linear-quadratic systems.
method Theoretical analysis of randomized certainty equivalent policy.
result Establishes square-root of time regret bounds and linear scaling with parameters.
Deep reinforcement learning has made significant progress in the field of continuous control, such as physical control and autonomous driving. However, it is challenging for a reinforcement model to learn a policy for each task sequentially due to catastrophic forgetting. Specifically, the model would forget knowledge …
High-dimensional always-changing environments constitute a hard challenge for current reinforcement learning techniques. Artificial agents, nowadays, are often trained off-line in very static and controlled conditions in simulation such that training observations can be thought as sampled i.i.d. from the entire observa…
Improved reinforcement learning with deep learning.
problem Extending MultiGrid Reinforcement Learning to work with deep learning.
method Combining potential-based reward shaping with a learned potential function from interaction, and adapting it for deep learning algorithms.
result DQN augmented with the approach performs significantly better on continuous control tasks.
Continuous-time reinforcement learning: ellipticity enables model-free value function approximationcs.LG
New algorithm learns value and advantage functions for continuous-time Markov processes without structural assumptions.
problem Learning value and advantage functions for continuous-time Markov processes without structural assumptions.
method Proposes Sobolev-prox fitted -learning algorithm based on Hilbert-space positive definiteness and boundedness properties of Bellman operators.
result Identifies ellipticity as a key structural property enabling reinforcement learning for Markov diffusions.
New method learns policies from offline data using operator models.
problem Limited understanding of approximation errors in offline reinforcement learning.
method Linking reinforcement learning to Hamilton-Jacobi-Bellman equation, proposing operator-theoretic algorithm.
result Global convergence of the value function and finite-sample guarantees derived.
Paper tackles continual reinforcement learning by forgetting, proposing a planning method with online world models.
problem Catastrophic forgetting in reinforcement learning when learning new tasks.
method Planning with an online world model using model predictive control.
result The proposed FTL Online Agent (OA) learns new tasks without forgetting old skills.
This paper introduces a new reward shaping method for average-reward reinforcement learning.
problem Speeding up convergence to an optimal policy in average-reward reinforcement learning tasks.
method Developed a temporal logic-based approach to automatically generate reward shaping functions.
result The optimal policy can be recovered using the proposed reward shaping framework.
Deep reinforcement learning enables algorithms to learn complex behavior, deal with continuous action spaces and find good strategies in environments with high dimensional state spaces. With deep reinforcement learning being an active area of research and many concurrent inventions, we decided to focus on a relatively …
We solve continuous-time reinforcement learning using distributional Hamilton-Jacobi-Bellman equations.
problem Predicting the distribution of returns in continuous-time, stochastic environments.
method We derive a distributional Hamilton-Jacobi-Bellman equation for Itô diffusions and Feller-Dynkin processes, and propose an algorithm based on a JKO scheme.
result We propose an online control algorithm that can be used to approximately solve the distributional HJB equation.
New IRL algorithm for continuous state spaces with formal guarantees.
problem Finding a reward function for expert behavior in continuous state spaces.
method Modeling the system using orthonormal functions and providing correctness proofs.
result Proof of correctness and formal guarantees on sample and time complexity.
Novel Bayesian meta-reinforcement learning framework improves traffic signal control robustness.
problem Lack of robustness and stability in adaptation for traffic signal control.
method Value-based Bayesian meta-reinforcement learning framework BM-DQN with fast-adaptation variation and DQN fast-update advantage.
result Framework adapts more quickly and robustly to new scenarios than previous methods.
Safe reinforcement learning framework using optimal transport for robustness.
problem Robustness and safety in deep reinforcement learning with limited data assumptions.
method Optimal transport perturbations to construct worst-case virtual state transitions.
result Significantly improved safety at deployment time compared to standard methods.
New framework for policy gradient methods in continuous time reinforcement learning.
problem Addressing policy gradient methods for continuous time reinforcement learning.
method Control randomisation technique to derive policy gradient representation for various Markovian control problems.
result Demonstrated application to optimal switching problems in the energy sector.
New method improves RL in continuous spaces with kernel smoothing.
problem Sample efficiency and structural assumptions in classical RL.
method Kernel smoothing model-based approach with Bernstein-style exploration bonus.
result Achieves improved regret bound in finite-horizon settings.
New STDP rule for spiking neurons solves discrete action reinforcement learning tasks.
problem Applying standard STDP to discrete action reinforcement learning tasks.
method Feedback-modulated TD-STDP learning rule for spiking neuron networks.
result Feedback modulation improves credit assignment in reinforcement learning.
This paper tackles online reinforcement learning for unseen tasks with unknown boundaries.
problem Real-world tasks violate assumptions of task distributions, independence, and clear task delineations.
method A mixture of Gaussian Processes models different dynamics, and a transition prior handles temporal dependencies.
result The approach reliably handles task distribution shifts and outperforms alternatives in non-stationary tasks.
Ensuring Monotonic Policy Improvement in Entropy-regularized Value-based Reinforcement Learningcs.LG
This paper improves reinforcement learning policies in a scalable way.
problem Ensuring monotonic policy improvement in entropy-regularized RL.
method Derives an entropy-aware lower bound and proposes a novel RL algorithm.
result Demonstrates effectiveness in continuous-state tasks using a linear function approximator.
This paper proposes GProp, a deep reinforcement learning algorithm for continuous policies with compatible function approximation. The algorithm is based on two innovations. Firstly, we present a temporal-difference based method for learning the gradient of the value-function. Secondly, we present the deviator-actor-cr…
A new buffer system improves continual learning in RL agents by adapting to changing environments.
problem Improving RL agents' ability to learn from changing environments over time.
method Multi-timescale replay buffer combined with invariant risk minimization.
result The method shows improvement over baselines in continual learning settings.
We propose a method for tackling catastrophic forgetting in deep reinforcement learning that is \textit{agnostic} to the timescale of changes in the distribution of experiences, does not require knowledge of task boundaries, and can adapt in \textit{continuously} changing environments. In our \textit{policy consolidati…
Paper tackles RL with continuous actions and unmeasured confounders.
problem Offline policy learning with continuous actions and unmeasured confounders.
method Developed a novel identification result and a minimax estimator for nonparametric policy value estimation.
result Introduced a policy-gradient-based algorithm to identify the optimal policy.
Study policy gradient and actor-critic methods for continuous-time reinforcement learning.
problem Continuous-time reinforcement learning with policy gradient and actor-critic approaches.
method Regularized exploratory formulation, martingale approach, simultaneous policy and value function updates.
result Proposed two types of actor-critic algorithms for online and offline learning.
RL approach for continuous-time mean-variance portfolio selection with empirical validation.
problem Continuous-time mean-variance portfolio selection in unknown market coefficients.
method Reinforcement learning for diffusion processes, sublinear regret bound derivation.
result RL strategy consistently outperforms model-based counterparts, especially in volatile markets.
Continuous-time Q-learning theory developed for reinforcement learning.
problem Continuous-time reinforcement learning challenges.
method Entropy-regularized, exploratory diffusion process formulation; first-order approximation of Q-function; martingale conditions.
result Developed a q-learning theory independent of time discretization.
RANDPOL uses randomized networks for efficient reinforcement learning in continuous state and action MDPs.
problem Efficient reinforcement learning in environments with continuous state and action spaces.
method RANDPOL uses randomized function approximation to represent policy and value functions, providing finite time guarantees and improved numerical performance.
result RANDPOL achieves better numerical performance and provides finite time guarantees compared to deep neural network based algorithms.
Reinforcement learning for continuous-time risk-sensitive asset allocation
problem Continuous-time risk-sensitive asset allocation
method Free energy-entropy duality reformulation and -learning actor-critic method
result Optimal policy learning with high accuracy
Learn Continuously, Act Discretely: Hybrid Action-Space Reinforcement Learning For Optimal Executionq-fin.TR
Hybrid RL method optimizes trading by balancing continuous and discrete actions.
problem Optimal execution in algorithmic trading with continuous-discrete action space.
method Combines continuous and discrete RL agents for better trading decisions.
result Significantly outperforms existing methods in trading efficiency and stability.
This research introduces an autonomous robot navigation method using reinforcement learning.
problem Improving robot navigation in complex environments.
method Deep Q Network (DQN) and Proximal Policy Optimization (PPO) models for path planning and decision-making.
result The models enhance robot navigation ability and adaptive learning in unknown environments.
Reliable and effective multi-task learning is a prerequisite for the development of robotic agents that can quickly learn to accomplish related, everyday tasks. However, in the reinforcement learning domain, multi-task learning has not exhibited the same level of success as in other domains, such as computer vision. In…
This study analyzes convergence and stability of reinforcement learning algorithms.
problem Understanding the conditions under which reinforcement learning algorithms converge and remain stable.
method Theoretical analysis of convergence and stability of Episodic Upside-Down Reinforcement Learning, Goal-Conditioned Supervised Learning, and Online Decision Transformers.
result The algorithms can achieve near-optimal behavior if the transition kernel is close to a deterministic kernel.
Reinforcement learning mimics expert behavior.
problem Learning from expert demonstrations in reinforcement learning.
method Reduction to reinforcement learning with a stationary reward.
result Expert reward can be recovered and imitation learning is bounded.