arXiv research

Paper presents a reduction-based framework for conservative bandits and RL with improved lower and upper bounds.

RORL improves offline RL robustness with conservative smoothing.

BCPO optimizes offline RL policies by converting uncertainty into conservative bounds.

CQL learns conservative Q-functions to improve offline RL performance.

Paper proposes an RL algorithm to ensure policy performance guarantees.

The paper tackles safe exploration in RL by a conservative safety critic.

Offline RL policies should adapt to unknown aspects of the environment.

New offline RL algorithms tackle partial data coverage with optimal performance and practicality.

Proposes a method to improve few-shot transfer in off-dynamics RL.

Conservative Policy Iteration (CPI) is a founding algorithm of Approximate Dynamic Programming (ADP). Its core principle is to stabilize greediness through stochastic mixtures of consecutive policies. It comes with strong theoretical guarantees, and inspired approaches in deep Reinforcement Learning (RL). However, CPI …

New batch RL method avoids overly optimistic policies.

New method finds unseen states for RL, improving performance.

VaR-CPO optimizes VaR-constrained RL problems with conservative policy updates.

Canary optimizes VaR-constrained RL problems with a conservative bound using Cantelli's inequality.

Proposes a conservative exploration method for RL agents.

AACC improves RL performance in changing environments.

BRPO optimizes batch RL policies to better exploit state-action differences.

Dynamic Programming (DP) provides standard algorithms to solve Markov Decision Processes. However, these algorithms generally do not optimize a scalar objective function. In this paper, we draw connections between DP and (constrained) convex optimization. Specifically, we show clear links in the algorithmic structure b…

Pre-trained LLM adapted with LoRA improves offline RL for quantitative trading.

We provide a framework for incorporating robustness -- to perturbations in the transition dynamics which we refer to as model misspecification -- into continuous control Reinforcement Learning (RL) algorithms. We specifically focus on incorporating robustness into a state-of-the-art continuous control RL algorithm call…

AdaRL improves robust RL by adaptively adjusting policy complexity.

CQL (ReDS) learns from varied driving behaviors, improving offline RL performance.

Entropy regularized algorithms such as Soft Q-learning and Soft Actor-Critic, recently showed state-of-the-art performance on a number of challenging reinforcement learning (RL) tasks. The regularized formulation modifies the standard RL objective and thus generally converges to a policy different from the optimal gree…

Trust region policy optimization (TRPO) is a popular and empirically successful policy search algorithm in Reinforcement Learning (RL) in which a surrogate problem, that restricts consecutive policies to be 'close' to one another, is iteratively solved. Nevertheless, TRPO has been considered a heuristic algorithm inspi…

BRAID fine-tunes diffusion models to optimize reward models in offline scenarios.

RL helps optimize TVS fund composition for volatility control.

MoMA improves model-based RL by using unrestricted policy classes.

Study shows offline RL under $Q^\star$-approximation and partial coverage is harder than previously thought.

A novel approach for safe offline RL using latent safety constraints.

A new method, Count-MORL, improves offline reinforcement learning by using state-action frequency.

Real-world applications require RL algorithms to act safely. During learning process, it is likely that the agent executes sub-optimal actions that may lead to unsafe/poor states of the system. Exploration is particularly brittle in high-dimensional state/action space due to increased number of low-performing actions. …

Safe exploration method for RL under disturbance ensures safety with probabilistic guarantees.

Theoretical analysis confirms non-conservative algorithms can converge to optimal policies.

Study finds conserved quantities for two types of curves on conformal sphere.

Batch Reinforcement Learning (Batch RL) consists in training a policy using trajectories collected with another policy, called the behavioural policy. Safe policy improvement (SPI) provides guarantees with high probability that the trained policy performs better than the behavioural policy, also called baseline in this…

Paper presents a modular RL framework for Forex trading, addressing limitations of prior studies.

Survey on conservation laws for geometric PDEs.

The article discusses conservation laws for polyharmonic maps and their applications.

I consider the existence and structure of conservation laws for the general class of evolutionary scalar second-order differential equations with parabolic symbol. First I calculate the linearized characteristic cohomology for such equations. This provides an auxiliary differential equation satisfied by the conservatio…

Non-trivial conservation law found for a specific system.

In recent years, there has been significant progress in applying deep reinforcement learning (RL) for solving challenging problems across a wide variety of domains. Nevertheless, convergence of various methods has been shown to suffer from inconsistencies, due to algorithmic instability and variance, as well as stochas…

New RL algorithms learn policies competitive with best in class without assuming optimal policy.

This work connects symmetries and conserved quantities in machine learning.

Conservation law for weakly harmonic mappings in high dimensions.

The conservation laws of the third order quasilinear scalar evolution equations are considered via differential system and characteristic cohomology. We find a subspace of 2 forms in the infinite prolonged space in which every conservation law has a unique representative. The structure of this subspace naturally gives …

Given a vector field on a manifold M, we define a globally conserved quantity to be a differential form whose Lie derivative is exact. Integrals of conserved quantities over suitable submanifolds are constant under time evolution, the Kelvin circulation theorem being a well-known special case. More generally, conserved…

We study higher-order conservation laws of the non-linearizable elliptic Poisson equation $\frac{{\partial}^2 u}{\partial z \partial \bar{z}} = -f(u)$ as elements of the characteristic cohomology of the associated exterior differential system. The theory of characteristic cohomology determines a normal form for diffe…

New conservation laws found for polyharmonic maps in critical dimension.