arXiv research

Theoretical analysis confirms non-conservative algorithms can converge to optimal policies.

Proposes a conservative exploration method for RL agents.

A new algorithm balances exploration and exploitation in online decision-making.

This work extends balancing to various simulation-based inference algorithms for more conservative posterior approximations.

An online reinforcement learning algorithm is anytime if it does not need to know in advance the horizon T of the experiment. A well-known technique to obtain an anytime algorithm from any non-anytime algorithm is the "Doubling Trick". In the context of adversarial or stochastic multi-armed bandits, the performance of …

StepMix algorithm ensures safe exploration in reinforcement learning with near-optimal performance.

The problem of multi-armed bandits (MAB) asks to make sequential decisions while balancing between exploitation and exploration, and have been successfully applied to a wide range of practical scenarios. Various algorithms have been designed to achieve a high reward in a long term. However, its short-term performance m…

A challenging problem in complex networks is the network reconstruction problem from data. This work deals with a class of networks denoted as conserved networks, in which a flow associated with every edge and the flows are conserved at all non-source and non-sink nodes. We propose a novel polynomial time algorithm to …

In many fields such as digital marketing, healthcare, finance, and robotics, it is common to have a well-tested and reliable baseline policy running in production (e.g., a recommender system). Nonetheless, the baseline policy is often suboptimal. In this case, it is desirable to deploy online learning algorithms (e.g.,…

While learning in an unknown Markov Decision Process (MDP), an agent should trade off exploration to discover new information about the MDP, and exploitation of the current knowledge to maximize the reward. Although the agent will eventually learn a good or optimal policy, there is no guarantee on the quality of the in…

RORL improves offline RL robustness with conservative smoothing.

Develops ECD framework for optimizing machine learning problems.

A new AI optimization method uses energy-conserving dynamics inspired by Born-Infeld theory.

BNRE improves simulation-based inference by producing more conservative posteriors.

Paper proposes an RL algorithm to ensure policy performance guarantees.

Major internet companies routinely perform tens of thousands of A/B tests each year. Such large-scale sequential experimentation has resulted in a recent spurt of new algorithms that can provably control the false discovery rate (FDR) in a fully online fashion. However, current state-of-the-art adaptive algorithms can …

Unified routing and arbitrage with concave continuation.

Proposes a method to avoid excessive exploration in reinforcement learning.

Safety is a desirable property that can immensely increase the applicability of learning algorithms in real-world decision-making problems. It is much easier for a company to deploy an algorithm that is safe, i.e., guaranteed to perform at least as well as a baseline. In this paper, we study the issue of safety in cont…

ECD algorithm speeds up non-convex optimization, offering quantum and stochastic enhancements.

Study finds conserved quantities for two types of curves on conformal sphere.

Imitation learning, followed by reinforcement learning algorithms, is a promising paradigm to solve complex control tasks sample-efficiently. However, learning from demonstrations often suffers from the covariate shift problem, which results in cascading errors of the learned policy. We introduce a notion of conservati…

Survey on conservation laws for geometric PDEs.

The article discusses conservation laws for polyharmonic maps and their applications.

Paper presents a reduction-based framework for conservative bandits and RL with improved lower and upper bounds.

I consider the existence and structure of conservation laws for the general class of evolutionary scalar second-order differential equations with parabolic symbol. First I calculate the linearized characteristic cohomology for such equations. This provides an auxiliary differential equation satisfied by the conservatio…

Conservative Policy Iteration (CPI) is a founding algorithm of Approximate Dynamic Programming (ADP). Its core principle is to stabilize greediness through stochastic mixtures of consecutive policies. It comes with strong theoretical guarantees, and inspired approaches in deep Reinforcement Learning (RL). However, CPI …

SCPO learns robust policies without modeling disturbance, improving real-world task performance.

Global pairwise network alignment (GPNA) aims to find a one-to-one node mapping between two networks that identifies conserved network regions. GPNA algorithms optimize node conservation (NC) and edge conservation (EC). NC quantifies topological similarity between nodes. Graphlet-based degree vectors (GDVs) are a state…

Non-trivial conservation law found for a specific system.

Paper proposes COM-QEL to avoid overoptimistic solutions in offline optimization.

This work connects symmetries and conserved quantities in machine learning.

Two new algorithms optimize rewards while respecting safety constraints in sequential decisions.

Conservation law for weakly harmonic mappings in high dimensions.

The conservation laws of the third order quasilinear scalar evolution equations are considered via differential system and characteristic cohomology. We find a subspace of 2 forms in the infinite prolonged space in which every conservation law has a unique representative. The structure of this subspace naturally gives …

Given a vector field on a manifold M, we define a globally conserved quantity to be a differential form whose Lie derivative is exact. Integrals of conserved quantities over suitable submanifolds are constant under time evolution, the Kelvin circulation theorem being a well-known special case. More generally, conserved…

Automates discovery of interpretable Lagrangians from data.

We study higher-order conservation laws of the non-linearizable elliptic Poisson equation $\frac{{\partial}^2 u}{\partial z \partial \bar{z}} = -f(u)$ as elements of the characteristic cohomology of the associated exterior differential system. The theory of characteristic cohomology determines a normal form for diffe…

VaR-CPO optimizes VaR-constrained RL problems with conservative policy updates.

New conservation laws found for polyharmonic maps in critical dimension.

CQL learns conservative Q-functions to improve offline RL performance.

Noether's theorem clarifies how symmetries in neural networks influence learning.

MC-LSTM extends LSTM to conserve mass in neural networks.

The paper studies symmetries and conservation laws of non-diagonalisable hydrodynamic systems.

We present a connection between the Killing fields that arise in the loop-group approach to integrable systems and conservation laws viewed as elements of the characteristic cohomology. We use the connection to generate the complete set of conservation laws (as elements of the characteristic cohomology) for the Tzitzei…

New algorithms optimize a soft-robust criterion in reinforcement learning, reducing conservatism.

We obtain necessary and sufficient conditions for the existence of "conservation laws" on null hypersurfaces for the wave equation on general four-dimensional Lorentzian manifolds. Examples of null hypersurfaces exhibiting such conservation laws include the standard null cones of Minkowski spacetime and the degenerate …

New algorithm reduces pricing error by a factor of T^2/3.