arXiv research

DARLING tackles non-stationary RL with guarantees, improving dynamic regret.

While designing the state space of an MDP, it is common to include states that are transient or not reachable by any policy (e.g., in mountain car, the product space of speed and position contains configurations that are not physically reachable). This leads to defining weakly-communicating or multi-chain MDPs. In this…

In classical reinforcement learning, when exploring an environment, agents accept arbitrary short term loss for long term gain. This is infeasible for safety critical applications, such as robotics, where even a single unsafe action may cause system failure. In this paper, we address the problem of safely exploring fin…

Improved exploration algorithm for unknown MDPs with reduced sample complexity.

We prove a lower bound for feature dimension in linear MDPs and propose a novel dynamics aggregation framework.

Paper identifies reductive MDPs, solving them in polynomial time.

We provide the first solution for model-free reinforcement learning of ω-regular objectives for Markov decision processes (MDPs). We present a constructive reduction from the almost-sure satisfaction of ω-regular objectives to an almost- sure reachability problem and extend this technique to learning how to control an …

Study improves understanding of why agentic theorem provers succeed.

New complexity measure helps in agnostic reinforcement learning with or without access to MDP dynamics.

This work explores efficient reinforcement learning with density features in low-rank MDPs.

Paper studies linear CMDPs, improving sample complexity for both models.

Study reward-free RL in non-linear settings, improving efficiency and removing assumptions.

Polynomial-time reachability for LTI systems with TLL NN controllers is achieved.

Statistical model checking for PCTL on MDPs using reinforcement learning.

TensorPlan shows an exponential lower bound for planning in MDPs with linearly realizable value functions.

Verifying correctness of deep neural networks (DNNs) is challenging. We study a generic reachability problem for feed-forward DNNs which, for a given set of inputs to the network and a Lipschitz-continuous function over its outputs, computes the lower and upper bound on the function values. Because the network and the …

We quantify content availability and user discovery opportunities in recommender systems.

Lumbermark clusters data robustly, slicing limbs of mutual reachability trees.

Guaranteed reachable set for unknown nonlinear systems on manifolds.

This paper tackles data-efficient nonlinear control in Hamiltonian systems using symplectic geometry.

GraphReach improves GNN performance by incorporating node positions.

Study on Heisenberg group's Lorentzian problems using Pontryagin's principle.

CIfly simplifies causal inference tasks with linear-time reachability primitives.

Procedure verifies if machine learning models assign fixed predictions that preclude access.

Study analyzes and enhances robustness of neural networks for classification and regression.

Deep learning method proves convergence for solving HJI equations.

Study of 2D Lorentzian anti-de Sitter plane using geometric control theory.

Improves data-driven reachability estimation for complex systems.

Quantum neural networks need both data-dependent and trainable unitaries for effective geometric deformation.

New reinforcement learning algorithm achieves instance-optimal sample complexity.

The study proposes an audit to assess user control over recommendations in collaborative filtering systems.

Compact reachability embeddings for hierarchical data

Influence maximization (IM) is the problem of finding for a given $s\geq 1$ a set $S$ of $|S|=s$ nodes in a network with maximum influence. With stochastic diffusion models, the influence of a set $S$ of seed nodes is defined as the expectation of its reachability over simulations, where each simulation specifies a det…

CAST improves spectral clustering for multi-scale data by integrating reachability similarity.

Adaptive reduction scheme approximates optimal policy in regularized MDPs.

We study Exo-MDPs to reduce sample complexity in reinforcement learning.

In this paper, a sparse Markov decision process (MDP) with novel causal sparse Tsallis entropy regularization is proposed.The proposed policy regularization induces a sparse and multi-modal optimal policy distribution of a sparse MDP. The full mathematical analysis of the proposed sparse MDP is provided.We first analyz…

Paper establishes new lower bounds for MDPs with changing transition kernels.

We consider the Markov Decision Process (MDP) of selecting a subset of items at each step, termed the Select-MDP (S-MDP). The large state and action spaces of S-MDPs make them intractable to solve with typical reinforcement learning (RL) algorithms especially when the number of items is huge. In this paper, we present …

New method approximates POMDPs with PB-MDPs, providing error bounds and practical algorithms.

We consider large-scale Markov decision processes (MDPs) with parameter uncertainty, under the robust MDP paradigm. Previous studies showed that robust MDPs, based on a minimax approach to handle uncertainty, can be solved using dynamic programming for small to medium sized problems. However, due to the "curse of dimen…

Optimizes learning policies in MDPs with weakly communicating structure.

A Markov Decision Process (MDP) is a popular model for reinforcement learning. However, its commonly used assumption of stationary dynamics and rewards is too stringent and fails to hold in adversarial, nonstationary, or multi-agent problems. We study an episodic setting where the parameters of an MDP can differ across…

New method removes oracle and reduces memory usage for robust MDPs.

How can we design safe reinforcement learning agents that avoid unnecessary disruptions to their environment? We show that current approaches to penalizing side effects can introduce bad incentives, e.g. to prevent any irreversible changes in the environment, including the actions of other agents. To isolate the source…

In finance industry portfolio construction deals with how to divide the investors' wealth across an asset-classes' menu in order to maximize the investors' gain. Main approaches in use at the present are based on variations of the classical Markowitz model. However, recent evolutions of the world market showed limitati…

DeepAveragers solves offline RL by solving derived MDPs from static data.

New RL method learns to skip states in linearly $q^π$-realizable MDPs, simplifying to linear MDPs.