arXiv research

This study optimizes offline reinforcement learning methods for various tasks without rewards.

Introduces a new manifold from a graph subgraph.

Proposes a new method to minimize non-singleton predictions in conformal prediction.

VRCQ algorithm reduces variance in Q-learning for MDPs, achieving optimal sample complexity.

Identifies conditions for multiple invariant probabilities in Markov kernels.

This work considers the sample and computational complexity of obtaining an $ε$-optimal policy in a discounted Markov Decision Process (MDP), given only access to a generative model. In this work, we study the effectiveness of the most natural plug-in approach to model-based planning: we build the maximum likelihood es…

Solves optimal control for stochastic processes with absorbing states.

New theorem removes uniform finite upper bound for shrinkability of null decompositions.

Bayesian framework for SSP problem learns optimal strategy through interactions.

The family of Wilder continua in cubes of dimension > 2 and its two subfamilies-of continuum-wise Wilder continua and of hereditarily arcwise connected continua-are recognized as coanalytic absorbers in the hyperspace of subcontinua of the cubes. In particular, each of them is homeomorphic to the set of all nonempty co…

New framework tackles bi-level optimization without LLS condition.

Deep neural networks near edge of chaos show universal scaling laws.

Rank-one measurements limit feasible sets for low-rank PSD matrices.

Researchers use information geometry to analyze and improve DRWs for node classification.

UDM reparameterization improves language model generation.

Adaptive reduction scheme approximates optimal policy in regularized MDPs.

We provide initial seedings to the Quick Shift clustering algorithm, which approximate the locally high-density regions of the data. Such seedings act as more stable and expressive cluster-cores than the singleton modes found by Quick Shift. We establish statistical consistency guarantees for this modification. We then…

We study Exo-MDPs to reduce sample complexity in reinforcement learning.

In this paper, a sparse Markov decision process (MDP) with novel causal sparse Tsallis entropy regularization is proposed.The proposed policy regularization induces a sparse and multi-modal optimal policy distribution of a sparse MDP. The full mathematical analysis of the proposed sparse MDP is provided.We first analyz…

Paper establishes new lower bounds for MDPs with changing transition kernels.

We consider the Markov Decision Process (MDP) of selecting a subset of items at each step, termed the Select-MDP (S-MDP). The large state and action spaces of S-MDPs make them intractable to solve with typical reinforcement learning (RL) algorithms especially when the number of items is huge. In this paper, we present …

This paper introduces the concept of kernels on fuzzy sets as a similarity measure for $[0,1]$-valued functions, a.k.a. \emph{membership functions of fuzzy sets}. We defined the following classes of kernels: the cross product, the intersection, the non-singleton and the distance-based kernels on fuzzy sets. Applicabili…

New method approximates POMDPs with PB-MDPs, providing error bounds and practical algorithms.

We consider large-scale Markov decision processes (MDPs) with parameter uncertainty, under the robust MDP paradigm. Previous studies showed that robust MDPs, based on a minimax approach to handle uncertainty, can be solved using dynamic programming for small to medium sized problems. However, due to the "curse of dimen…

Optimizes learning policies in MDPs with weakly communicating structure.

A Markov Decision Process (MDP) is a popular model for reinforcement learning. However, its commonly used assumption of stationary dynamics and rewards is too stringent and fails to hold in adversarial, nonstationary, or multi-agent problems. We study an episodic setting where the parameters of an MDP can differ across…

New method removes oracle and reduces memory usage for robust MDPs.

DeepAveragers solves offline RL by solving derived MDPs from static data.

New RL method learns to skip states in linearly $q^π$-realizable MDPs, simplifying to linear MDPs.

Optimizes learning policies in average-reward MDPs with improved sample complexity.

Efficiently plans large MDPs with weak function approximations.

We solve POMDPs by approximating them as finite-state MDPs.

Defines new extremal potentials and measures for Kähler forms.

Reward suffices for convex MDPs, expanding RL to new problems.

Practical reinforcement learning problems are often formulated as constrained Markov decision process (CMDP) problems, in which the agent has to maximize the expected return while satisfying a set of prescribed safety constraints. In this study, we propose a novel simulator-based method to approximately solve a CMDP pr…

The paper addresses statistical estimation in MDPs with confounders using instrumental variables.

Study on regret minimization in deterministic MDPs.

Improved regret bound for MNL MDPs with variance-aware approach.

This work uses action equivariance to learn structured latent spaces for reinforcement learning.

New method tackles MDPs by learning normalized representations efficiently.

Let $X$ be a Banach space and $Conv_H(X)$ be the space of non-empty closed convex subsets of $X$, endowed with the Hausdorff metric $d_H$. We prove that each connected component of the space $Conv_H(X)$ is homeomorphic to one of the spaces: a singleton, the real line, a closed half-plane, the Hilbert cube multiplied by…

A new parallel algorithm for learning optimal policies in MDPs with low communication costs.

It has recently been shown that if feedback effects of decisions are ignored, then imposing fairness constraints such as demographic parity or equality of opportunity can actually exacerbate unfairness. We propose to address this challenge by modeling feedback effects as Markov decision processes (MDPs). First, we prop…

The paper introduces MDP homomorphic networks for faster reinforcement learning.

We introduce and study a new family of extensions for the Borsuk-Ulam and topological Radon type theorems. The defining idea for this new family is to replace requirements of the form `a subset that is large in some sense goes to a singleton' with requirements of the milder form `a subset that is large in some sense go…

UCBVI-γ algorithm minimizes regret in discounted MDPs.

We introduce and analyse two algorithms for exploration-exploitation in discrete and continuous Markov Decision Processes (MDPs) based on exploration bonuses. SCAL$^+$ is a variant of SCAL (Fruit et al., 2018) that performs efficient exploration-exploitation in any unknown weakly-communicating MDP for which an upper bo…

New RL approach tackles non-linear MDPs without linear assumptions.