arXiv research

New algorithm achieves optimal regret in average reward MDPs without prior bias information.

New algorithm reduces sample complexity for constrained MDPs.

New method optimizes policies without assuming known link functions between preferences and rewards.

Study improves unbiased recommender learning by addressing missing-reward bias.

VaR-CPO optimizes VaR-constrained RL problems with conservative policy updates.

Novel methods generate diverse policies in reinforcement learning.

FOCOPS optimizes agent's behavior while adhering to constraints.

KL-constrained API shows optimization issues and improved with regularization.

We find the optimal error for a constrained regression model under a linear model.

PESCAL uses mediators to learn from confounded offline data.

A new algorithm CAP learns optimal policies from observational data with confounding bias and missing observations.

The paper optimizes policies constrained to Schur stabilizing controllers using a Newton-type algorithm.

Solving tasks in Reinforcement Learning is no easy feat. As the goal of the agent is to maximize the accumulated reward, it often learns to exploit loopholes and misspecifications in the reward signal resulting in unwanted behavior. While constraints may solve this issue, there is no closed form solution for general co…

New methods improve off-policy evaluation for survival outcomes with censoring.

Oracle-efficient algorithm for offline RL with partial data coverage.

A new approach combines prior knowledge with learning to adapt quickly to new tasks.

A new Q-learning variant reduces underestimation bias in deep reinforcement learning.

ConQUR tackles delusional bias in deep Q-learning, improving performance in Atari games.

Offline RL tackles resource-constrained online deployment with improved policy transfer.

Proposes a method to learn policies from offline data with reduced bias.

DML-IV improves IV regression for learning decision policies by reducing bias.

Framework improves policy generalizability under biased training data.

This work overcomes bias in concave multi-objective reinforcement learning.

Infinite horizon off-policy policy evaluation is a highly challenging task due to the excessively large variance of typical importance sampling (IS) estimators. Recently, Liu et al. (2018a) proposed an approach that significantly reduces the variance of infinite-horizon off-policy evaluation by estimating the stationar…

New estimator reduces bias-variance tradeoff in Markovian interference experiments.

Canary optimizes VaR-constrained RL problems with a conservative bound using Cantelli's inequality.

Paper proposes a bias-constrained deep learning approach to non-linear estimation.

VRER selectively reuses past observations to reduce variance in policy optimization.

In importance sampling (IS)-based reinforcement learning algorithms such as Proximal Policy Optimization (PPO), IS weights are typically clipped to avoid large variance in learning. However, policy update from clipped statistics induces large bias in tasks with high action dimensions, and bias from clipping makes it di…

Machine learning models predict brain age with systematic bias, corrected in this study.

Automates bias control in reinforcement learning algorithms.

Proposes Constrained Q-learning for reinforcement learning with constraints.

PDCA algorithm learns policies for RL with constraints using a primal-dual approach.

New model reduces matrix factorization bias, yielding truly low-rank solutions.

BRPO optimizes batch RL policies to better exploit state-action differences.

Deep reinforcement learning (DRL) on Markov decision processes (MDPs) with continuous action spaces is often approached by directly training parametric policies along the direction of estimated policy gradients (PGs). Previous research revealed that the performance of these PG algorithms depends heavily on the bias-var…

A new estimator reduces bias and variance in ranking policy evaluation.

Paper proposes risk-averse reinforcement learning algorithms.

We propose a policy improvement algorithm for Reinforcement Learning (RL) which is called Rerouted Behavior Improvement (RBI). RBI is designed to take into account the evaluation errors of the Q-function. Such errors are common in RL when learning the $Q$-value from finite past experience data. Greedy policies or even …

Paper tackles overestimation bias in continuous control, improving performance by 25%.

This work compares regularization and constrained inference for label constraints in machine learning.

For an autonomous agent, executing a poor policy may be costly or even dangerous. For such agents, it is desirable to determine confidence interval lower bounds on the performance of any given policy without executing said policy. Current methods for exact high confidence off-policy evaluation that use importance sampl…

Algorithm optimizes constrained reinforcement learning with dual variables.

DOLCE improves off-policy evaluation and learning by decomposing effects.

QFIL improves offline RL by filtering data to reduce bias and variance.

FFN addresses spectral bias in neural value approximation, improving reinforcement learning performance.

We consider the core reinforcement-learning problem of on-policy value function approximation from a batch of trajectory data, and focus on various issues of Temporal Difference (TD) learning and Monte Carlo (MC) policy evaluation. The two methods are known to achieve complementary bias-variance trade-off properties, w…

KCRL learns stable policies for nonlinear systems with formal guarantees.