arXiv research

Unified reinforcement learning methods using hybrid inference.

New method for robust inference on optimal treatment regimes without model specification.

Proposes HSIC-Lasso for selective inference in non-linear data.

MFRL-BI controls manufacturing processes without needing accurate models.

New algorithms cluster nodes in SBM graphs faster and more accurately.

VSE estimates complex processes from noisy measurements without a model.

Paper proposes a privacy-preserving knockoff inference method.

New RLHF algorithm identifies optimal policies from human feedback without explicit reward inference.

NIPA aims to translate brain learning mechanisms into scalable Bayesian inference.

New tools connect CP to GF inference for better probabilistic prediction.

Framework uses probabilistic programming for physics simulation in games.

Efficiently find near-optimal medical treatments with less trial and error.

We extend conformal inference to general settings that allow for time series data. Our proposal is developed as a randomization method and accounts for potential serial dependence by including block structures in the permutation scheme. As a result, the proposed method retains the exact, model-free validity when the da…

We propose a novel reformulation of the stochastic optimal control problem as an approximate inference problem, demonstrating, that such a interpretation leads to new practical methods for the original problem. In particular we characterise a novel class of iterative solutions to the stochastic optimal control problem …

Study compares model-free valuation to actual financial outcomes, finds it slightly conservative.

Dirichlet process mixture models (DPMM) are a cornerstone of Bayesian non-parametrics. While these models free from choosing the number of components a-priori, computationally attractive variational inference often reintroduces the need to do so, via a truncation on the variational distribution. In this paper we presen…

The paper reclassifies RL algorithms using inference concepts.

VMBPO optimizes model and policy jointly using variational lower-bound.

New model-free DR-RL algorithm with finite sample complexity.

Discovering and exploiting the causal structure in the environment is a crucial challenge for intelligent agents. Here we explore whether causal reasoning can emerge via meta-reinforcement learning. We train a recurrent network with model-free reinforcement learning to solve a range of problems that each contain causal…

We prove the existence and uniqueness of solutions of SDEs with Lipschitz coefficients, driven by continuous, model-free martingales. The main tool in our reasoning is Picard's iterative procedure and a model-free version of the Burkholder-Davis-Gundy inequality for integrals driven by model-free, continuous martingale…

Two new algorithms improve model-free RL for infinite-horizon MDPs.

Proposes a new model for estimating individual treatment effects.

Active inference implemented for high-dimensional tasks shows efficient exploration and improved sample efficiency.

New method uses transport maps for efficient Bayesian inference.

A new model-free algorithm achieves near-optimal regret for infinite-horizon MDPs.

New model-free algorithm achieves similar LQR regret guarantees.

Intelligent agents need a physical understanding of the world to predict the impact of their actions in the future. While learning-based models of the environment dynamics have contributed to significant improvements in sample efficiency compared to model-free reinforcement learning algorithms, they typically fail to g…

The paper proposes a new method to estimate optimal policies using MCMC.

We propose a second-order (Hessian or Hessian-free) based optimization method for variational inference inspired by Gaussian backpropagation, and argue that quasi-Newton optimization can be developed as well. This is accomplished by generalizing the gradient computation in stochastic backpropagation via a reparametriza…

FORK improves model-free reinforcement learning performance.

Advanced Dropout improves DNN performance without requiring model-specific dropout techniques.

Study improves probabilistic circuits using transformations for better predictions.

New hybrid RL algorithm outperforms model-free and model-based methods.

Simple object representations improve model-free RL performance.

Novel framework for Bayesian reinforcement learning infers value function distributions.

Develops model-free methods for event history analysis and efficient covariate adjustment.

New algorithms for model selection in off-policy evaluation of reinforcement learning.

In recent studies on model-based reinforcement learning (MBRL), incorporating uncertainty in forward dynamics is a state-of-the-art strategy to enhance learning performance, making MBRLs competitive to cutting-edge model free methods, especially in simulated robotics tasks. Probabilistic ensembles with trajectory sampl…

Proposes inference for DNNs in GNRMs, addressing non-independence issues.

We develop a general framework for distribution-free predictive inference in regression, using conformal inference. The proposed methodology allows for the construction of a prediction band for the response variable using any estimator of the regression function. The resulting prediction band preserves the consistency …

New algorithm borrows future randomness to stabilize model-free control.

Improved model-free RL algorithm with reduced sample complexity.

Improves sample efficiency in RL by matching model-based gradients.

We study an exploration method for model-free RL that generalizes the counter-based exploration bonus methods and takes into account long term exploratory value of actions rather than a single step look-ahead. We propose a model-free RL method that modifies Delayed Q-learning and utilizes the long-term exploration bonu…

A debiasing method improves nonparametric regression's statistical properties.

RandQL is a new model-free algorithm for MDPs with a novel learning rate randomization approach.

Model-free Reinforcement Learning (RL) algorithms such as Q-learning [Watkins, Dayan 92] have been widely used in practice and can achieve human level performance in applications such as video games [Mnih et al. 15]. Recently, equipped with the idea of optimism in the face of uncertainty, Q-learning algorithms [Jin, Al…