arXiv research

CPME embeds counterfactual outcomes in RKHS for flexible policy evaluation.

A new method estimates multi-dimensional value distributions using Hilbert space embeddings.

CAEL-MIPS learns embeddings to improve MIPS for better OPE in contextual bandits.

New OPE estimator improves offline policy evaluation for large action spaces.

Bayesian approach for modeling counterfactual distribution and off-policy evaluation.

Embedding RL policies in RKHS for robustness and theoretical guarantees.

Recently, deep reinforcement learning (RL) methods have been applied successfully to multi-agent scenarios. Typically, these methods rely on a concatenation of agent states to represent the information content required for decentralized decision making. However, concatenation scales poorly to swarm systems with a large…

New algorithms learn graphons in GMFGs without knowing them.

The classical dynamic programming-based optimal stochastic control methods fail to cope with nonseparable dynamic optimization problems as the principle of optimality no longer applies in such situations. Among these notorious nonseparable problems, the dynamic mean-variance portfolio selection formulation had posted a…

New algorithm tackles dynamic query routing to multiple embedding models.

The paper solves multi-period portfolio selection with constraints using a dynamic factor model.

New method learns state embeddings from demonstrations for improved reinforcement learning.

SymNet uses neural models to solve RMDPs efficiently.

This paper optimizes off-policy evaluation in reinforcement learning with function approximation.

A nonparametric approach for policy learning for POMDPs is proposed. The approach represents distributions over the states, observations, and actions as embeddings in feature spaces, which are reproducing kernel Hilbert spaces. Distributions over states given the observations are obtained by applying the kernel Bayes' …

New estimator GMIPS reduces variance in ranking policy evaluation.

We propose a novel framework for multi-task reinforcement learning (MTRL). Using a variational inference formulation, we learn policies that generalize across both changing dynamics and goals. The resulting policies are parametrized by shared parameters that allow for transfer between different dynamics and goal condit…

New model combines neural networks and embeddings for better choice modeling interpretability.

Two algorithms improve Federated RL in diverse environments.

In this paper we consider self-supervised representation learning to improve sample efficiency in reinforcement learning (RL). We propose a forward prediction objective for simultaneously learning embeddings of states and action sequences. These embeddings capture the structure of the environment's dynamics, enabling e…

New method improves reinforcement learning generalization.

Study policy gradient for large-agent mean-field control and game in continuous time.

Naive investors make riskier choices than optimal strategies in continuous-time finance.

Softmax policy gradient achieves global optimality in wide neural networks with entropy regularization.

Counterfactual inference has become a ubiquitous tool in online advertisement, recommendation systems, medical diagnosis, and econometrics. Accurate modeling of outcome distributions associated with different interventions -- known as counterfactual distributions -- is crucial for the success of these applications. In …

This paper provides a dictionary of closed-form kernel mean embeddings.

Unified q-learning for mean-field jump-diffusion models with unobservable population distribution.

A new RL algorithm POWR learns world models to estimate action-values.

Proposes Deep LTMLE for estimating dynamic treatment effects in longitudinal studies.

This note optimizes distributions using kernel mean embeddings with a new parameterization.

Study shows policy gradient convergence for entropy-regularized MDPs with neural nets in mean-field regime.

We offer a new, rigorous approach to conditional mean embeddings without operator constraints.

We consider the problem of off-policy evaluation in Markov decision processes. Off-policy evaluation is the task of evaluating the expected return of one policy with data generated by a different, behavior policy. Importance sampling is a technique for off-policy evaluation that re-weights off-policy returns to account…

In hierarchical reinforcement learning a major challenge is determining appropriate low-level policies. We propose an unsupervised learning scheme, based on asymmetric self-play from Sukhbaatar et al. (2018), that automatically learns a good representation of sub-goals in the environment and a low-level policy that can…

MF-TRPO optimizes MFGs with finite sample guarantees.

Agent learns directed exploration policies to improve performance in hard games.

We propose a new algorithm, Mean Actor-Critic (MAC), for discrete-action continuous-state reinforcement learning. MAC is a policy gradient algorithm that uses the agent's explicit representation of all action values to estimate the gradient of the policy, rather than using only the actions that were actually executed. …

Foreign policy analysis has been struggling to find ways to measure policy preferences and paradigm shifts in international political systems. This paper presents a novel, potential solution to this challenge, through the application of a neural word embedding (Word2vec) model on a dataset featuring speeches by heads o…

This paper proposes a method to evaluate policies using quantile metrics, improving upon existing mean-based approaches.

We provide a theoretical foundation for non-parametric estimation of functions of random variables using kernel mean embeddings. We show that for any continuous function $f$, consistent estimators of the mean embedding of a random variable $X$ lead to consistent estimators of the mean embedding of $f(X)$. For Matérn ke…

New RL algorithms achieve optimal policies with polynomial sample complexity for mean-field problems.

Efficiently approximates kernel mean embeddings using Nyström method.

The ability to transfer in reinforcement learning is key towards building an agent of general artificial intelligence. In this paper, we consider the problem of learning to simultaneously transfer across both environments (ENV) and tasks (TASK), probably more importantly, by learning from only sparse (ENV, TASK) pairs …

New KQEs improve probability metrics without mean function constraints.

Study optimal treatment assignment policies under strategic agent responses.

Authors construct hypertori with constant negative mean curvature in a sphere.

Investors benefit from long horizons in a market with mean-reverting equity returns.

Conditional kernel mean embeddings form an attractive nonparametric framework for representing conditional means of functions, describing the observation processes for many complex models. However, the recovery of the original underlying function of interest whose conditional mean was observed is a challenging inferenc…