arXiv research

New BE dimension measure reveals rich RL problems with sample-efficient algorithms.

New algorithm tackles non-stationary reinforcement learning with general function approximation.

New insights on eluder dimension for function approximation in machine learning.

Eluder dimension and information gain are equivalent for reproducing kernel Hilbert spaces.

Paper analyzes distributional reinforcement learning with value function approximation, introducing Bellman unbiasedness and a new algorithm.

Develops a new model for RLHF accounting for partially observed states and intermediate feedback.

This work improves sample efficiency in neural function approximation for reinforcement learning.

New algorithms achieve uniform-PAC guarantees for RL with bounded eluder dimension.

New insights show coverage conditions are crucial for efficient online reinforcement learning.

Survey of reinforcement learning guarantees with data constraints.

New RL method reduces sample complexity for large policy spaces.

Study optimal policy regret in partially observable Markov games with adaptive opponents.

We consider the problem of learning to optimize an unknown Markov decision process (MDP). We show that, if the MDP can be parameterized within some known function class, we can obtain regret bounds that scale with the dimensionality, rather than cardinality, of the system. We characterize this dependence explicitly as …

Paper develops efficient RL algorithm for general value function approximation.

New algorithm LOOP learns infinite-horizon AMDPs efficiently with function approximation.

Improved machine learning for reservoir optimization problems.

Deep neural nets approximate high-dimensional HJB equations efficiently.

Thompson Sampling is a well established approach to bandit and reinforcement learning problems. However its use in continuum armed bandit problems has received relatively little attention. We provide the first bounds on the regret of Thompson Sampling for continuum armed bandits under weak conditions on the function cl…

A new framework reduces RL sample complexity for complex MDPs.

Paper introduces dynamic strategies for multi-period investment models.

Unified framework for efficient RL in MDP, POMDP, and PSR.

Unified framework for constrained online decision-making.

Study experiment planning with function approximation in contextual bandit problems.

New neural net learns time-reversible symplectic dynamics.

Deep nets solve MDPs without high dimensions.

New algorithm finds Nash equilibrium in multi-agent games.

Sequential decision making in the presence of uncertainty and stochastic dynamics gives rise to distributions over state/action trajectories in reinforcement learning (RL) and optimal control problems. This observation has led to a variety of connections between RL and inference in probabilistic graphical models (PGMs)…

Paper proposes an algorithm for lifelong learning with shared structure.

Richard Bellman's Principle of Optimality, formulated in 1957, is the heart of dynamic programming, the mathematical discipline which studies the optimal solution of multi-period decision problems. In this paper, we look at the main trading principles of Jesse Livermore, the legendary stock operator whose method was pu…

Du, Kakade, Wang, and Yang recently established intriguing lower bounds on sample complexity, which suggest that reinforcement learning with a misspecified representation is intractable. Another line of work, which centers around a statistic called the eluder dimension, establishes tractability of problems similar to t…

New method solves continuous time mean-variance model for consistent investment strategy.

The paper studies the First Order BSPDEs (Backward Stochastic Partial Differential Equations) suggested earlier for a case of multidimensional state domain with a boundary. These equations represent analogs of Hamilton-Jacobi-Bellman equations and allow to construct the value function for stochastic optimal control pro…

Framework reduces contextual bandit learning to offline regression with near-optimal regret.

ENIAC method optimizes and explores complex RL problems with non-linear policies.

OE2D framework reduces contextual bandits to offline regression for near-optimal regret.

By using an explicit Bellman function, we prove a bilinear embedding theorem for the Laplacian associated with a weighted Riemannian manifold $(M,μ_φ)$ having the Bakry-Emery curvature bounded from below. The embedding, acting on the cartesian product of $L^p(M,μ_φ)$ and $L^q(T^*M,μ_φ)$, $1/p+1/q=1$, involves estimates…

We study the structure of a simple dynamic optimization problem consisting of one state and one control variable, from a physicist's point of view. By using an analogy to a physical model, we study this system in the classical and quantum frameworks. Classically, the dynamic optimization problem is equivalent to a clas…

Study contextual bandits with stage-wise constraints, proving regret bounds and extending results.

Choosing a portfolio of risky assets over time that maximizes the expected return at the same time as it minimizes portfolio risk is a classical problem in Mathematical Finance and is referred to as the dynamic Markowitz problem (when the risk is measured by variance) or more generally, the dynamic mean-risk problem. I…

RL in MFGs is as hard as solving many single-agent RL problems.

New findings on representation learning beyond linear functions.

The paper argues that attracting more economists and adopting a more-precise definition of dynamic complexity might help econophysics acquire more attention in the economics community and bring new lymph to economic research. It may be necessary to concentrate less on the applications than on the basics of economic com…

In this paper we propose and analyze a method based on the Riccati transformation for solving the evolutionary Hamilton-Jacobi-Bellman equation arising from the stochastic dynamic optimal allocation problem. We show how the fully nonlinear Hamilton-Jacobi-Bellman equation can be transformed into a quasi-linear paraboli…

New algorithm finds optimal policy with polynomial trajectories in deterministic systems.

Unified theory of $θ$-expectations derived from chaotic dynamics.

Theory explains how deep nets learn features from data.

This paper considers a non-Markov control problem arising in a financial market where asset returns depend on hidden factors. The problem is non-Markov because nonlinear filtering is required to make inference on these factors, and hence the associated dynamic program effectively takes the filtering distribution as one…

New actor-critic algorithm achieves optimal sample efficiency in RL.