arXiv research

New BE dimension measure reveals rich RL problems with sample-efficient algorithms.

New algorithm tackles non-stationary reinforcement learning with general function approximation.

New insights on eluder dimension for function approximation in machine learning.

Eluder dimension and information gain are equivalent for reproducing kernel Hilbert spaces.

Paper analyzes distributional reinforcement learning with value function approximation, introducing Bellman unbiasedness and a new algorithm.

Develops a new model for RLHF accounting for partially observed states and intermediate feedback.

This work improves sample efficiency in neural function approximation for reinforcement learning.

New algorithms achieve uniform-PAC guarantees for RL with bounded eluder dimension.

New insights show coverage conditions are crucial for efficient online reinforcement learning.

Improved Thompson Sampling for smoother functions with noise.

Researchers compare lower bounds and tractability in reinforcement learning.

Survey of reinforcement learning guarantees with data constraints.

New RL method reduces sample complexity for large policy spaces.

Study optimal policy regret in partially observable Markov games with adaptive opponents.

We consider the problem of learning to optimize an unknown Markov decision process (MDP). We show that, if the MDP can be parameterized within some known function class, we can obtain regret bounds that scale with the dimensionality, rather than cardinality, of the system. We characterize this dependence explicitly as …

Paper develops efficient RL algorithm for general value function approximation.

New algorithm LOOP learns infinite-horizon AMDPs efficiently with function approximation.

Improved machine learning for reservoir optimization problems.

Deep neural nets approximate high-dimensional HJB equations efficiently.

A new framework reduces RL sample complexity for complex MDPs.

Paper introduces dynamic strategies for multi-period investment models.

Unified framework for efficient RL in MDP, POMDP, and PSR.

Unified framework for constrained online decision-making.

Study experiment planning with function approximation in contextual bandit problems.

New neural net learns time-reversible symplectic dynamics.

Deep nets solve MDPs without high dimensions.

New algorithm finds Nash equilibrium in multi-agent games.

Sequential decision making in the presence of uncertainty and stochastic dynamics gives rise to distributions over state/action trajectories in reinforcement learning (RL) and optimal control problems. This observation has led to a variety of connections between RL and inference in probabilistic graphical models (PGMs)…

Paper proposes an algorithm for lifelong learning with shared structure.

Richard Bellman's Principle of Optimality, formulated in 1957, is the heart of dynamic programming, the mathematical discipline which studies the optimal solution of multi-period decision problems. In this paper, we look at the main trading principles of Jesse Livermore, the legendary stock operator whose method was pu…

New method solves continuous time mean-variance model for consistent investment strategy.

The paper studies the First Order BSPDEs (Backward Stochastic Partial Differential Equations) suggested earlier for a case of multidimensional state domain with a boundary. These equations represent analogs of Hamilton-Jacobi-Bellman equations and allow to construct the value function for stochastic optimal control pro…

Framework reduces contextual bandit learning to offline regression with near-optimal regret.

ENIAC method optimizes and explores complex RL problems with non-linear policies.

OE2D framework reduces contextual bandits to offline regression for near-optimal regret.

Optimal reinsurance strategies for multi-line insurance companies.

By using an explicit Bellman function, we prove a bilinear embedding theorem for the Laplacian associated with a weighted Riemannian manifold $(M,μ_φ)$ having the Bakry-Emery curvature bounded from below. The embedding, acting on the cartesian product of $L^p(M,μ_φ)$ and $L^q(T^*M,μ_φ)$, $1/p+1/q=1$, involves estimates…

We study the structure of a simple dynamic optimization problem consisting of one state and one control variable, from a physicist's point of view. By using an analogy to a physical model, we study this system in the classical and quantum frameworks. Classically, the dynamic optimization problem is equivalent to a clas…

Study contextual bandits with stage-wise constraints, proving regret bounds and extending results.

Choosing a portfolio of risky assets over time that maximizes the expected return at the same time as it minimizes portfolio risk is a classical problem in Mathematical Finance and is referred to as the dynamic Markowitz problem (when the risk is measured by variance) or more generally, the dynamic mean-risk problem. I…

RL in MFGs is as hard as solving many single-agent RL problems.

New findings on representation learning beyond linear functions.

In this paper we propose and analyze a method based on the Riccati transformation for solving the evolutionary Hamilton-Jacobi-Bellman equation arising from the stochastic dynamic optimal allocation problem. We show how the fully nonlinear Hamilton-Jacobi-Bellman equation can be transformed into a quasi-linear paraboli…

New algorithm finds optimal policy with polynomial trajectories in deterministic systems.

Unified theory of $θ$-expectations derived from chaotic dynamics.

Theory explains how deep nets learn features from data.

This paper considers a non-Markov control problem arising in a financial market where asset returns depend on hidden factors. The problem is non-Markov because nonlinear filtering is required to make inference on these factors, and hence the associated dynamic program effectively takes the filtering distribution as one…

Econophysics needs more economists and precise definitions to gain attention.