A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
A new method reduces compounding errors in model-based reinforcement learning.
Model-based reinforcement learning is an appealing framework for creating agents that learn, plan, and act in sequential environments. Model-based algorithms typically involve learning a transition model that takes a state and an action and outputs the next state---a one-step model. This model can be composed with itse…
The paper tackles model-based RL's inaccuracy issue by dynamically adjusting planning horizons.
Action chunking and data exploration improve behavior cloning in robotics.
The task of multi-step ahead prediction in language models is challenging considering the discrepancy between training and testing. At test time, a language model is required to make predictions given past predictions as input, instead of the past targets that are provided during training. This difference, known as exp…
Abstract MDPs enable strategic exploration and fast reward transfer in complex environments.
We present a novel view that unifies two frameworks that aim to solve sequential prediction problems: learning to search (L2S) and recurrent neural networks (RNN). We point out equivalences between elements of the two frameworks. By complementing what is missing from one framework comparing to the other, we introduce a…
Generative model learns to mimic time-series behavior to generate accurate trajectories.
Unified framework for imitation learning via moment matching.
Imitation learning trains a policy from expert demonstrations. Imitation learning approaches have been designed from various principles, such as behavioral cloning via supervised learning, apprenticeship learning via inverse reinforcement learning, and GAIL via generative adversarial learning. In this paper, we propose…
Sequence prediction models can be learned from example sequences with a variety of training algorithms. Maximum likelihood learning is simple and efficient, yet can suffer from compounding error at test time. Reinforcement learning such as policy gradient addresses the issue but can have prohibitively poor exploration …
Generalized linear models (GLMs) -- such as logistic regression, Poisson regression, and robust regression -- provide interpretable models for diverse data types. Probabilistic approaches, particularly Bayesian ones, allow coherent estimates of uncertainty, incorporation of prior information, and sharing of power acros…
We present a deep recurrent neural network architecture to solve a class of stochastic optimal control problems described by fully nonlinear Hamilton Jacobi Bellmanpartial differential equations. Such PDEs arise when one considers stochastic dynamics characterized by uncertainties that are additive and control multipli…
End-to-end framework learns LLM routing from observational data.
Model-based Reinforcement Learning approaches have the promise of being sample efficient. Much of the progress in learning dynamics models in RL has been made by learning models via supervised learning. But traditional model-based approaches lead to `compounding errors' when the model is unrolled step by step. Essentia…
Auto-CEI improves LLM reasoning by balancing assertiveness and conservativeness.
Agents compose pre-trained policies for complex tasks, improving zero-shot performance.
Interactive IL beats BC by state-wise annotation cost.
Improves data efficiency in multi-agent control tasks using model-based reinforcement learning.
PBC improves AI and dynamical subseasonal forecasts by reducing biases.
EBIL simplifies IL by estimating expert energy as reward, achieving effective performance.