We present relay policy learning, a method for imitation and reinforcement learning that can solve multi-stage, long-horizon robotic tasks. This general and universally-applicable, two-phase approach consists of an imitation learning stage that produces goal-conditioned hierarchical policies, and a reinforcement learni…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “long-horizon simulation”
Compositional diffusion models simulate coupled PDEs efficiently.
problem Efficiently simulating long-horizon coupled PDE systems.
method Diffusion models trained on decoupled data are composed at inference time.
result Compositional diffusion models recover coupled trajectories with low error.
Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generationcs.LG
Video prediction models combined with planning algorithms have shown promise in enabling robots to learn to perform many vision-based tasks through only self-supervision, reaching novel goals in cluttered scenes with unseen objects. However, due to the compounding uncertainty in long horizon video prediction and poor s…
Action-bisimulation learns long-horizon controllability for reinforcement learning.
problem Learning relevant state features in high-dimensional observations for robust reinforcement learning.
method Action-bisimulation encoding, inspired by bisimulation invariance, extends single-step controllability to multi-step.
result Action-bisimulation pretraining improves sample efficiency in various environments.
State-of-the-art forecasting methods using Recurrent Neural Net- works (RNN) based on Long-Short Term Memory (LSTM) cells have shown exceptional performance targeting short-horizon forecasts, e.g given a set of predictor features, forecast a target value for the next few time steps in the future. However, in many appli…
Max entropy exploration guides reinforcement learning agents to pursue achievable goals.
problem Achieving distant test-time goals in long-horizon tasks.
method Optimize entropy of historical achieved goals by focusing on sparsely explored areas.
result Order of magnitude better sample efficiency on long-horizon multi-goal tasks.
FDS tackles long horizon hyperparameter optimization issues.
problem Memory scaling and gradient degradation in long horizon tasks.
method Forward-mode differentiation with sharing (FDS).
result Significantly outperforms greedy gradient-based alternatives.
Exploring with Sticky Mittens: Reinforcement Learning with Expert Interventions via Option Templatescs.LG
Framework uses expert intervention to solve long-horizon reinforcement learning tasks.
problem Long horizon robot learning tasks with sparse rewards.
method Option templates and expert intervention to enable high-level task understanding.
result Framework outperforms state-of-the-art approaches by two orders of magnitude.
TRM improves long-horizon LLM RL by masking divergent sequences.
problem Long-horizon reinforcement learning with LLMs suffers from off-policy mismatch and approximation errors.
method Derives and applies trust region bounds to control divergence, proposing Trust Region Masking.
result First non-vacuous monotonic improvement guarantees for long-horizon LLM-RL.
TRM improves long-horizon reinforcement learning for LLMs by masking divergent sequences.
problem Long-horizon reinforcement learning for LLMs suffers from off-policy mismatch and approximation errors.
method Derives and applies trust region bounds to control divergence, proposing Trust Region Masking.
result First non-vacuous monotonic improvement guarantees for long-horizon LLM-RL.
Predicts long-term return distributions with time-varying volatility.
problem Risk management in long-horizon returns.
method Predicts future return distributions without specifying volatility dynamics or shock distribution.
result Derives risk measures like VaR and CTE from the predicted return distribution.
Improved neural PDEs trained on augmented data enhance model accuracy and efficiency.
problem Training neural PDEs on limited data to accurately represent complex systems.
method Space-filling sampling of local states to generate augmented training data.
result Data-augmented neural PDEs outperform traditional emulators in accuracy and stability.
The paper improves model-based reinforcement learning by using multi-timestep objectives.
problem Compounding errors in one-step dynamics models as trajectory length increases.
method Developed a multi-timestep objective as a weighted sum of losses at various future horizons.
result Exponentially decaying weights significantly improve long-horizon performance.
Accelerates TD learning for long-horizon reinforcement learning problems.
problem Slow convergence of conventional TD learning in long-horizon tasks.
method Introduces PID Accelerated Temporal Difference (PID TD) learning algorithms.
result Accelerates convergence of TD learning compared to conventional methods.
This work improves RL for complex robotic tasks by guiding exploration with task-specific goal distributions.
problem Solving long-horizon, complex sequential tasks in robotics with sparse rewards.
method Extends hindsight relabelling to task-specific goal distributions using a small set of demonstrations.
result Significantly higher overall performance on complex robotic manipulation tasks.
New method produces coherent forecasts for long-range data.
problem Inaccurate and non-coherent forecasts on long-horizon data.
method Probabilistic forecasting with KL-divergence for coherent aggregates.
result Improves forecast performance across base levels and aggregates.
HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulatorscs.LG
Most common navigation tasks in human environments require auxiliary arm interactions, e.g. opening doors, pressing buttons and pushing obstacles away. This type of navigation tasks, which we call Interactive Navigation, requires the use of mobile manipulators: mobile bases with manipulation capabilities. Interactive N…
This review tackles long horizon forecasting in time series analysis using deep learning.
problem Long horizon forecasting in time series analysis.
method Incorporates deep learning techniques such as trend, seasonality, Fourier and wavelet transforms, and various model architectures.
result LHF is an error propagation problem, with models like xLSTM and Triformer showing better performance.
The study reveals distinct patterns in retail investors' holding periods affecting stock returns.
problem Understanding the impact of retail investors' investment horizons on stock returns.
method Using self-reported holding periods from StockTwits, the study categorizes retail investors into long-horizon and short-horizon groups and analyzes their return patterns.
result Long-horizon retail investors exhibit underreaction to earnings announcements, while short-horizon investors show overreaction.
SGM combines deep learning and planning for robust long-horizon tasks.
problem Combining deep learning and planning for robust long-horizon tasks.
method Sparse Graphical Memory (SGM) that stores states and feasible transitions in a sparse memory, aggregating states according to a two-way consistency objective.
result SGM significantly outperforms current state of the art methods on long horizon, sparse-reward visual navigation tasks.
Is Long Horizon Reinforcement Learning More Difficult Than Short Horizon Reinforcement Learning?cs.LG
Long horizon reinforcement learning is as hard as short horizon learning.
problem Understanding the difficulty of long horizon reinforcement learning problems.
method Introduced new concepts: ε-net for optimal policies and Online Trajectory Synthesis algorithm.
result Proved that sample complexity scales logarithmically with the planning horizon, refuting the conjecture.
LLapDiff models irregular multivariate time series without step-by-step integration.
problem Trade-off between discrete and continuous methods for long-horizon forecasting.
method Generative framework that models target as a low-dimensional latent trajectory, guided by modal parameterization and Laplace domain poles.
result Improves long-horizon forecasting over baselines and supports missing-value imputation.
The paper clarifies long-horizon investment and DCA, showing no risk reduction but different exposure profiles.
problem Misleading claims about reducing risk with longer investment horizons and DCA.
method Unified probabilistic framework, defining risk and uncertainty, and introducing effective investment exposure.
result Different investment timing strategies can lead to distinct exposure profiles over time, affecting risk and uncertainty.
Temporal Coarse-Graining of Latent Default-Probability Paths Generates Effective Default Correlationq-fin.ST
Temporal aggregation reveals latent default correlation from monthly data.
problem Understanding effective default correlation from monthly default data.
method Temporal coarse-graining of latent default-probability paths.
result Temporal coarse-graining improves identifiability and reduces over-allocation of long-horizon fluctuations.
Temporal Coarse-Graining of Latent Default-Probability Paths Generates Effective Default Correlationq-fin.ST
Temporal coarse-graining of latent default paths explains effective correlation in corporate defaults.
problem Understanding effective default correlation in corporate defaults.
method Temporal coarse-graining of latent default-probability paths, applied to corporate default-count data.
result Temporal coarse-graining provides a scale-consistent baseline that improves identifiability and reduces over-allocation of long-horizon fluctuations.
The objective of this work is to augment the basic abilities of a robot by learning to use new sensorimotor primitives to enable the solution of complex long-horizon problems. Solving long-horizon problems in complex domains requires flexible generative planning that can combine primitive abilities in novel combination…
Many robotic applications require the agent to perform long-horizon tasks in partially observable environments. In such applications, decision making at any step can depend on observations received far in the past. Hence, being able to properly memorize and utilize the long-term history is crucial. In this work, we pro…
For an investor with constant absolute risk aversion and a long horizon, who trades in a market with constant investment opportunities and small proportional transaction costs, we obtain explicitly the optimal investment policy, its implied welfare, liquidity premium, and trading volume. We identify these quantities as…
Behavior cloning training instabilities amplified by SGD noise over long horizons.
problem Training instabilities in behavior cloning with deep neural networks.
method Empirical dissection of minibatch SGD updates and their effects on long-horizon rewards.
result Exponential moving average (EMA) of iterates effectively mitigates gradient variance amplification (GVA).
A new model decomposes equity returns and volatilities into memory components.
problem Understanding long-term equity dynamics and volatility patterns.
method Proposes a multivariate generalization of the variance ratio to decompose long-horizon equity dynamics.
result Identifies a five-factor model capturing persistent, antipersistent, and multi-scale memory in returns and volatility.
Agents compose pre-trained policies for complex tasks, improving zero-shot performance.
problem Challenges in long-horizon predictions and estimating visitation distributions induced by policy sequences.
method Learn predictive jumpy world models of multi-step dynamics, enhancing predictions with a consistency objective.
result Compositional planning with jumpy world models yields, on average, a 200% relative improvement over primitive actions on long-horizon tasks.
Plan2Vec learns image representations without labels, improving control tasks.
problem Learning image representations without labeled data.
method Constructs a weighted graph using near-neighbor distances and extrapolates to global embedding.
result Plan2Vec achieves accurate long-term value estimates in control tasks with reduced computational and memory costs.
We consider a market consisting of one safe and one risky asset, which offer constant investment opportunities. Taking into account both proportional transaction costs and linear price impact, we derive optimal rebalancing policies for representative investors with constant relative risk aversion and a long horizon.
This work tackles long-term visual planning by goal-conditioned hierarchical predictors.
problem Current learning approaches fail on long-horizon tasks due to lack of goal information and coarse-to-fine planning.
method Formulate goal-conditioned predictors (GCPs) and hierarchical models to predict trajectories between observations.
result GCPs enable effective long-term planning with much longer horizons than before.
New algorithms for efficient learning with long-term rewards in contextual bandits.
problem Efficient learning with long-term rewards in contextual bandits.
method Proposes new algorithms leveraging sparsity to discover dependence patterns and arm parameters.
result Regret upper bounds for data-poor and data-rich regimes, showing improved sample complexity.
Single model learns physics from diverse data.
problem Lack of universal physics models for diverse applications.
method General Physics Transformer (GPhyT) trained on diverse physics data.
result Single model achieves superior performance across multiple physics domains.
Soft geometric bias improves physical dynamics predictions.
problem Learning physical dynamics with exact group equivariance can degrade performance.
method Object-centric world models using geometric algebra neural networks.
result Soft geometric inductive bias leads to better physical fidelity predictions.
Learning to imitate expert behavior from demonstrations can be challenging, especially in environments with high-dimensional, continuous observations and unknown dynamics. Supervised learning methods based on behavioral cloning (BC) suffer from distribution shift: because the agent greedily imitates demonstrated action…
Financial event studies often misestimate causal effects due to misspecified factor models.
problem Misspecification of factor models in financial event studies leads to inconsistent estimates of causal effects.
method Proposed synthetic control methods to construct replicating portfolios from control securities.
result Synthetic control methods provide more accurate estimates of causal effects in event studies.
This survey reviews portfolio choice in settings where investment opportunities are stochastic due to, e.g., stochastic volatility or return predictability. It is explained how to heuristically compute candidate optimal portfolios using tools from stochastic control, and how to rigorously verify their optimality by mea…
Lo-Hp decouples weight generation into local and global policies to improve flexibility and efficiency.
problem Over-coupling and long-horizon issues in current optimization methods.
method Hybrid-Policy Sub-Trajectory Balance objective.
result Learning local optimization policies addresses long-horizon issues and enhances global weight generation.
Intention-aware Long Horizon Trajectory Prediction of Surrounding Vehicles using Dual LSTM Networkscs.LG
As autonomous vehicles (AVs) need to interact with other road users, it is of importance to comprehensively understand the dynamic traffic environment, especially the future possible trajectories of surrounding vehicles. This paper presents an algorithm for long-horizon trajectory prediction of surrounding vehicles usi…
A new reinforcement learning method uses model derivatives to improve policy optimization.
problem Improving sample efficiency and performance in model-based reinforcement learning.
method Constructs an actor-critic algorithm that uses the pathwise derivative of the learned model and policy.
result Consistently more sample efficient and matches model-free algorithms' asymptotic performance.
New method estimates off-policy data without needing known behavior policy.
problem Long-horizon reinforcement learning with limited on-policy data.
method Formulates problem as fixed point of operator, uses RKHS for estimation.
result Asymptotic consistency and finite-sample generalization proven.
Graph Neural Simulators improve data efficiency for PDE surrogates.
problem Lack of data efficiency in neural operators for PDE systems.
method Graph Neural Simulators (GNS) leverage message-passing and numerical time-stepping to learn PDE dynamics efficiently.
result GNS achieves less than 1% relative L2 error using only 3% of available trajectories.
Q-chunking improves RL for long tasks by chunking actions.
problem Improving sample efficiency and exploration in offline-to-online RL.
method Action chunking in TD-based RL methods.
result Q-chunking outperforms prior methods on long-horizon tasks.
In this work, we take a representation learning perspective on hierarchical reinforcement learning, where the problem of learning lower layers in a hierarchy is transformed into the problem of learning trajectory-level generative models. We show that we can learn continuous latent representations of trajectories, which…
Study forecasts sub-city real estate prices weekly using radar and news sentiment.
problem Limited availability of reliable real estate price indicators at neighborhood and long horizons.
method Combining satellite radar signals and news sentiment to forecast sub-city real estate prices.
result The multimodal model reduces mean absolute error by 35% at long horizons (26-34 weeks).