arXiv research

New RL method learns K-step lookahead Q-functions for fixed-horizon MDPs.

New RL algorithms use lookahead info to maximize rewards.

We detect lookahead bias in LLM forecasts using a novel statistical method.

New algorithm learns optimal policy with multi-step lookahead information.

Lookahead pruning extends single-layer optimization to multi-layer, outperforming magnitude-based pruning.

A new Bayesian method optimizes time-dependent expensive functions with lookahead.

The Lookahead optimizer improves SGD's performance and generalization without restrictive assumptions.

LBQL improves Q-learning by using lookahead bounds for better performance.

New models avoid lookahead bias by training on past data only.

A new random forest algorithm uncovers feature interdependencies better than traditional methods.

The vast majority of successful deep neural networks are trained using variants of stochastic gradient descent (SGD) algorithms. Recent attempts to improve SGD can be broadly categorized into two approaches: (1) adaptive learning rate schemes, such as AdaGrad and Adam, and (2) accelerated schemes, such as heavy-ball an…

Study improves self-driving safety in dynamic environments.

A new algorithm improves GAN training stability and performance.

NoLBERT avoids lookback and lookahead biases for better econometric inference.

Improves tree model performance by considering future node splits.

DatedGPT prevents lookahead bias in financial forecasting models.

Efficiently optimizes expensive functions with multi-step lookahead using one-shot optimization.

Enhances PMD with lookahead to improve RL performance.

Real Time Dynamic Programming (RTDP) is an online algorithm based on Dynamic Programming (DP) that acts by 1-step greedy planning. Unlike DP, RTDP does not require access to the entire state space, i.e., it explicitly handles the exploration. This fact makes RTDP particularly appealing when the state space is large and…

This paper provides fast estimates for complex option types.

A new algorithm reduces sample complexity for learning Q-functions in reinforcement learning.

Transforms game optimization dynamics into frequency domain for precise hyperparameter analysis.

Look-Ahead-Bench evaluates financial LLMs for lookahead bias, revealing significant differences in model performance.

Introduces lookahead counterfactual fairness to account for downstream effects of ML predictions.

Chronologically consistent models maintain accuracy with time-restricted data.

SAVO actor improves reinforcement learning by avoiding local optima in complex Q-functions.

The paper introduces a new method for estimating optimal policies in dynamic treatment regimes using information geometry.

Study on $Q$-function estimation for continuous state-action MDPs, deriving rates and conditions.

Lookahead, also known as non-myopic, Bayesian optimization (BO) aims to find optimal sampling policies through solving a dynamic program (DP) that maximizes a long-term reward over a rolling horizon. Though promising, lookahead BO faces the risk of error propagation through its increased dependence on a possibly mis-sp…

This paper develops q-learning methods for mean-field control problems.

This paper shows ARMs and EBMs are equivalent, revealing ARM lookahead capabilities.

Efficiently optimizes constrained problems with two-step lookahead BO.

New algorithms estimate Q-functions under partial coverage and realizability, improving offline RL guarantees.

This work analyzes the value of future reward information in RL.

Q-learning is one of the most popular methods in Reinforcement Learning (RL). Transfer Learning aims to utilize the learned knowledge from source tasks to help new tasks to improve the sample complexity of the new tasks. Considering that data collection in RL is both more time and cost consuming and Q-learning converge…

Improves DRL for long-term causal inference with semiparametric methods.

Deep Reinforcement Learning (DRL) has achieved impressive success in many applications. A key component of many DRL models is a neural network representing a Q function, to estimate the expected cumulative reward following a state-action pair. The Q function neural network contains a lot of implicit knowledge about the…

Continuous-time Q-learning theory developed for reinforcement learning.

ESAC improves reinforcement learning by lookahead and intuition.

Value-based methods constitute a fundamental methodology in planning and deep reinforcement learning (RL). In this paper, we propose to exploit the underlying structures of the state-action value function, i.e., Q function, for both planning and deep RL. In particular, if the underlying system dynamics lead to some glo…

This paper explores how IV methods can improve Q-function estimates in offline policy evaluation.

The objective is to study an on-line Hidden Markov model (HMM) estimation-based Q-learning algorithm for partially observable Markov decision process (POMDP) on finite state and action sets. When the full state observation is available, Q-learning finds the optimal action-value function given the current action (Q func…

Proposes a new reinforcement learning algorithm using Q-function.

New methods improve global optimisation for expensive functions using lookahead strategies.

We present a new Q-function operator for temporal difference (TD) learning methods that explicitly encodes robustness against significant rare events (SRE) in critical domains. The operator, which we call the $κ$-operator, allows to learn a robust policy in a model-based fashion without actually observing the SRE. We i…

The paper proves the convergence of Q-value for Gaussian rewards.

New RL algorithm explains why deep learning works in stochastic environments.

We compare the model-free reinforcement learning with the model-based approaches through the lens of the expressive power of neural networks for policies, $Q$-functions, and dynamics. We show, theoretically and empirically, that even for one-dimensional continuous state space, there are many MDPs whose optimal $Q$-func…