On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “lookahead Q-function”

Fast Non-Episodic Finite-Horizon RL with K-Step Lookahead Thresholdingcs.LG

New RL method learns K-step lookahead Q-functions for fixed-horizon MDPs.

problem Challenges in online reinforcement learning for non-episodic, finite-horizon MDPs.

method Introduces a K-step lookahead Q-function with a time-varying threshold for selecting actions.

result Achieves minimax optimal constant regret for K=1 and

\mathcal{O}(\max((K-1),C_{K-1})\sqrt{SAT\log(T)})

regret for K ≥ 2.

reinforcement learning finite horizon lookahead Q-function thresholding tabular learning

2026-01-31abs ↗pdf ↗

Reinforcement Learning with Lookahead Informationcs.LG

New RL algorithms use lookahead info to maximize rewards.

problem RL in unknown environments without lookahead info.

method Planning using empirical reward and transition distributions.

result Achieves tight regret compared to lookahead info.

Reinforcement Learning Lookahead Information Regret Minimization Empirical Distributions Planning Algorithms

2024-06-04abs ↗pdf ↗

Detecting Lookahead Bias in LLM Forecastsq-fin.GN

We detect lookahead bias in LLM forecasts using a novel statistical method.

problem Detecting lookahead bias in LLM-generated economic forecasts.

method Developed a statistical procedure using date-only recall queries and estimated Lookahead Propensity (LAP).

result LLM forecasts are contaminated with lookahead bias, as indicated by a positive interaction between LAP and the forecast in accuracy regressions.

LLM forecasts lookahead bias statistical procedure date-only recall Lookahead Propensity (LAP)

2025-12-29abs ↗pdf ↗

Reinforcement Learning with Multi-Step Lookahead Information Via Adaptive Batchingcs.LG

New algorithm learns optimal policy with multi-step lookahead information.

problem Learning optimal policy in reinforcement learning with multi-step lookahead information is NP-hard.

method Adaptive batching policies that process lookahead in state-dependent chunks.

result Order-optimal regret bounds up to a constant factor of lookahead horizon.

Reinforcement Learning Multi-step Lookahead Adaptive Batching Optimal Policy Regret Minimization

2026-01-15abs ↗pdf ↗

Recursive Two-Step Lookahead Expected Payoff for Time-Dependent Bayesian Optimizationmath.OC

A new Bayesian method optimizes time-dependent expensive functions with lookahead.

problem Maximizing a time-dependent, expensive oracle with limited evaluations.

method Recursive, two-step lookahead expected payoff (r2LEY) acquisition function.

result r2LEY outperforms myopic methods in synthetic and real-world datasets.

Bayesian optimization time-dependent lookahead expected payoff expensive oracle

2020-06-14abs ↗pdf ↗

Generalization and Optimization of SGD with Lookaheadcs.LG

The Lookahead optimizer improves SGD's performance and generalization without restrictive assumptions.

problem Improving the generalization of SGD with Lookahead.

method A rigorous stability and generalization analysis of the Lookahead optimizer with minibatch SGD, leveraging on-average model stability.

result Derives generalization bounds for convex and strongly convex problems without the restrictive Lipschitzness assumption, demonstrating a linear speedup with batch size.

SGD Lookahead optimizer minibatch generalization stability

2025-09-19abs ↗pdf ↗

Lookahead-Bounded Q-Learningcs.LG

LBQL improves Q-learning by using lookahead bounds for better performance.

problem Improving Q-learning in stochastic environments.

method LBQL uses lookahead bounds to construct dual penalties and track upper and lower bounds via stochastic approximation.

result LBQL converges faster and is more robust to hyperparameters than standard Q-learning.

Q-learning lookahead bounds stochastic environments convergence robustness

2020-06-28abs ↗pdf ↗

Instruction Tuning Chronologically Consistent Language Modelscs.LG

New models avoid lookahead bias by training on past data only.

problem Lookahead bias in language models.

method Chronologically consistent training on data before a knowledge-cutoff date.

result Elimination of lookahead bias in predictions.

language models lookahead bias chronological training predictability

2025-10-13abs ↗pdf ↗

Uncovering Feature Interdependencies in High-Noise Environments with Stepwise Lookahead Decision Forestscs.LG

A new random forest algorithm uncovers feature interdependencies better than traditional methods.

problem Tackles the sub-optimality of greedy decision tree implementations in random forests.

method Presented a 'stepwise lookahead' variation of random forests that considers multiple split nodes simultaneously.

result Significantly outperforms greedy random forests in uncovering feature interdependencies, especially in high-noise environments.

feature interdependencies random forests stepwise lookahead greedy algorithms high-noise environments

2020-09-30abs ↗pdf ↗

Lookahead Optimizer: k steps forward, 1 step backcs.LG

The vast majority of successful deep neural networks are trained using variants of stochastic gradient descent (SGD) algorithms. Recent attempts to improve SGD can be broadly categorized into two approaches: (1) adaptive learning rate schemes, such as AdaGrad and Adam, and (2) accelerated schemes, such as heavy-ball an…

2019-07-19abs ↗pdf ↗

Neurosymbolic Meta-Reinforcement Lookahead Learning Achieves Safe Self-Driving in Non-Stationary Environmentscs.RO

Study improves self-driving safety in dynamic environments.

problem Safe self-driving in non-stationary urban settings.

method Neurosymbolic Meta-Reinforcement Lookahead Learning (NUMERLA).

result Self-driving agents can adapt safely in real-time.

self-driving safety meta-reinforcement lookahead non-stationary

2023-09-05abs ↗pdf ↗

Taming GANs with Lookahead-Minmaxstat.ML

A new algorithm improves GAN training stability and performance.

problem Training GANs is difficult due to variance and rotational dynamics.

method Proposes Lookahead-Minmax algorithm for minmax optimization.

result Significant improvement in GAN performance and stability.

GANs minmax optimization Lookahead stability performance

2020-06-25abs ↗pdf ↗

NoLBERT: A No Lookahead(back) Foundational Language Modelecon.GN

NoLBERT avoids lookback and lookahead biases for better econometric inference.

problem Information leakage in language models affects econometric inference.

method Pretrained on text from 1976-1995, avoiding lookback and lookahead biases.

result NoLBERT outperforms domain-specific baselines and predicts higher profit growth.

language models econometrics timestamped innovation networks profit growth

2025-09-01abs ↗pdf ↗

Next-Depth Lookahead Treestat.ML

Improves tree model performance by considering future node splits.

problem Improving tree model performance.

method Next-Depth Lookahead Tree (NDLT) model that evaluates future node splits.

result Enhanced tree model performance.

tree model performance improvement future node splits

2025-09-18abs ↗pdf ↗

Lookahead: A Far-Sighted Alternative of Magnitude-based Pruningcs.LG

Magnitude-based pruning is one of the simplest methods for pruning neural networks. Despite its simplicity, magnitude-based pruning and its variants demonstrated remarkable performances for pruning modern architectures. Based on the observation that magnitude-based pruning indeed minimizes the Frobenius distortion of a…

2020-02-12abs ↗pdf ↗

DatedGPT: Preventing Lookahead Bias in Large Language Models with Time-Aware Pretrainingcs.CL

DatedGPT prevents lookahead bias in financial forecasting models.

problem Lookahead bias in large language models trained on internet-scale data.

method Time-aware pretraining with annual data cutoffs and instruction fine-tuning.

result Models' knowledge is effectively bounded by their data cutoff year, improving forecasting validity.

lookahead bias large language models financial forecasting time-aware pretraining instruction fine-tuning

2026-03-12abs ↗pdf ↗

Efficient Nonmyopic Bayesian Optimization via One-Shot Multi-Step Treescs.LG

Efficiently optimizes expensive functions with multi-step lookahead using one-shot optimization.

problem Optimizing expensive functions with long-term impacts using myopic approaches.

method Formulated as nested optimization problems within a multi-step scenario tree, optimized in one-shot fashion.

result Multi-step expected improvement is computationally tractable and outperforms existing methods.

Bayesian optimization multi-step lookahead one-shot optimization fantasization Gaussian process

2020-06-29abs ↗pdf ↗

Policy Mirror Descent with Lookaheadcs.LG

Enhances PMD with lookahead to improve RL performance.

problem Improving RL performance with greedy policies over 1-step.

method Integrates multi-step greedy policies into PMD with lookahead.

result Shows faster convergence rate for

h

-PMD.

Policy Mirror Descent Lookahead RL Convergence Rate Multi-step Greedy

2024-03-21abs ↗pdf ↗

Online Planning with Lookahead Policiescs.LG

Real Time Dynamic Programming (RTDP) is an online algorithm based on Dynamic Programming (DP) that acts by 1-step greedy planning. Unlike DP, RTDP does not require access to the entire state space, i.e., it explicitly handles the exploration. This fact makes RTDP particularly appealing when the state space is large and…

2019-09-10abs ↗pdf ↗

Fast Lower and Upper Estimates for the Price of Constrained Multiple Exercise American Options by Single Pass Lookahead Search and Nearest-Neighbor Martingaleq-fin.CP

This paper provides fast estimates for complex option types.

problem Estimating prices for constrained multiple exercise American options.

method Lookahead search for lower estimates and nearest-neighbor martingale for upper estimates.

result Probabilistic convergence guarantees for the algorithms.

American options swing options passport options lookahead search nearest-neighbor

2020-02-26abs ↗pdf ↗

Sample Efficient Reinforcement Learning via Low-Rank Matrix Estimationcs.LG

A new algorithm reduces sample complexity for learning Q-functions in reinforcement learning.

problem Efficiently learning Q-functions in reinforcement learning with continuous state and action spaces.

method Developed a simple, iterative learning algorithm that estimates low-rank Q-functions.

result Achieved exponential improvement in sample complexity for low-rank Q-functions.

reinforcement learning Q-function sample efficiency low-rank matrix estimation kernel methods

2020-06-11abs ↗pdf ↗

Understanding Lookahead Dynamics Through Laplace Transformmath.OC

Transforms game optimization dynamics into frequency domain for precise hyperparameter analysis.

problem Analyzing convergence of hyperparameters in game optimization.

method Frequency-domain framework using High-Resolution Differential Equations (HRDEs) and Laplace transforms.

result Derives precise convergence criteria for the Lookahead algorithm.

Hyperparameter optimization Game optimization Laplace transforms High-Resolution Differential Equations Convergence analysis

2025-06-16abs ↗pdf ↗

Look-Ahead-Bench: a Standardized Benchmark of Look-ahead Bias in Point-in-Time LLMs for Financecs.AI

Look-Ahead-Bench evaluates financial LLMs for lookahead bias, revealing significant differences in model performance.

problem Measuring and mitigating lookahead bias in financial LLMs.

method Standardized benchmark evaluating model behavior in practical financial scenarios, analyzing performance decay across market regimes.

result Standard LLMs exhibit significant lookahead bias, while Pitinf models show improved generalization and reasoning abilities.

lookahead bias financial LLMs point-in-time models model evaluation temporal performance

2026-01-20abs ↗pdf ↗

Lookahead Counterfactual Fairnesscs.LG

Introduces lookahead counterfactual fairness to account for downstream effects of ML predictions.

problem Downstream effects of ML predictions on individuals not considered by counterfactual fairness.

method Introduces lookahead counterfactual fairness (LCF), a new fairness notion that considers future status. Proposes an algorithm based on theoretical conditions.

result Proposes an algorithm to achieve lookahead counterfactual fairness and validates it on synthetic and real data.

counterfactual fairness downstream effects lookahead counterfactual fairness path-dependent fairness

2024-12-02abs ↗pdf ↗

Chronologically Consistent Large Language Modelsq-fin.GN

Chronologically consistent models maintain accuracy with time-restricted data.

problem Training data introduces lookahead bias and training leakage in large language models.

method ChronoBERT and ChronoGPT trained with only available data at each time point.

result Models achieve strong performance and competitive with larger models, mitigating lookahead bias.

chronological consistency lookahead bias training leakage large language models natural language processing

2025-02-28abs ↗pdf ↗

Mitigating Suboptimality of Deterministic Policy Gradients in Complex Q-functionscs.LG

SAVO actor improves reinforcement learning by avoiding local optima in complex Q-functions.

problem Gradient ascent in complex Q-functions leads to suboptimal solutions.

method SAVO actor generates multiple action proposals and truncates poor local optima.

result SAVO actor finds optimal actions more frequently and outperforms other architectures.

reinforcement learning actor-critic DDPG TD3 local optima

2024-10-15abs ↗pdf ↗

Minimum information divergence of Q-functions for dynamic treatment resumesstat.ME

The paper introduces a new method for estimating optimal policies in dynamic treatment regimes using information geometry.

problem Estimating optimal policies in dynamic treatment regimes.

method Minimum information divergence method based on

γ

-power divergence.

result The

γ

-power divergence method effectively seeks the optimal policy by vanishing the divergence between policy-equivalent Q-functions.

reinforcement learning dynamic treatment regimes information geometry Q-function policy equivalence

2022-11-16abs ↗pdf ↗

On Well-posedness and Minimax Optimal Rates of Nonparametric Q-function Estimation in Off-policy Evaluationmath.ST

Study on $Q$ -function estimation for continuous state-action MDPs, deriving rates and conditions.

problem Estimating

Q

-function in off-policy evaluation for continuous state-action Markov decision processes.

method Reformulated as nonparametric instrumental variables (NPIV) problem, derived minimax lower bounds, proposed sieve two-stage least squares estimator.

result First minimax lower bounds for

Q

-function and its derivatives in sup-norm and

L^2

-norm, same as classical nonparametric regression.

Q-function estimation off-policy evaluation nonparametric instrumental variables Markov decision processes minimax lower bounds

2022-01-17abs ↗pdf ↗

Why Non-myopic Bayesian Optimization is Promising and How Far Should We Look-ahead? A Study via Rolloutcs.LG

Lookahead, also known as non-myopic, Bayesian optimization (BO) aims to find optimal sampling policies through solving a dynamic program (DP) that maximizes a long-term reward over a rolling horizon. Though promising, lookahead BO faces the risk of error propagation through its increased dependence on a possibly mis-sp…

2019-11-04abs ↗pdf ↗

Continuous-time q-learning for mean-field control problemscs.LG

This paper develops q-learning methods for mean-field control problems.

problem Continuous-time mean-field control problems with interaction between agents.

method Introduces two q-functions and devises model-free learning algorithms.

result Developed algorithms can learn optimal value functions and q-functions.

q-learning mean-field control reinforcement learning continuous time martingale condition

2023-06-28abs ↗pdf ↗

Autoregressive Language Models are Secretly Energy-Based Models: Insights into the Lookahead Capabilities of Next-Token Predictioncs.LG

This paper shows ARMs and EBMs are equivalent, revealing ARM lookahead capabilities.

problem Understanding the lookahead capabilities of next-token prediction models.

method Unified view of ARMs and EBMs, establishing a bijection and deriving equivalence.

result ARMs and EBMs are equivalent, revealing ARM lookahead capabilities.

autoregressive models energy-based models next-token prediction lookahead capabilities maximum entropy reinforcement learning

2025-12-17abs ↗pdf ↗

Two-step Lookahead Bayesian Optimization with Inequality Constraintsstat.ML

Efficiently optimizes constrained problems with two-step lookahead BO.

problem Optimizing constrained problems with limited computational resources.

method Two-step lookahead Bayesian optimization with inequality constraints, using a novel unbiased gradient estimator.

result Significantly improves query efficiency over previous methods.

Bayesian optimization constraint handling lookahead optimization unbiased gradient estimation

2021-12-06abs ↗pdf ↗

Offline Minimax Soft-Q-learning Under Realizability and Partial Coveragecs.LG

New algorithms estimate Q-functions under partial coverage and realizability, improving offline RL guarantees.

problem Offline RL with limited exploration and assumptions about data coverage and Q-function realizability.

method Proposes minimax learning algorithms to estimate soft or vanilla Q-functions with

L^2

-convergence guarantees.

result PAC guarantees for offline RL under partial coverage and realizability conditions.

offline reinforcement learning minimax optimization soft Q-function partial coverage realizability

2023-02-05abs ↗pdf ↗

The Value of Reward Lookahead in Reinforcement Learningcs.LG

This work analyzes the value of future reward information in RL.

problem Analyzing the impact of knowing future rewards in reinforcement learning.

method Competitive analysis and worst-case reward distribution.

result Exact ratios between standard RL agents and those with future-reward lookahead.

reinforcement learning competitive analysis future rewards reward lookahead offline RL

2024-03-18abs ↗pdf ↗

Target Transfer Q-Learning and Its Convergence Analysiscs.LG

Q-learning is one of the most popular methods in Reinforcement Learning (RL). Transfer Learning aims to utilize the learned knowledge from source tasks to help new tasks to improve the sample complexity of the new tasks. Considering that data collection in RL is both more time and cost consuming and Q-learning converge…

2018-09-21abs ↗pdf ↗

Semiparametric Double Reinforcement Learning with Applications to Long-Term Causal Inferencestat.ML

Improves DRL for long-term causal inference with semiparametric methods.

problem Efficient inference for policy values in nonparametric MDPs with stringent conditions.

method Semiparametric Double Reinforcement Learning (DRL) with superefficient nonparametric estimators.

result Relaxes overlap conditions and reduces high-dimensional density-ratio estimation.

Double Reinforcement Learning Long-term Causal Inference Semiparametric Methods Q-function Occupancy Density Ratios

2025-01-12abs ↗pdf ↗

Toward Interpretable Deep Reinforcement Learning with Linear Model U-Treescs.LG

Deep Reinforcement Learning (DRL) has achieved impressive success in many applications. A key component of many DRL models is a neural network representing a Q function, to estimate the expected cumulative reward following a state-action pair. The Q function neural network contains a lot of implicit knowledge about the…

2018-07-16abs ↗pdf ↗

q-Learning in Continuous Timecs.LG

Continuous-time Q-learning theory developed for reinforcement learning.

problem Continuous-time reinforcement learning challenges.

method Entropy-regularized, exploratory diffusion process formulation; first-order approximation of Q-function; martingale conditions.

result Developed a q-learning theory independent of time discretization.

reinforcement learning continuous time Q-learning entropy regularization martingales

2022-07-02abs ↗pdf ↗

Reinforcement Learning Architectures: SAC, TAC, and ESACcs.LG

ESAC improves reinforcement learning by lookahead and intuition.

problem Designing efficient reinforcement learning architectures for intelligent agents.

method Integrates selector, tuner, model-learner, estimator, and lookahead into ESAC.

result ESAC outperforms other architectures in optimizing policies.

reinforcement learning actor-critic lookahead intuition ESAC

2020-04-05abs ↗pdf ↗

Harnessing Structures for Value-Based Planning and Reinforcement Learningcs.LG

Value-based methods constitute a fundamental methodology in planning and deep reinforcement learning (RL). In this paper, we propose to exploit the underlying structures of the state-action value function, i.e., Q function, for both planning and deep RL. In particular, if the underlying system dynamics lead to some glo…

2019-09-26abs ↗pdf ↗

On Instrumental Variable Regression for Deep Offline Policy Evaluationcs.LG

This paper explores how IV methods can improve Q-function estimates in offline policy evaluation.

problem Confounding in estimating Q-function using reinforcement learning.

method Integrates IV techniques into offline policy evaluation (OPE) to improve Q-function estimates.

result State-of-the-art OPE methods are closely matched in performance by some IV methods.

Instrumental Variables Offline Policy Evaluation Q-function Estimation Reinforcement Learning Confounding

2021-05-21abs ↗pdf ↗

Hidden Markov Model Estimation-Based Q-learning for Partially Observable Markov Decision Processcs.LG

The objective is to study an on-line Hidden Markov model (HMM) estimation-based Q-learning algorithm for partially observable Markov decision process (POMDP) on finite state and action sets. When the full state observation is available, Q-learning finds the optimal action-value function given the current action (Q func…

2018-09-17abs ↗pdf ↗

Logistic Q-Learningcs.LG

Proposes a new reinforcement learning algorithm using Q-function.

problem Optimal control in Markov Decision Processes (MDPs).

method Regularized linear-programming formulation, Q-function, saddle-point optimization.

result Demonstrates effectiveness on various benchmark problems.

reinforcement learning optimal control MDPs Q-function saddle-point optimization

2020-10-21abs ↗pdf ↗

Nonmyopic Global Optimisation via Approximate Dynamic Programmingcs.LG

New methods improve global optimisation for expensive functions using lookahead strategies.

problem Optimising expensive functions without gradient info in high dimensions.

method Nonmyopic acquisition strategies based on approximate dynamic programming.

result Nonmyopic methods outperform myopic approaches in various applications.

global optimisation approximate dynamic programming nonmyopic acquisition expensive functions sequential decision making

2024-12-06abs ↗pdf ↗

Robust Temporal Difference Learning for Critical Domainscs.LG

We present a new Q-function operator for temporal difference (TD) learning methods that explicitly encodes robustness against significant rare events (SRE) in critical domains. The operator, which we call the $κ$ -operator, allows to learn a robust policy in a model-based fashion without actually observing the SRE. We i…

2019-01-23abs ↗pdf ↗

Convergence of Q-value in case of Gaussian rewardsmath.OC

The paper proves the convergence of Q-value for Gaussian rewards.

problem Existing proofs cannot guarantee convergence of the Q-function for Gaussian rewards.

method Using the central limit theorem and relaxing the condition to

E[r(s,a)^2]<\infty

result Proves the convergence of the Q-function under the condition of

E[r(s,a)^2]<\infty

reinforcement learning Q-value Gaussian rewards central limit theorem policy gradient theorem

2020-03-07abs ↗pdf ↗

The Effective Horizon Explains Deep RL Performance in Stochastic Environmentsstat.ML

New RL algorithm explains why deep learning works in stochastic environments.

problem Why deep RL algorithms perform well in practice despite using random exploration.

method Introducing SQIRL, an iterative RL algorithm that separates exploration and learning.

result Effective horizon explains why deep RL works in stochastic environments.

Reinforcement Learning Deep Learning Stochastic Environments Random Exploration Fitted-Q Iteration

2023-12-13abs ↗pdf ↗

On the Expressivity of Neural Networks for Deep Reinforcement Learningcs.LG

We compare the model-free reinforcement learning with the model-based approaches through the lens of the expressive power of neural networks for policies, $Q$ -functions, and dynamics. We show, theoretically and empirically, that even for one-dimensional continuous state space, there are many MDPs whose optimal $Q$ -func…

2019-10-14abs ↗pdf ↗