On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “Reward Processing”

Inferring Probabilistic Reward Machines from Non-Markovian Reward Processes for Reinforcement Learningcs.LG

Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.

problem Lack of structured representation for non-Markovian stochastic rewards in reinforcement learning.

method Introduces probabilistic reward machines (PRMs) and presents an algorithm to learn them from decision processes.

result Algorithm proves correct and convergent for learning PRMs from decision processes.

reinforcement learning probabilistic reward machines non-Markovian rewards stochastic rewards reinforcement algorithms

2021-07-09abs ↗pdf ↗

Reinforcement Learning with Exogenous States and Rewardscs.LG

The paper tackles reinforcement learning with exogenous variables and rewards.

problem Exogenous state variables and rewards slow reinforcement learning by introducing uncontrolled variation.

method Formalizes exogenous state variables and rewards, decomposes MDP into exogenous and endogenous components, and introduces algorithms to discover these components.

result Optimal policies for the endogenous MDP are also optimal for the original MDP, but the endogenous MDP is easier to solve due to reduced variance.

Reinforcement Learning Exogenous Variables Endogenous Rewards Markov Reward Process Markov Decision Process

2023-03-22abs ↗pdf ↗

Batch Policy Learning in Average Reward Markov Decision Processesmath.ST

The paper tackles batch policy learning in Markov Decision Processes, focusing on average reward maximization.

problem Maximizing long-term average reward in Markov Decision Processes with batch learning.

method Doubly robust estimator for average reward, optimization algorithm for optimal policy, finite-sample regret guarantee.

result The proposed method achieves semiparametric efficiency and provides a finite-sample regret guarantee.

Markov Decision Processes Average Reward Batch Learning Doubly Robust Estimator Optimization Algorithm

2020-07-23abs ↗pdf ↗

Learning Process Rewards via Success Visitation Matching for Efficient RLcs.LG

Transforming sparse outcomes into dense process rewards for efficient reinforcement learning.

problem Training RL policies to maximize sparse outcomes.

method Incentivizing policy matching state-action visitations of successful episodes.

result Significantly faster RL finetuning performance.

reinforcement learning sparse outcomes dense process rewards credit assignment finetuning

2026-06-22abs ↗pdf ↗

Trade-R1: Bridging Verifiable Rewards to Stochastic Environments via Process-Level Reasoning Verificationcs.AI

Trade-R1 bridges verifiable rewards to stochastic financial markets via process-level reasoning verification.

problem Extending RL to financial markets where rewards are verifiable but noisy.

method A verification method that transforms reasoning over financial documents into a structured RAG task, using a triangular consistency metric.

result DSR achieves superior cross-market generalization while maintaining reasoning consistency.

Reinforcement Learning Financial Markets Stochastic Environments Verifiable Rewards Reasoning Verification

2026-01-07abs ↗pdf ↗

Risk-Sensitive Reinforcement Learning: a Martingale Approach to Reward Uncertaintycs.LG

Novel framework for risk-sensitive reinforcement learning using martingale decomposition.

problem Risk sensitivity in sequential decision-making with uncertain rewards.

method Martingale decomposition and chaotic variation for reward uncertainty, integrated into model-free reinforcement learning algorithms.

result Demonstrated relevance of risk-sensitive reinforcement learning in grid world and portfolio optimization problems.

reinforcement learning risk sensitivity martingale uncertain rewards chaotic variation

2020-06-23abs ↗pdf ↗

Active Inverse Reward Designcs.LG

Designers of AI agents often iterate on the reward function in a trial-and-error process until they get the desired behavior, but this only guarantees good behavior in the training environment. We propose structuring this process as a series of queries asking the user to compare between different reward functions. Thus…

2018-09-09abs ↗pdf ↗

Generalized Linear Markov Decision Processstat.ML

Extends linear MDP to handle nonlinear rewards.

problem Restrictive linear MDP assumption limits real-world applicability.

method Proposes Generalized Linear MDP (GLMDP) with GLMs for rewards.

result Develops offline RL algorithms achieving suboptimality guarantees.

reinforcement learning Markov decision process generalized linear model nonlinear rewards sample efficiency

2025-06-01abs ↗pdf ↗

Discovering and Removing Exogenous State Variables and Rewards for Reinforcement Learningcs.LG

Exogenous state variables and rewards can slow down reinforcement learning by injecting uncontrolled variation into the reward signal. We formalize exogenous state variables and rewards and identify conditions under which an MDP with exogenous state can be decomposed into an exogenous Markov Reward Process involving on…

2018-06-05abs ↗pdf ↗

Active Reinforcement Learning: Observing Rewards at a Costcs.LG

ARL uses queries to learn rewards, focusing on cost vs. reward value.

problem How to efficiently use queries to learn rewards in reinforcement learning.

method Proposed and evaluated heuristic approaches for ARL in multi-armed bandits and MDPs.

result Challenging aspects of ARL highlighted, including intractability of value computation.

reinforcement learning active learning multi-armed bandits Markov decision processes query cost

2020-11-13abs ↗pdf ↗

Learning from Expert Factors: Trajectory-level Reward Shaping for Formulaic Alpha Miningcs.LG

TLRS improves predictive power of mined formulaic alpha factors.

problem Sparse rewards in RL for mining formulaic alpha factors.

method Trajectory-level Reward Shaping (TLRS) with reward centering.

result TLRS boosts predictive power by 9.29% over existing methods.

reinforcement learning formulaic alpha reward shaping sparse rewards stock indices

2025-07-27abs ↗pdf ↗

Concentration of Cumulative Reward in Markov Decision Processescs.LG

The paper studies reward concentration in MDPs, covering asymptotic and non-asymptotic settings.

problem Reward concentration in Markov Decision Processes (MDPs).

method Unified approach to reward concentration in MDPs, including asymptotic and non-asymptotic bounds.

result Rate-equivalent definitions of regret for learning policies.

Markov Decision Processes Reward Concentration Asymptotic Results Non-Asymptotic Bounds Martingale Decomposition

2024-11-27abs ↗pdf ↗

RUDDER: Return Decomposition for Delayed Rewardscs.LG

We propose RUDDER, a novel reinforcement learning approach for delayed rewards in finite Markov decision processes (MDPs). In MDPs the Q-values are equal to the expected immediate reward plus the expected future rewards. The latter are related to bias problems in temporal difference (TD) learning and to high variance p…

2018-06-20abs ↗pdf ↗

Unified Models of Human Behavioral Agents in Bandits, Contextual Bandits and RLcs.AI

Unified framework for human-like decision making in various sequential tasks.

problem Real-life decision-making involves diverse strategies leading to similar outcomes.

method Two-stream reward processing mechanism for flexible and unified models.

result Framework unified MAB, CB, and RL with comparable performance.

sequential decision making reward processing bandits contextual bandits reinforcement learning

2020-05-10abs ↗pdf ↗

HMM for Discovering Decision-Making Dynamics Using Reinforcement Learning Experimentscs.LG

Modeling decision-making dynamics in MDD patients using RL-HMM.

problem Characterize reward learning strategies in MDD patients.

method Proposed RL-HMM framework to analyze reward-based decision-making.

result MDD patients show reduced engagement in RL compared to healthy controls.

Major Depressive Disorder Reward Processing Decision-Making Reinforcement Learning Hidden Markov Model

2024-01-25abs ↗pdf ↗

Learning from an Exploring Demonstrator: Optimal Reward Estimation for Banditsstat.ML

Estimates rewards from a demonstrator's learning process.

problem Estimating rewards from a demonstrator's behavior.

method Leveraging the demonstrator's exploration phase for reward estimation.

result Consistent reward estimation possible without identifiability issues.

reward estimation bandits inverse reinforcement learning exploration demonstrator

2021-06-28abs ↗pdf ↗

Maximum Expected Hitting Cost of a Markov Decision Process and Informativeness of Rewardscs.LG

We propose a new complexity measure for Markov decision processes (MDPs), the maximum expected hitting cost (MEHC). This measure tightens the closely related notion of diameter [JOA10] by accounting for the reward structure. We show that this parameter replaces diameter in the upper bound on the optimal value span of a…

2019-07-03abs ↗pdf ↗

On the Power of (Approximate) Reward Models for Inference-Time Scalingcs.CL

This work shows how approximate reward models can significantly improve inference-time scaling.

problem Improving the efficiency of inference for large language models.

method Identifying the Bellman error of approximate reward models and using Sequential Monte Carlo (SMC) for inference.

result Approximate reward models can reduce computational complexity from exponential to polynomial in

T

Inference-time scaling Sequential Monte Carlo Reward models Bellman error Large language models

2026-02-01abs ↗pdf ↗

A Story of Two Streams: Reinforcement Learning Models from Human Behavior and Neuropsychiatrycs.LG

Drawing an inspiration from behavioral studies of human decision making, we propose here a more general and flexible parametric framework for reinforcement learning that extends standard Q-learning to a two-stream model for processing positive and negative rewards, and allows to incorporate a wide range of reward-proce…

2019-06-21abs ↗pdf ↗

Restless Bandit Problem with Rewards Generated by a Linear Gaussian Dynamical Systemstat.ML

Method predicts future rewards from past actions in a linear Gaussian system.

problem Maximizing cumulative reward in a stochastic multi-armed bandit with linear Gaussian dynamics.

method Proposes a method using a modified Kalman filter to predict future rewards based on past rewards.

result Reward from any action can be used to predict another action's future reward.

Bandit problem Linear Gaussian system Kalman filter Stochastic process Reward prediction

2024-05-15abs ↗pdf ↗

Evolutionary Reinforcement Learning for Sample-Efficient Multiagent Coordinationcs.LG

Many cooperative multiagent reinforcement learning environments provide agents with a sparse team-based reward, as well as a dense agent-specific reward that incentivizes learning basic skills. Training policies solely on the team-based reward is often difficult due to its sparsity. Furthermore, relying solely on the a…

2019-06-18abs ↗pdf ↗

Gaussian Process Bandits with Aggregated Feedbackcs.LG

New algorithm for recommending best arms with aggregated feedback.

problem Finding the best arm under aggregated feedback when precise rewards are unavailable.

method Gaussian Process Optimistic Optimisation (GPOO) algorithm with adaptive tree construction.

result The proposed algorithm achieves new simple regret bounds with aggregated feedback.

bandits Gaussian process aggregated feedback optimistic optimization

2021-12-24abs ↗pdf ↗

On Reward-Free Reinforcement Learning with Linear Function Approximationcs.LG

New algorithm for reward-free RL with linear function approximation, reducing sample complexity.

problem Efficiently learning optimal policies without prior reward information in complex environments.

method Developed an algorithm for reward-free RL in linear Markov decision processes, proving sample complexity bounds.

result Polynomial sample complexity in feature dimension and planning horizon, independent of states and actions.

reinforcement learning reward-free linear function approximation Markov decision processes sample complexity

2020-06-19abs ↗pdf ↗

Adversarial Rewards in Universal Learning for Contextual Banditsstat.ML

New findings on universal learning in contextual bandits with adversarial rewards.

problem Learning in contextual bandits with time-varying, adversarial rewards.

method Characterization of learnable processes and necessary/sufficient conditions for universal learning.

result Optimistic universal learning for contextual bandits with adversarial rewards is impossible in general.

contextual bandits adversarial rewards universal learning optimistic consistency time-varying rewards

2023-02-14abs ↗pdf ↗

Logarithmic regret bounds for continuous-time average-reward Markov decision processescs.LG

Logarithmic regret for continuous-time reinforcement learning.

problem Continuous-time Markov decision processes with unknown transition probabilities and holding times.

method Upper confidence reinforcement learning, mean holding time estimation, stochastic comparison of point processes.

result Logarithmic regret bound achieved in finite time.

reinforcement learning continuous-time MDPs average-reward logarithmic regret stochastic comparison

2022-05-23abs ↗pdf ↗

Detecting Spiky Corruption in Markov Decision Processescs.LG

Current reinforcement learning methods fail if the reward function is imperfect, i.e. if the agent observes reward different from what it actually receives. We study this problem within the formalism of Corrupt Reward Markov Decision Processes (CRMDPs). We show that if the reward corruption in a CRMDP is sufficiently "…

2019-06-30abs ↗pdf ↗

Output-Weighted Sampling for Multi-Armed Bandits with Extreme Payoffscs.LG

New acquisition function for extreme rewards in bandits.

problem Online decision making with extreme payoffs in multi-armed bandits.

method Modeling payoffs as Gaussian processes and using a novel UCB acquisition function.

result Demonstrated benefits across synthetic and real-world benchmarks.

multi-armed bandits extreme rewards Gaussian processes upper confidence bound attention mechanism

2021-02-19abs ↗pdf ↗

Achieving

\varepsilon^{-2}

Dependence for Average-Reward Q-Learning with a New Contraction Principlecs.LG

New Q-learning method achieves optimal sample complexity for average-reward problems.

problem Challenges in achieving optimal sample complexity for average-reward Q-learning.

method Synchronous and asynchronous Q-learning with a new contraction principle.

result Optimal

\widetilde{O}(\varepsilon^{-2})

sample complexity guarantees.

Q-learning Markov decision processes Contraction principle Average-reward Sample complexity

2026-01-29abs ↗pdf ↗

Tackling Decision Processes with Non-Cumulative Objectives using Reinforcement Learningcs.LG

This work extends reinforcement learning to handle non-cumulative objectives.

problem Optimizing functions of rewards rather than their sum in decision processes.

method Mapping NCMDPs to standard MDPs for reinforcement learning.

result Reinforcement learning techniques can be applied to NCMDPs.

reinforcement learning Markov decision processes non-cumulative objectives portfolio optimization dynamic programming

2024-05-22abs ↗pdf ↗

Multi-Armed Bandit Strategies for Non-Stationary Reward Distributions and Delayed Feedback Processescs.LG

A survey is performed of various Multi-Armed Bandit (MAB) strategies in order to examine their performance in circumstances exhibiting non-stationary stochastic reward functions in conjunction with delayed feedback. We run several MAB simulations to simulate an online eCommerce platform for grocery pick up, optimizing …

2019-02-22abs ↗pdf ↗

Contextual Combinatorial Multi-output GP Bandits with Group Constraintscs.LG

A new algorithm balances global reward and group constraints in federated multi-armed bandits.

problem Maximizing global reward while protecting client privacy in federated learning.

method Combinatorial contextual bandit with group constraints, using a two-output Gaussian process.

result TCGP-UCB incurs low regret, balancing super arm reward and group reward constraints.

federated learning bandit algorithms contextual bandits group constraints Gaussian processes

2021-11-29abs ↗pdf ↗

Split Q Learning: Reinforcement Learning with Two-Stream Rewardscs.LG

Drawing an inspiration from behavioral studies of human decision making, we propose here a general parametric framework for a reinforcement learning problem, which extends the standard Q-learning approach to incorporate a two-stream framework of reward processing with biases biologically associated with several neurolo…

2019-06-21abs ↗pdf ↗

Reward Estimation for Variance Reduction in Deep Reinforcement Learningcs.LG

Reinforcement Learning (RL) agents require the specification of a reward signal for learning behaviours. However, introduction of corrupt or stochastic rewards can yield high variance in learning. Such corruption may be a direct result of goal misspecification, randomness in the reward signal, or correlation of the rew…

2018-05-09abs ↗pdf ↗

Best arm identification in rare eventscs.LG

New algorithm identifies best arm in rare event scenarios.

problem Identifying the best arm with tiny reward probability.

method Approximated Compound Poisson process for faster algorithms.

result Improved computational efficiency with minor sample complexity increase.

stochastic bandits Compound Poisson process rare events best arm identification online advertising

2023-03-14abs ↗pdf ↗

Temporal-Logic-Based Reward Shaping for Continuing Reinforcement Learning Taskscs.AI

This paper introduces a new reward shaping method for average-reward reinforcement learning.

problem Speeding up convergence to an optimal policy in average-reward reinforcement learning tasks.

method Developed a temporal logic-based approach to automatically generate reward shaping functions.

result The optimal policy can be recovered using the proposed reward shaping framework.

reinforcement learning average-reward temporal logic reward shaping continuing tasks

2020-07-03abs ↗pdf ↗

Maximum Entropy Multi-Task Inverse RLcs.LG

New method learns multiple reward functions for complex tasks.

problem Learning reward functions for tasks with multiple ways of solving.

method Combines maximum entropy approach with Dirichlet process clustering.

result Method accurately learns reward functions for complex tasks.

maximum entropy multi-task inverse reinforcement learning Dirichlet process robotics reward function learning

2020-04-27abs ↗pdf ↗

Reward is enough for convex MDPscs.AI

Reward suffices for convex MDPs, expanding RL to new problems.

problem Capturing goals as convex functions of stationary distribution.

method Reformulated as a min-max game using Fenchel duality.

result Convex MDPs require non-stationary reward functions.

reinforcement learning Markov decision processes convex optimization Fenchel duality min-max games

2021-06-01abs ↗pdf ↗

Learning in Markov Decision Processes under Constraintscs.LG

UCRL-CMDP algorithm optimizes RL with constraints on average costs.

problem Optimizing RL in MDPs with average cost constraints.

method Model-based RL algorithms maximizing reward while keeping costs within bounds.

result UCRL-CMDP algorithm's expected regret is upper-bounded by $T^{2\slash 3}$.

reinforcement learning Markov decision processes average cost constraints UCRL-CMDP regret analysis

2020-02-27abs ↗pdf ↗

Faster Fixed-Point Methods for Multichain MDPsmath.OC

Faster algorithms for solving multichain MDPs under average-reward criterion.

problem Navigating towards the best connected component in multichain MDPs.

method Developed algorithms to better solve the navigational subproblem, achieving faster convergence rates.

result Improved rates of convergence and sharper complexity measures for multichain MDPs.

Markov decision processes value iteration average-reward multichain fixed-point methods

2025-06-26abs ↗pdf ↗

Optimal Sample Complexity for Average Reward Markov Decision Processescs.LG

New algorithm learns optimal policy for average reward MDPs with sample complexity matching lower bound.

problem Learning optimal policy for average reward in uniformly ergodic MDPs.

method Developed an estimator with sample complexity of O(|S||A|t_{mix}ε^{-2}).

result First algorithm to match lower bound of existing literature.

Markov Decision Processes Average Reward Policy Learning Sample Complexity Uniform Ergodicity

2023-10-13abs ↗pdf ↗

Stochastic first-order methods for average-reward Markov decision processescs.LG

Develops first-order methods for average-reward MDPs with strong guarantees.

problem Lack of strong theoretical guarantees for first-order methods in AMDPs.

method Average-reward stochastic policy mirror descent (SPMD) and variance-reduced temporal difference (VRTD) methods.

result Establishes sample complexity results for solving AMDPs.

Markov decision processes policy optimization policy evaluation first-order methods average-reward

2022-05-11abs ↗pdf ↗

Quick-Draw Bandits: Quickly Optimizing in Nonstationary Environments with Extremely Many Armscs.LG

A new method optimizes in nonstationary environments with many arms efficiently.

problem Optimizing in nonstationary environments with a large number of arms.

method Gaussian interpolation to learn continuous Lipschitz reward functions in nonstationary environments.

result Efficiently learns continuous Lipschitz reward functions with

\mathcal{O}^*(\sqrt{T})

cumulative regret.

bandits nonstationary continuous Gaussian interpolation Lipschitz

2025-05-30abs ↗pdf ↗

Quantifying Differences in Reward Functionscs.LG

EPIC quantifies reward differences without policy optimization.

problem Distinguishing reward function quality from policy optimization issues.

method EPIC distance to compare reward functions directly.

result EPIC bounds policy training success and regret.

reward functions policy optimization regret coverage distribution equivalence class

2020-06-24abs ↗pdf ↗

Sample Complexity of Average-Reward Q-Learning: From Single-agent to Federated Reinforcement Learningstat.ML

This work improves Q-learning for average-reward MDPs, reducing sample and communication complexities in federated settings.

problem Improving sample complexity of Q-learning for average-reward MDPs.

method Simple Q-learning algorithm with carefully chosen parameters for both single-agent and federated scenarios.

result Established first federated Q-learning algorithm for average-reward MDPs with provable efficiency in sample and communication complexities.

Q-learning average-reward federated learning sample complexity communication complexity

2026-01-20abs ↗pdf ↗

Model-free Reinforcement Learning in Infinite-horizon Average-reward Markov Decision Processescs.LG

Model-free reinforcement learning is known to be memory and computation efficient and more amendable to large scale problems. In this paper, two model-free algorithms are introduced for learning infinite-horizon average-reward Markov Decision Processes (MDPs). The first algorithm reduces the problem to the discounted-r…

2019-10-15abs ↗pdf ↗

Making the most of your day: online learning for optimal allocation of timestat.ML

Study on optimizing task allocation for agents receiving proposals sequentially.

problem Optimizing task allocation for agents receiving proposals sequentially.

method An agent receives task proposals sequentially and can either accept or reject a proposal. The study considers two scenarios: known reward function but unknown task duration distribution, and unknown reward function.

result Regret incurred by the agent in both scenarios.

online learning task allocation regret Poisson process bandits

2021-02-16abs ↗pdf ↗

Piecewise-Stationary Combinatorial Semi-Bandit with Causally Related Rewardscs.LG

Study adapts combinatorial semi-bandit for piecewise stationary, causally related rewards.

problem Nonstationary environment with changing base arms' distributions and causal relationships.

method Upper Confidence Bound (UCB) algorithm with change-point detector and group restart strategy.

result Regret upper bound reflecting effects of structural and distribution changes.

combinatorial semi-bandit piecewise stationarity causally related rewards change-point detection group restart

2023-07-26abs ↗pdf ↗

Achieving Tractable Minimax Optimal Regret in Average Reward MDPscs.LG

New algorithm achieves optimal regret in average reward MDPs without prior bias information.

problem Achieving optimal regret in average reward MDPs with computational efficiency and without prior bias information.

method Projective Mitigated Extended Value Iteration (PMEVI) to compute bias-constrained optimal policies efficiently.

result First tractable algorithm with minimax optimal regret of

\widetilde{\mathrm{O}}(\sqrt{\mathrm{sp}(h^*) S A T})

Markov Decision Processes Average Reward Minimax Optimal Regret Tractable Algorithms Bias Constrained Policies

2024-06-03abs ↗pdf ↗