On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “Inverse RL”

Rewriting History with Inverse RL: Hindsight Inference for Policy Improvementcs.LG

Rewriting history improves RL algorithms for solving multiple tasks.

problem Improving sample efficiency in multi-task reinforcement learning.

method Introducing hindsight relabeling as inverse RL to generalize goal-relabeling techniques.

result Relabeling data using inverse RL accelerates learning in multi-task settings.

reinforcement learning inverse reinforcement learning multi-task sample efficiency hindsight relabeling

2020-02-25abs ↗pdf ↗

Solving Bayesian inverse problems with diffusion priors and off-policy RLcs.LG

Paper uses RL and diffusion models to solve Bayesian inverse problems.

problem Bayesian inverse problems with latent biases.

method Relative Trajectory Balance (RTB) for RL, conditional diffusion models, off-policy backtracking exploration.

result RTB improves diffusion model posteriors for inverse problems.

Bayesian inverse problems RL Diffusion models Off-policy learning Posterior inference

2025-03-12abs ↗pdf ↗

Toward Computationally Efficient Inverse Reinforcement Learning via Reward Shapingcs.LG

This work simplifies IRL by using potential-based reward shaping.

problem Computational inefficiency in inverse reinforcement learning.

method Potential-based reward shaping to reduce RL sub-problems.

result Reduces computational burden of inverse reinforcement learning.

Inverse Reinforcement Learning Reward Shaping Computational Efficiency

2023-12-15abs ↗pdf ↗

Adversarial recovery of agent rewards from latent spaces of the limit order bookcs.LG

Inverse reinforcement learning has proved its ability to explain state-action trajectories of expert agents by recovering their underlying reward functions in increasingly challenging environments. Recent advances in adversarial learning have allowed extending inverse RL to applications with non-stationary environment …

2019-12-09abs ↗pdf ↗

Robo-Advising: Enhancing Investment with Inverse Optimization and Deep Reinforcement Learningq-fin.PM

Robo-advisor uses ML to optimize investment performance.

problem Maximizing investment performance with historical data.

method Inverse optimization and deep reinforcement learning.

result Robo-advisor consistently outperformed S&P 500.

Robo-advising Inverse Optimization Deep Reinforcement Learning Machine Learning Investment Management

2021-05-19abs ↗pdf ↗

Combining Reinforcement Learning and Inverse Reinforcement Learning for Asset Allocation Recommendationscs.LG

Combines human and AI to optimize fund managers' investment decisions.

problem Improving fund managers' investment practices.

method Combines Inverse Reinforcement Learning and Reinforcement Learning.

result Improves fund managers' investment performance.

Reinforcement Learning Inverse Reinforcement Learning Asset Allocation Fund Managers Optimization

2022-01-06abs ↗pdf ↗

Unlocking the Power of Representations in Long-term Novelty-based Explorationcs.LG

RECODE uses clustering and embedding to track state visitation counts in RL.

problem Efficient novelty-based exploration in nonstationary RL environments.

method Non-parametric clustering, online density estimation, inverse dynamics loss.

result RECODE achieves state-of-the-art performance in challenging RL tasks.

novelty-based exploration Deep RL clustering embedding inverse dynamics

2023-05-02abs ↗pdf ↗

The QLBS Q-Learner Goes NuQLear: Fitted Q Iteration, Inverse RL, and Option Portfoliosq-fin.CP

The QLBS model is a discrete-time option hedging and pricing model that is based on Dynamic Programming (DP) and Reinforcement Learning (RL). It combines the famous Q-Learning method for RL with the Black-Scholes (-Merton) model's idea of reducing the problem of option pricing and hedging to the problem of optimal reba…

2018-01-17abs ↗pdf ↗

Generalized Hindsight for Reinforcement Learningcs.LG

Generalized Hindsight improves RL by reusing data from one task for another.

problem High sample complexity in reinforcement learning due to wasted uninformative data.

method Approximate inverse reinforcement learning to relabel behaviors with better-suited tasks.

result Efficient reuse of samples, reducing sample complexity on multi-task RL tasks.

reinforcement learning inverse reinforcement learning multi-task learning sample efficiency behavior relabeling

2020-02-26abs ↗pdf ↗

Decoupling Dynamics and Reward for Transfer Learningcs.LG

Current reinforcement learning (RL) methods can successfully learn single tasks but often generalize poorly to modest perturbations in task domain or training procedure. In this work, we present a decoupled learning strategy for RL that creates a shared representation space where knowledge can be robustly transferred. …

2018-04-27abs ↗pdf ↗

Off-Policy Adversarial Inverse Reinforcement Learningcs.LG

Improves imitation learning in RL by learning reward function efficiently.

problem Lack of effective reward function approximation in AIRL for imitation tasks.

method Proposes Off-Policy AIRL that combines adversarial learning with efficient reward function approximation.

result Shows superior imitation performance and efficiency compared to state-of-the-art AIL algorithms.

Reinforcement Learning Adversarial Learning Reward Function Inverse Reinforcement Learning Efficiency

2020-05-03abs ↗pdf ↗

Show me the Way: Intrinsic Motivation from Demonstrationscs.LG

Study shows how to motivate AI agents like humans through learning from demonstrations.

problem Transfer human-like intrinsic motivation to AI agents in RL.

method Inverse RL approach to learn an exploration bonus from demonstrations.

result AI agents can learn and use complex exploration behaviors from demonstrations.

Intrinsic Motivation Deep Reinforcement Learning Inverse RL Demonstrations Exploration

2020-06-23abs ↗pdf ↗

A Noise-Robust Elicit-to-Optimize Framework for Distortion Riskmetrics via Inverse Reinforcement Learningcs.LG

Framework uses IRL and RL to elicit and optimize risk preferences robustly to noise.

problem Eliciting and optimizing risk preferences in noisy environments.

method Adaptive Bayesian IRL for elicitation, model-free RL for optimization, using quantile networks.

result Framework achieves convergence rate of

O(\exp(-cm+O(\sqrt{m\log m})))

Inverse Reinforcement Learning Reinforcement Learning Risk Metrics Distortion Risk Elicitation

2026-07-15abs ↗pdf ↗

Minimax Optimal Reinforcement Learning with Quasi-Optimismcs.LG

EQO uses a simple bonus term for efficient exploration in tabular RL.

problem Achieving minimax optimal reinforcement learning with practical efficiency.

method Exploration via Quasi-Optimism, employing a bonus proportional to state-action visit count.

result Achieves the sharpest known regret bound for tabular RL.

reinforcement learning minimax optimal tabular RL quasi-optimism exploration

2025-03-02abs ↗pdf ↗

Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHFcs.LG

New framework solves dynamic bilevel optimization problems in reinforcement learning.

problem Dynamic objective functions in reinforcement learning and human feedback.

method Principled penalty-based methods for bilevel reinforcement learning.

result Demonstrated effectiveness of penalty-based algorithms in simulations.

bilevel optimization reinforcement learning penalty methods dynamic objectives human feedback

2024-02-10abs ↗pdf ↗

Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspectivestat.ML

Paper tackles efficient IRL in offline settings with polynomial samples and runtime.

problem Efficiently learning reward functions from expert demonstrations in offline settings.

method Adapting the pessimism principle for offline RL, achieving strong guarantees.

result Achieved efficient IRL in offline and online settings with polynomial samples and runtime.

Inverse Reinforcement Learning Offline Reinforcement Learning Reward Learning Pessimism Principle Polynomial Samples

2023-11-29abs ↗pdf ↗

Policy Continuation with Hindsight Inverse Dynamicscs.LG

Solving goal-oriented tasks is an important but challenging problem in reinforcement learning (RL). For such tasks, the rewards are often sparse, making it difficult to learn a policy effectively. To tackle this difficulty, we propose a new approach called Policy Continuation with Hindsight Inverse Dynamics (PCHID). Th…

2019-10-30abs ↗pdf ↗

Evaluation of Active Feature Acquisition Methods for Static Feature Settingsstat.ML

Study improves feature acquisition for static settings in AFAPE.

problem Evaluate AFAPE performance in static feature settings.

method Derive and adapt IPW, DM, and DRL estimators for MAR and MNAR missingness.

result Improved data efficiency in synthetic and real-world experiments.

active feature acquisition reinforcement learning missing data static features inverse probability weighting

2023-12-06abs ↗pdf ↗

Robust Inverse Reinforcement Learning under Transition Dynamics Mismatchcs.LG

New algorithm reduces performance loss in IRL with mismatched transition dynamics.

problem Performance degradation in inverse reinforcement learning due to mismatched transition dynamics.

method Proposed a robust Maximum Causal Entropy (MCE) IRL algorithm leveraging robust reinforcement learning insights.

result Empirically demonstrated stable performance improvement under transition dynamics mismatches.

Inverse Reinforcement Learning Transition Dynamics Mismatch Maximum Causal Entropy Robust Reinforcement Learning

2020-07-02abs ↗pdf ↗

Robust Hierarchical-Optimization RLS Against Sparse Outlierscs.LG

This paper fortifies the recently introduced hierarchical-optimization recursive least squares (HO-RLS) against outliers which contaminate infrequently linear-regression models. Outliers are modeled as nuisance variables and are estimated together with the linear filter/system variables via a sparsity-inducing (non-)co…

2019-10-11abs ↗pdf ↗

Adversarial Reinforcement Learning under Partial Observability in Autonomous Computer Network Defencestat.ML

Recent studies have demonstrated that reinforcement learning (RL) agents are susceptible to adversarial manipulation, similar to vulnerabilities previously demonstrated in the supervised learning setting. While most existing work studies the problem in the context of computer vision or console games, this paper focuses…

2019-02-25abs ↗pdf ↗

Rank-1 Approximation of Inverse Fisher for Natural Policy Gradients in Deep Reinforcement Learningcs.LG

Paper presents a rank-1 approximation method for natural policy gradients in deep RL.

problem Computing natural gradients requires inverting the Fisher Information Matrix, which is computationally expensive.

method Develops a rank-1 approximation to the inverse Fisher Information Matrix for efficient natural policy optimization.

result The rank-1 approximation converges faster and has similar sample complexity to stochastic policy gradient methods.

natural policy gradients deep reinforcement learning Fisher Information Matrix rank-1 approximation covariant weight updates

2026-01-26abs ↗pdf ↗

Accelerated Portfolio Optimization and Option Pricing with Reinforcement Learningq-fin.PM

RL accelerates portfolio optimization and option pricing by dynamically adjusting preconditioner sizes.

problem Large linear systems in portfolio optimization and option pricing lead to slow convergence.

method Reinforcement Learning (RL) dynamically adjusts block-preconditioner sizes to accelerate convergence.

result RL-driven solver significantly reduces computational cost and accelerates convergence.

Reinforcement Learning Portfolio Optimization Option Pricing Preconditioning Convergence Acceleration

2025-06-23abs ↗pdf ↗

Beyond Winning and Losing: Modeling Human Motivations and Behaviors Using Inverse Reinforcement Learningcs.LG

In recent years, reinforcement learning (RL) methods have been applied to model gameplay with great success, achieving super-human performance in various environments, such as Atari, Go, and Poker. However, those studies mostly focus on winning the game and have largely ignored the rich and complex human motivations, w…

2018-07-01abs ↗pdf ↗

Joint Goal and Strategy Inference across Heterogeneous Demonstrators via Reward Network Distillationcs.LG

Method distills reward and strategies from diverse demonstrators.

problem Reward ambiguity and heterogeneity in human demonstrations.

method Reward network distillation to infer task goal and strategies.

result Better recovery of task and strategy rewards.

reinforcement learning inverse reinforcement learning reward network task goal strategic preferences

2020-01-02abs ↗pdf ↗

Sample-Efficient Reinforcement Learning Is Feasible for Linearly Realizable MDPs with Limited Revisitingcs.LG

Paper tackles sample-efficient RL for linearly realizable MDPs with limited revisiting.

problem Sample-efficient reinforcement learning for linearly realizable MDPs with limited revisiting.

method Develops a new sampling protocol that allows for backtracking and revisiting states in a controlled manner.

result Achieves polynomial sample complexity scaling with feature dimension, horizon, and inverse sub-optimality gap.

reinforcement learning sample efficiency linear realizability backtracking exploration

2021-05-17abs ↗pdf ↗

Reinforcement Learning with a Corrupted Reward Channelcs.AI

No real-world reward function is perfect. Sensory errors and software bugs may result in RL agents observing higher (or lower) rewards than they should. For example, a reinforcement learning agent may prefer states where a sensory error gives it the maximum reward, but where the true reward is actually small. We formal…

2017-05-23abs ↗pdf ↗

Scalable Reinforcement Learning via Adaptive Batch Scalingstat.ML

ABS dynamically adjusts batch size based on policy stability, improving RL performance.

problem Diminishing returns with large batch sizes in RL due to non-stationary data.

method Adaptive Batch Scaling (ABS) with Behavioral Divergence metric.

result Larger batch sizes can improve RL performance, contrary to conventional wisdom.

Reinforcement Learning Batch Scaling Behavioral Divergence Policy Volatility Large Batch Training

2026-05-20abs ↗pdf ↗

On the Exponential Convergence for Offline RLHF with Pairwise Comparisonscs.LG

RL-LOW algorithm achieves exponential simple regret in offline RLHF with pairwise comparisons.

problem Offline reinforcement learning from human feedback with pairwise comparisons.

method Proposes RL-LOW algorithm to minimize simple regret with exponential convergence.

result Achieves an exponential form of simple regret of \(\exp ( - Ω(n/H) )\).

offline reinforcement learning human feedback pairwise comparisons simple regret exponential convergence

2024-06-18abs ↗pdf ↗

State-Only Imitation Learning for Dexterous Manipulationcs.RO

State-only imitation learning improves dexterous manipulation learning from videos.

problem High sample complexity in complex domains like dexterous manipulation.

method Train an inverse dynamics model to predict actions from states and train the policy jointly.

result Performs on par with state-action approaches and outperforms RL alone.

state-only imitation dexterous manipulation inverse dynamics reinforcement learning demonstrations

2020-04-07abs ↗pdf ↗

Reinforcement Learning with Non-Exponential Discountingcs.LG

New RL approach handles non-exponential discounting for sequential decisions.

problem Modeling human discounting in sequential decision-making tasks.

method Generalized model-based reinforcement learning with arbitrary discount functions, using Hamilton-Jacobi-Bellman equation and collocation method.

result Validated approach on simulated problems, showing applicability to human discounting.

reinforcement learning non-exponential discounting sequential decision-making Hamilton-Jacobi-Bellman collocation method

2022-09-27abs ↗pdf ↗

S-RL Toolbox: Environments, Datasets and Evaluation Metrics for State Representation Learningcs.LG

State representation learning aims at learning compact representations from raw observations in robotics and control applications. Approaches used for this objective are auto-encoders, learning forward models, inverse dynamics or learning using generic priors on the state characteristics. However, the diversity in appl…

2018-09-25abs ↗pdf ↗

Neural-Kernel Conditional Mean Embeddingsstat.ML

Neural-Kernel CME tackles scalability and expressiveness challenges in conditional distribution representation.

problem Scalability and expressiveness challenges in kernel conditional mean embeddings.

method Combines deep learning with CMEs using a neural network optimization framework.

result Achieves competitive and often superior performance in conditional density estimation and RL.

kernel conditional mean embeddings deep learning neural networks reinforcement learning scalability

2024-03-16abs ↗pdf ↗

Model-based Offline Reinforcement Learning with Count-based Conservatismcs.LG

A new method, Count-MORL, improves offline reinforcement learning by using state-action frequency.

problem Improving offline reinforcement learning performance.

method Integrates count-based conservatism into model-based offline reinforcement learning.

result The learned policy is near-optimal and outperforms existing methods.

reinforcement learning offline learning model-based conservatism state-action frequency

2023-07-21abs ↗pdf ↗

SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewardscs.LG

Learning to imitate expert behavior from demonstrations can be challenging, especially in environments with high-dimensional, continuous observations and unknown dynamics. Supervised learning methods based on behavioral cloning (BC) suffer from distribution shift: because the agent greedily imitates demonstrated action…

2019-05-27abs ↗pdf ↗

A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscs.LG

Inverse reinforcement learning (IRL) is the problem of inferring the reward function of an agent, given its policy or observed behavior. Analogous to RL, IRL is perceived both as a problem and as a class of methods. By categorically surveying the current literature in IRL, this article serves as a reference for researc…

2018-06-18abs ↗pdf ↗

Flow Shape Design for Microfluidic Devices Using Deep Reinforcement Learningcs.LG

Microfluidic devices are utilized to control and direct flow behavior in a wide variety of applications, particularly in medical diagnostics. A particularly popular form of microfluidics -- called inertial microfluidic flow sculpting -- involves placing a sequence of pillars to controllably deform an initial flow field…

2018-11-29abs ↗pdf ↗

Maximum Entropy Multi-Task Inverse RLcs.LG

New method learns multiple reward functions for complex tasks.

problem Learning reward functions for tasks with multiple ways of solving.

method Combines maximum entropy approach with Dirichlet process clustering.

result Method accurately learns reward functions for complex tasks.

maximum entropy multi-task inverse reinforcement learning Dirichlet process robotics reward function learning

2020-04-27abs ↗pdf ↗

From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Followingcs.LG

Reinforcement learning is a promising framework for solving control problems, but its use in practical situations is hampered by the fact that reward functions are often difficult to engineer. Specifying goals and tasks for autonomous machines, such as robots, is a significant challenge: conventionally, reward function…

2019-02-20abs ↗pdf ↗

Hybrid Reinforcement Learning Breaks Sample Size Barriers in Linear MDPsstat.ML

Hybrid RL algorithms improve offline and online RL in linear MDPs.

problem Improving RL performance without single-policy concentrability.

method Developed computationally efficient algorithms for PAC and regret-minimizing RL in linear MDPs.

result Achieved sharper error or regret bounds for linear MDPs.

Hybrid RL PAC RL Regret-minimizing RL Linear MDPs Offline RL

2024-08-08abs ↗pdf ↗

SymCircuit: Bayesian Structure Inference for Tractable Probabilistic Circuits via Entropy-Regularized Reinforcement Learningcs.LG

SymCircuit learns PC structure via entropy-regularized RL, improving inference efficiency and accuracy.

problem Greedy algorithms in PC structure learning lead to suboptimal solutions.

method Entropy-regularized reinforcement learning to train a learned generative policy for PC structure inference.

result SymCircuit learns the optimal policy as a tempered Bayesian posterior, improving inference efficiency and accuracy.

probabilistic circuits reinforcement learning structure learning Bayesian inference entropy regularization

2026-03-20abs ↗pdf ↗

Rethinking Model-based, Policy-based, and Value-based Reinforcement Learning via the Lens of Representation Complexitycs.LG

This work explores representation complexity in RL paradigms, revealing model-based RL as the easiest task.

problem Investigating the representation complexity gap among model-based, policy-based, and value-based RL.

method Demonstrated through analysis of Markov decision processes (MDPs) and introduced new classes of MDPs.

result Representation complexity hierarchy: model-based RL > policy-based RL > value-based RL.

Reinforcement Learning Representation Complexity Model-Based RL Policy-Based RL Value-Based RL

2023-12-28abs ↗pdf ↗

Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamicscs.LG

New method defends RL agents from poisoning attacks without MDP knowledge.

problem Poisoning attacks on RL systems can cause learning failures.

method Generic poisoning framework for online RL, Vulnerability-Aware Adversarial Critic Poison (VA2C-P).

result Successfully prevents RL agents from learning good policies or converging to target policies.

Reinforcement Learning Poisoning Attacks Vulnerability-Aware Adversarial Critic Stability Radius

2020-09-02abs ↗pdf ↗

Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscs.LG

Reincarnating RL reuses prior work to accelerate RL progress.

problem Efficiency and accessibility in reinforcement learning for large-scale applications.

method Transfer of learned policies between RL agents or design iterations, focusing on value-based RL.

result Demonstrated gains in performance over tabula rasa RL on various tasks.

reinforcement learning policy transfer value-based RL efficiency democratization

2022-06-03abs ↗pdf ↗

Is RLHF More Difficult than Standard RL?cs.LG

Paper shows RLHF can be solved similarly to standard RL.

problem Difficulty of RLHF compared to standard RL.

method Reduction to reward-based RL techniques.

result RLHF can be solved using existing algorithms for reward-based RL.

Reinforcement Learning Human Feedback Reward-based RL Preference-based RL Robust RL

2023-06-25abs ↗pdf ↗

Inverse Reinforcement Learning from Summary Datacs.LG

Inverse reinforcement learning (IRL) aims to explain observed strategic behavior by fitting reinforcement learning models to behavioral data. However, traditional IRL methods are only applicable when the observations are in the form of state-action paths. This assumption may not hold in many real-world modeling setting…

2017-03-28abs ↗pdf ↗

Statistical and Algorithmic Foundations of Reinforcement Learningstat.ML

RL tackles decision making in unknown environments, focusing on efficiency and efficacy.

problem Efficiency and efficacy in RL algorithms for sample-starved situations.

method Markov Decision Processes, model-based and value-based approaches, policy optimization.

result Enhanced understanding and improvements in sample and computational efficacies of RL algorithms.

reinforcement learning sequential decision making sample complexity Markov decision processes algorithmic developments

2025-07-19abs ↗pdf ↗

Sample Efficient Ensemble Learning with Catalyst.RLcs.LG

Catalyst.RL accelerates RL research with efficient training.

problem Efficient reinforcement learning training in complex environments.

method Open-source PyTorch framework with distributed training and RL algorithms.

result Catalyst.RL achieved 2nd place in a computationally expensive RL challenge.

reinforcement learning PyTorch asynchronous training sample efficiency neurips

2020-03-29abs ↗pdf ↗