arXiv research

Adapts GRPO for off-policy RL, improving reward.

Unified framework for understanding GRPO as U-statistic.

GRPO optimizes LLMs with verifiable rewards, amplifying policy success.

Improved text-to-image and multimodal understanding through adaptive generation order optimization.

CoDistill-GRPO improves small models in GRPO by distilling knowledge from a larger model.

A fair policy for hiring candidates from different groups is proposed in a linear contextual bandit problem.

SAGE enhances reinforcement learning by injecting hints to prevent model stagnation.

Reinforcement Patching optimizes dynamic sequence patching for efficient time series forecasting.

The paper tackles fair sharing of exploration costs across groups in online learning.

SPO optimizes LLMs by eliminating group-based baselines and variance issues.

SFPO optimizes LLM reasoning by repositioning before updating, improving stability and efficiency.

Develops a new model to predict training dynamics of large language models.

Study minimax off-policy evaluation in multi-armed bandits with known and unknown behavior policies.

Extends reinforcement learning alignment to scalar rewards, improving math reasoning.

DFL framework improves action and outcome fairness in policy learning.

A new method to protect enterprise data privacy in AI models.

ADPO optimizes relative advantage in reinforcement learning from human feedback.

This work extends ME-RL using diffusion models to sample optimal policies.

VRER selectively reuses samples to improve policy optimization in complex systems.

Active-GRPO improves molecular optimization by actively deciding when to imitate or self-improve.

Evolutionary Strategies optimize hyper-parameters for off-policy learning.

Develops an anytime-valid framework for optimal policy identification from logged contextual bandit data.

Study uses causal machine learning to assess coupon campaign impact on retailer sales.

Investigates optimal pension policies in PAYG systems with forward utility and ageing population.

We accelerate PMD algorithms for reinforcement learning using functional methods.

A new approach optimizes weights in DLP for better risk-adjusted performance.

Success conditioning optimizes policies by imitating successful trajectories, solving a trust-region optimization problem.

ReSkill reconciles RL skill creation with policy optimization.

Unified framework for risk-aware policy learning in contextual bandits.

Although reinforcement learning (RL) can provide reliable solutions in many settings, practitioners are often wary of the discrepancies between the RL solution and their status quo procedures. Therefore, they may be reluctant to adapt to the novel way of executing tasks proposed by RL. On the other hand, many real-worl…

Three training methods for language models are shown to be variations of one another.

Proposes a new reinforcement learning algorithm using Q-function.

This paper first describes a class of uncertain stochastic control systems with Markovian switching, and derives an Itô-Liu formula for Markov-modulated processes. And we characterize an optimal control law, which satisfies the generalized Hamilton-Jacobi-Bellman (HJB) equation with Markovian switching. Then, by using …

The Exploration-Exploitation tradeoff arises in Reinforcement Learning when one cannot tell if a policy is optimal. Then, there is a constant need to explore new actions instead of exploiting past experience. In practice, it is common to resolve the tradeoff by using a fixed exploration mechanism, such as $ε$-greedy ex…

Bayesian regularization improves policy performance in noisy MDPs.

We present a new method of blackbox optimization via gradient approximation with the use of structured random orthogonal matrices, providing more accurate estimators than baselines and with provable theoretical guarantees. We show that this algorithm can be successfully applied to learn better quality compact policies …

Policy gradient methods with aggregated states can achieve better performance than approximate policy iteration.

Study optimizes experimental design for best treatment arm identification.

Methods for learning to search for structured prediction typically imitate a reference policy, with existing theoretical guarantees demonstrating low regret compared to that reference. This is unsatisfactory in many applications where the reference policy is suboptimal and the goal of learning is to improve upon it. Ca…

GOPO optimizes large models in Hilbert space, avoiding Kullback-Leibler's curvature.

This paper uses NLDT to find interpretable control rules from complex DRL policies.

Off-policy learning exhibits greater instability when compared to on-policy learning in reinforcement learning (RL). The difference in probability distribution between the target policy ($π$) and the behavior policy (b) is a major cause of instability. High variance also originates from distributional mismatch. The var…

We consider a market consisting of one safe and one risky asset, which offer constant investment opportunities. Taking into account both proportional transaction costs and linear price impact, we derive optimal rebalancing policies for representative investors with constant relative risk aversion and a long horizon.

An investor with constant relative risk aversion trades a safe and several risky assets with constant investment opportunities. For a small fixed transaction cost, levied on each trade regardless of its size, we explicitly determine the leading-order corrections to the frictionless value function and optimal policy.

POTEC tackles off-policy learning in large action spaces, improving effectiveness.

One-step policy improvement outperforms iterative RL methods on D4RL.

In recent years, deep learning has achieved remarkable achievements in many fields, including computer vision, natural language processing, speech recognition and others. Adequate training data is the key to ensure the effectiveness of the deep models. However, obtaining valid data requires a lot of time and labor reso…

Opioid overdose rates have reached an epidemic level and state-level policy innovations have followed suit in an effort to prevent overdose deaths. State-level drug law is a set of policies that may reinforce or undermine each other, and analysts have a limited set of tools for handling the policy collinearity using st…