Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,932 papers · 148 categories

Trend · papers per month

68136204272 · Jun 202019922001200920172026
48 results for shared policy

This work formalizes and extends parameter sharing in multi-agent reinforcement learning.

problem Parameter sharing limits multi-agent learning to a single policy, preventing different tasks or action spaces.
method Introduces agent indication and extends parameter sharing to heterogeneous observation and action spaces.
result Proves convergence to optimal policies for parameter sharing in heterogeneous environments.

Developed policy gradient methods for stochastic control with exit time, outperforming traditional techniques in share repurchase pricing.

problem Optimal control with exit time in stochastic models.
method Two types of algorithms: direct policy learning and alternately learning value function and control.
result Policy gradient methods outperform PDE or neural networks in share repurchase pricing.

PPG separates policy and value function training phases for better reinforcement learning efficiency.

problem Challenges in traditional reinforcement learning methods for policy and value function optimization.
method Integrates Phasic Policy Gradient framework that splits policy and value function training into distinct phases.
result Significantly improves sample efficiency on Procgen Benchmark compared to PPO.

Optimal vehicle repositioning policy found for shared mobility services.

problem Matching fixed supply with spatial customer demand under uncertain and correlated demand.
method Base-stock repositioning policy, asymptotic optimality, regret analysis, adaptive repositioning algorithm.
result Surrogate Optimization and Adaptive Repositioning algorithm achieves optimal regret of O(n2.5T)O(n^{2.5} \sqrt{T}).

Most deep reinforcement learning algorithms are data inefficient in complex and rich environments, limiting their applicability to many scenarios. One direction for improving data efficiency is multitask learning with shared neural network parameters, where efficiency may be improved through transfer across related tas…

2017-07-13abs ↗pdf ↗

Most reinforcement learning algorithms are inefficient for learning multiple tasks in complex robotic systems, where different tasks share a set of actions. In such environments a compound policy may be learnt with shared neural network parameters, which performs multiple tasks concurrently. However such compound polic…

2018-02-27abs ↗pdf ↗

Deep RL policies share adversarial features across different MDPs.

problem Understanding decision boundaries and loss landscapes in neural policies.
method Investigating similarities in high sensitivity directions across MDPs using Arcade Learning Environment.
result High sensitivity directions for neural policies are correlated across MDPs, suggesting shared non-robust features.

New estimator uses clustering to improve off-policy evaluation accuracy.

problem Improving off-policy evaluation accuracy when logging and evaluation policies differ.
method Proposes an estimator that shares information across similar contexts using clustering.
result Clustering contexts improves estimation accuracy, especially in deficient information settings.

ABPS improves RL training efficiency by sharing policies and evolving hyper-params.

problem Data inefficiency in training deep RL models for real-world applications.
method ABPS: adaptive behavior policy sharing; ABPS-PBT: hybridizing ABPS with PBT for evolving hyper-params.
result ABPS achieves superior performance and reduced variance compared to conventional hyper-parameter tuning.

RL agents fail to generalize to unseen environments, even when dynamics are similar.

problem RL agents fail to generalize to unseen environments despite similar dynamics.
method Analyzed policy learning in POMDPs, formalized training dynamics as instances, and introduced a shared belief representation over an ensemble of specialized policies.
result Maximizing rewards induces instance-specific policies that are suboptimal on the training set.

Efficient actor-critic learning with shared experience replay improves data efficiency.

problem Challenges in actor-critic reinforcement learning with experience replay and off-policy learning stability.
method Combining actor-critic algorithms with shared experience replay, analyzing V-trace, proposing a trust region scheme.
result State-of-the-art data efficiency on Atari achieved with 200M environment frames.

PFedRL-Rep learns shared and personalized policies for heterogeneous environments.

problem Poor performance of single policy in heterogeneous environments.
method Develops PFedRL-Rep framework with shared feature representation and personalized weights.
result Proves linear convergence speedup with respect to the number of agents.

SHHK Stock Connect increases A-H share price premium, more for less efficient markets.

problem Impact of financial liberalization on cross-market pricing efficiency.
method Monthly data for 67 A-H dual-listed firms, system GMM dynamic models.
result Heterogeneous effect of SHHK Stock Connect on A-H price premium, more pronounced for less efficient markets.

We consider the problem of dynamic buying and selling of shares from a collection of NN stocks with random price fluctuations. To limit investment risk, we place an upper bound on the total number of shares kept at any time. Assuming that prices evolve according to an ergodic process with a mild decaying memory proper…

2009-09-22abs ↗pdf ↗

BASIS improves LLM reasoning by sharing batchwise rollout info, reducing MSE by 69%.

problem Improving large language model reasoning with limited rollouts and batch information.
method BASIS samples only one rollout per prompt but uses batch information to improve value function estimation.
result BASIS reduces MSE in value function estimation by 69% compared to REINFORCE++.

This paper improves auto-augment efficiency by sharing augmentation weights.

problem Efficient evaluation of augmentation policies for model training.
method Augmentation-Wise Weight Sharing (AWS) to create a fast yet accurate proxy task.
result Augmentation policies found achieve superior accuracies compared to existing methods.

The paper introduces return parity for fairness in MDPs, addressing delayed and adverse effects.

problem Fairness in MDPs for dynamic domains with delayed and adverse effects.
method Proposes return parity, decomposes return disparity, and develops algorithms for state visitation distributional alignment.
result The proposed algorithms can successfully close the disparity gap while maintaining policy performance.

Learning policies that generalize across multiple tasks is an important and challenging research topic in reinforcement learning and robotics. Training individual policies for every single potential task is often impractical, especially for continuous task variations, requiring more principled approaches to share and t…

2013-07-02abs ↗pdf ↗

FLAP adapts policies quickly to new tasks using shared linear representations.

problem Adapting policies to new tasks efficiently and effectively.
method FLAP uses a shared linear representation and a separate adapter network for quick adaptation.
result FLAP achieves up to 8X faster adaptation and significantly better performance on out-of-distribution tasks.

A new framework uses multi-agent reinforcement learning for evaluating policies in two-sided markets.

problem Evaluating the effects of different policies in two-sided markets with spatial and temporal interference.
method Introduces a multi-agent reinforcement learning (MARL) framework to address policy evaluation challenges in large-scale fleet management.
result Proposes novel estimators for mean outcomes under different products that are consistent despite high-dimensionality.

Soft modularization improves sample efficiency and performance in reinforcement learning.

problem Challenges in training multiple tasks jointly in reinforcement learning.
method Explicit modularization technique on policy representation, soft modularization method.
result Improves sample efficiency and performance over strong baselines in robotics manipulation tasks.

Improved SAC with AWMP for better control tasks.

problem Discontinuous and non-smooth optimal policies in reinforcement learning.
method Advantage Weighted Mixture Policy (AWMP) for SAC, learning state-specific weights.
result SAC with AWMP outperforms SAC in four control tasks.

This paper studies the deviations of the regret in a stochastic multi-armed bandit problem. When the total number of plays n is known beforehand by the agent, Audibert et al. (2009) exhibit a policy such that with probability at least 1-1/n, the regret of the policy is of order log(n). They have also shown that such a …

2011-07-22abs ↗pdf ↗

RHPO improves data-efficiency for hierarchical reinforcement learning.

problem High data requirements for general reinforcement learning algorithms in robotics.
method RHPO employs compositional inductive biases and task sharing mechanisms.
result RHPO enables stable and fast learning for complex domains with positive transfer.

We introduce reinforcement learning for heterogeneous teams in which rewards for an agent are additively factored into local costs, stimuli unique to each agent, and global rewards, those shared by all agents in the domain. Motivating domains include coordination of varied robotic platforms, which incur different costs…

2018-05-23abs ↗pdf ↗

We present POLO --- a C++ library for large-scale parallel optimization research that emphasizes ease-of-use, flexibility and efficiency in algorithm design. It uses multiple inheritance and template programming to decompose algorithms into essential policies and facilitate code reuse. With its clear separation between…

2018-10-08abs ↗pdf ↗

This work ensures policy gradient methods converge to global optima for certain control problems.

problem Non-convex optimization challenges in policy gradient methods for complex control problems.
method Identifies structural properties ensuring non-convex objective functions have no suboptimal stationary points.
result Policy gradient methods converge to global optima under certain conditions, satisfying a Polyak-Lojasiewicz condition.

We present Multitask Soft Option Learning(MSOL), a hierarchical multitask framework based on Planning as Inference. MSOL extends the concept of options, using separate variational posteriors for each task, regularized by a shared prior. This ''soft'' version of options avoids several instabilities during training in a …

2019-04-01abs ↗pdf ↗

A mechanism to share risks and costs with guarantees against extreme outcomes.

problem Softening extreme individual burdens in risk sharing schemes.
method Formalizes Certified Allocation Problem; uses Conformal Risk Sharing with interpretable sharing policy and split conformal calibration.
result Reduces extreme obligations for high-risk agents while controlling harm to others.

The paper tackles robust policy learning in multitask contextual bandits with adversarial users.

problem Learning optimal policies in multitask contextual bandits with a small fraction of adversarial users.
method Developed efficient robust mean estimators for both uni-variate and high-dimensional random variables.
result Lower bound of ildeΩ(min(S,A)α2/ε2) ildeΩ(\min(S,A) \cdot α^2 / ε^2) per-user interactions to learn an εε-optimal policy for good users.

Safe-M3^3-UCRL learns safe policies for multi-agent systems with global constraints.

problem Global constraints in mean-field reinforcement learning for multi-agent systems.
method Safe-M3^3-UCRL uses epistemic uncertainty and log-barrier approach to ensure constraints satisfaction.
result Safe-M3^3-UCRL learns safe policies for multi-agent systems with global constraints.

Enhances RL performance with a population-guided parallel learning scheme.

problem Improving off-policy reinforcement learning performance.
method Population-guided parallel learning scheme with shared experience replay buffer and soft policy update.
result Monotone improvement of the expected cumulative return proved theoretically and demonstrated in practice.

Paper tackles robust decision-making from multiple sites with shared structure.

problem Learning robust sequential decisions from heterogeneous multi-site datasets.
method Group-Robust MDPs with d-rectangular uncertainty sets, feature-wise worst-case aggregation, and cluster-level pooling.
result Proves suboptimality bound for robust planning policy under robust partial coverage assumption.

Optimizes insurance pricing by accounting for policyholders' price sensitivity.

problem Traditional insurance pricing does not consider policyholders' price sensitivity.
method Formulates insurance pricing as a decision-making problem and uses off-policy evaluation and stochastic control.
result Neural networks outperform existing techniques for policy optimization.

Study multi-agent RL in OTC markets, learning from agents' interactions.

problem Designing efficient RL solutions for OTC market interactions.
method Parameterized reward functions, shared policy learning, RL calibration.
result Agents learn to balance hedging and skewing in market simulations.

Investigates optimal pension policies in PAYG systems with forward utility and ageing population.

problem Optimal investment and pension policies in PAYG systems with sustainability and adequacy constraints.
method Non-zero volatility forward CRRA utilities, closed-form optimal policies, detailed numerical analysis.
result Characterization of optimal policies and detailed impact analysis under various scenarios.