Discovering and exploiting the causal structure in the environment is a crucial challenge for intelligent agents. Here we explore whether causal reasoning can emerge via meta-reinforcement learning. We train a recurrent network with model-free reinforcement learning to solve a range of problems that each contain causal…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “reinforced reasoning”
A framework helps reinforcement learning agents understand and decompose tasks from human demonstrations.
problem Sparse-reward tasks where demonstrations are used as sources of causal knowledge.
method Develops causal models through observation and reasons from this knowledge to decompose tasks.
result A basic implementation of Reasoning from Demonstration (RfD) is effective in sparse-reward tasks.
Enhances large language models' reasoning through simpler off-policy reinforcement learning.
problem Improving large language models' ability to reason and solve problems.
method EM Policy Gradient, optimizing expected return over reasoning trajectories using Expectation-Maximization (EM) optimization.
result Achieves comparable or slightly superior performance to state-of-the-art methods on reasoning datasets, with additional cognitive behaviors.
Method integrates logical rules into neural multi-hop reasoning for drug repurposing.
problem Capturing long-range dependencies in biomedical data.
method Combines logical rules with neural multi-hop reasoning using reinforcement learning.
result Our method outperforms baseline methods in drug repurposing tasks.
AER dynamically adjusts entropy regularization for better LLM reinforcement learning.
problem Policy entropy collapse in RLVR training limits exploration and reasoning performance.
method Adaptive Entropy Regularization (AER) with difficulty-aware coefficient allocation, initial-anchored target entropy, and dynamic global coefficient adjustment.
result AER consistently outperforms baselines on mathematical reasoning benchmarks, improving both accuracy and exploration.
Improved RL for knowledge graph reasoning with entity types.
problem Challenges in path-based relational reasoning over knowledge graphs.
method Type-enhanced RL agent using GNN for neighborhood information.
result Outperforms state-of-the-art RL methods and discovers novel paths.
HRM-Agent: Training a recurrent reasoning model in dynamic environments using reinforcement learningcs.AI
HRM-Agent learns to navigate dynamic mazes using reinforcement learning.
problem Training HRM in dynamic, uncertain, partially observable environments.
method Reinforcement learning to train HRM-Agent.
result HRM-Agent successfully learns to navigate dynamic mazes.
Platform combines RL and language models to study narrative influence on AI decisions.
problem Understanding how narrative elements shape AI decision-making.
method Dual-system architecture with reinforcement learning and language model integration.
result Initial experiments show narrative frameworks can influence AI decision-making.
Alpha-R1 uses LLMs to reason about economic factors and news for better alpha screening.
problem Challenges in data-driven investment strategies due to signal decay and regime shifts.
method Reinforcement learning trained on 8B parameters to evaluate alpha relevance under changing market conditions.
result Empirically outperforms benchmark strategies and shows improved robustness to alpha decay.
Curriculum learning improves model training efficiency and performance.
problem Expensive training of reasoning models using human or synthetic data.
method Autocurriculum, where models use their own performance to decide training problems.
result Autocurriculum reduces training costs and improves model performance.
Active-GRPO improves molecular optimization by actively deciding when to imitate or self-improve.
problem Training robust and efficient molecular optimization models with large language models.
method Active-GRPO combines imitation and reinforcement learning, upgrading references and policies dynamically.
result Improves molecular optimization performance, achieving statistically significant gains.
Researchers use DT to transfer policies from one environment to another using causal reasoning.
problem Adapting to changes in environmental dynamics in reinforcement learning.
method Applying causal counterfactual reasoning to Decision Transformer (DT) architecture for policy transfer.
result DT successfully transfers a learned policy to new environments while retaining most of the reward.
XLVINs improve deep reinforcement learning by combining self-supervised learning and neural algorithmic reasoning.
problem Limitations of Value Iteration Networks (VINs) in deep reinforcement learning.
method Combining contrastive self-supervised learning, graph representation learning, and neural algorithmic reasoning.
result XLVINs match VIN-like models on discrete, fixed MDPs and significantly outperform model-free baselines.
Trading-R1 uses LLMs for financial trading, improving risk-adjusted returns.
problem Lack of interpretability and trust in AI for finance.
method Supervised fine-tuning and reinforcement learning with a curriculum.
result Improved risk-adjusted returns and lower drawdowns compared to other models.
Hybrid framework injects TSLM insights into GRLM for robust time-series reasoning.
problem Lack of domain-specific knowledge in large language models for time-series reasoning.
method Hybrid knowledge-injection framework combining RLVR for efficient knowledge transfer.
result Consistently outperforms existing models by 7.9%-26.1% on multivariate time-series benchmarks.
R2-B2 optimizes game interactions with recursive reasoning.
problem Optimizing interactions between boundedly rational agents with unknown payoff functions.
method Recursive Reasoning-Based Bayesian Optimization (R2-B2) for repeated games.
result R2-B2 achieves faster asymptotic convergence to no regret than non-recursive methods.
BASIS: Batchwise Advantage Estimation from Single-Rollout Information Sharing for LLM Reasoningcs.LG
BASIS improves LLM reasoning by sharing batchwise rollout info, reducing MSE by 69%.
problem Improving large language model reasoning with limited rollouts and batch information.
method BASIS samples only one rollout per prompt but uses batch information to improve value function estimation.
result BASIS reduces MSE in value function estimation by 69% compared to REINFORCE++.
A minimalist approach improves LLM reasoning by filtering incorrect responses.
problem Improving large language model (LLM) reasoning on complex tasks.
method Revisit GRPO from a reinforce-like algorithm perspective, proposing Reinforce-Rej.
result RAFT, a simple rejection sampling baseline, outperforms GRPO and PPO.
The paper proposes a framework to reason about object dynamics for faster reinforcement learning.
problem Current reinforcement learning approaches lack prior knowledge about the environment, limiting efficiency.
method Integrates object dynamics and behavior into reinforcement learning to improve efficiency.
result Demonstrates the need for reasoning about object behavior and dynamics, leading to faster learning.
ConCuR generates high-quality CUDA kernels with concise reasoning traces.
problem Scarce high-quality data for kernel generation.
method Developed a pipeline to generate and curate high-quality CUDA kernels with reasoning traces.
result Our model achieves significant improvements in KernelBench setup.
G1 uses RL to enhance LLMs' graph reasoning, improving performance on diverse tasks.
problem Limited graph reasoning abilities of LLMs, especially in synthetic graph-theoretic tasks.
method Curated synthetic graph dataset, RL training on LLMs.
result Significant improvements in graph reasoning, zero-shot generalization to unseen tasks.
Trade-R1 bridges verifiable rewards to stochastic financial markets via process-level reasoning verification.
problem Extending RL to financial markets where rewards are verifiable but noisy.
method A verification method that transforms reasoning over financial documents into a structured RAG task, using a triangular consistency metric.
result DSR achieves superior cross-market generalization while maintaining reasoning consistency.
f-GRPO and Beyond: Divergence-Based Reinforcement Learning Algorithms for General LLM Alignmentcs.LG
Extends reinforcement learning alignment to scalar rewards, improving math reasoning.
problem Designing reinforcement learning algorithms for general LLM alignment.
method Introduces f-GRPO and f-HAL, estimating f-divergences between reward-aligned and unaligned distributions.
result Improves math reasoning RLVR tasks and mitigates reward hacking.
Dynamic abstention improves LLM accuracy by selectively terminating unpromising reasoning.
problem LLMs waste compute on incorrect responses, leading to inefficiency.
method Formal reinforcement learning framework with abstention reward parameter.
result Dynamic abstention outperforms natural baselines in selective accuracy.
The framework of reinforcement learning or optimal control provides a mathematical formalization of intelligent decision making that is powerful and broadly applicable. While the general form of the reinforcement learning problem enables effective reasoning about uncertainty, the connection between reinforcement learni…
Humans are capable of attributing latent mental contents such as beliefs or intentions to others. The social skill is critical in daily life for reasoning about the potential consequences of others' behaviors so as to plan ahead. It is known that humans use such reasoning ability recursively by considering what others …
This study explains RL training dynamics in LLMs, focusing on token-level optimization and reasoning pattern reshaping.
problem Understanding the training dynamics of RL in LLMs to improve their reasoning capabilities.
method Empirical analysis and theoretical modeling of RL training process, focusing on reasoning patterns and token optimization.
result RL primarily optimizes a sparse subset of critical tokens, reshaping reasoning pattern distributions and affecting model performance.
MIM-Reasoner learns seed users for maximizing influence in multiplex networks.
problem Maximizing influence in multiplex networks with multiple layers of social interactions.
method Combining reinforcement learning with probabilistic graphical models.
result Established a theoretical guarantee for MIM-Reasoner's performance.
Kernel smoothing improves LLM reasoning efficiency.
problem Efficiently estimate value functions with limited samples for reinforcement learning.
method Kernelized advantage estimation using classical nonparametric statistics.
result Improved policy optimization with accurate value and gradient estimation.
RLVR maintains safety while improving reasoning capabilities in LLMs.
problem Safety-capability tradeoff in fine-tuning LLMs.
method Reinforcement Learning with Verifiable Rewards (RLVR) and theoretical analysis.
result RLVR can enhance reasoning while maintaining safety guardrails.
This research tackles intervention-centric causal reasoning in learning agents by using meta-learning.
problem Learning agents lack the concept of interventions, making causal learning challenging.
method A meta-reinforcement learning algorithm is used to learn causal relationships from observational data.
result The approach enables agents to learn and manipulate the environment effectively.
New reinforcement learning framework for adapting to new actions.
problem Making reinforcement learning agents adaptable to new actions without retraining.
method Two-stage framework: infer action representations first, then train a flexible policy.
result Agents can make decisions from new action sets without retraining.
Improved reasoning model by sampling from power distribution without additional training.
problem Efficiently sampling from a sharpened distribution to improve reasoning models.
method Entropy-Cut Metropolis-Hastings algorithm that identifies key decision points for resampling.
result The method consistently improves reasoning models across various datasets.
Variational Proximal Policy Optimization improves reinforcement learning from human feedback.
problem Policy mode collapse and brittle exploration loops in reinforcement learning.
method Particle-based variational inference framework with Mixture-of-Experts architecture.
result Significant improvements in complex reasoning benchmarks.
SFPO optimizes LLM reasoning by repositioning before updating, improving stability and efficiency.
problem Noisy gradients from low-quality rollouts cause instability and inefficient exploration in on-policy RL algorithms.
method Decomposes each step into three stages: a short fast trajectory, repositioning, and slow correction, preserving the objective and rollout process unchanged.
result SFPO consistently improves stability, reduces rollouts, and accelerates convergence, outperforming GRPO on math reasoning benchmarks.
MACC learns communication protocols by adapting counterfactual reasoning.
problem Credit assignment and non-stationarity in communication environments.
method Adapts counterfactual reasoning to overcome credit assignment and uses action policy and Q-function of other agents to handle non-stationarity.
result MACC outperforms state-of-the-art baselines in four scenarios.
Develops models for temporally abstract reasoning and attention.
problem Temporal abstraction and attention in reinforcement learning.
method Defines affordances for options and develops partial option models.
result Identifies trade-offs between estimation and approximation error.
UCPO improves diversity in reinforcement learning models, maintaining high accuracy.
problem RLVR objectives often lead to diversity collapse, reducing coverage of correct solutions.
method UCPO adds a conditional uniformity penalty to GRPO, redistributing probability mass.
result UCPO improves Pass@K and diversity while maintaining competitive Pass@1 accuracy.
KG-A2C agent learns natural language IF games by reasoning and constraining action spaces.
problem Challenges of natural language understanding, partial observability, and combinatorially large action spaces in IF games.
method Builds a dynamic knowledge graph while exploring, constraining actions using templates.
result Outperforms current IF agents across various games with larger action spaces.
Reinforcement learning after next-token prediction aids in learning from diverse sequence lengths.
problem Learning from sequences of varying lengths and complexity.
method Introducing a framework to study reinforcement learning with autoregressive transformers, focusing on next-token prediction and mixture distributions of short and long sequences.
result Reinforcement learning after next-token prediction enables autoregressive transformers to generalize from long sequences, even when they are rare.
Chemical plants are complex and dynamical systems consisting of many components for manipulation and sensing, whose state transitions depend on various factors such as time, disturbance, and operation procedures. For the purpose of supporting human operators of chemical plants, we are developing an AI system that can s…
An important goal of research in Deep Reinforcement Learning in mobile robotics is to train agents capable of solving complex tasks, which require a high level of scene understanding and reasoning from an egocentric perspective. When trained from simulations, optimal environments should satisfy a currently unobtainable…
We introduce an approach for deep reinforcement learning (RL) that improves upon the efficiency, generalization capacity, and interpretability of conventional approaches through structured perception and relational reasoning. It uses self-attention to iteratively reason about the relations between entities in a scene a…
DPFRL uses particle filters for decision making with complex visual observations.
problem Decision making with partial complex visual observations.
method Discriminative Particle Filter Reinforcement Learning (DPFRL) with a differentiable particle filter in the neural network policy.
result DPFRL outperforms state-of-the-art POMDP RL models in complex visual observation tasks.
Develops RL algorithm for lifelong non-stationary environments.
problem Challenges of reinforcement learning in environments with persistent change.
method Formalizes lifelong non-stationarity, uses latent variable models, and leverages online learning and probabilistic inference.
result Substantial improvement in performance over non-reasoning approaches in lifelong non-stationary environments.
How to best explore in domains with sparse, delayed, and deceptive rewards is an important open problem for reinforcement learning (RL). This paper considers one such domain, the recently-proposed multi-agent benchmark of Pommerman. This domain is very challenging for RL --- past work has shown that model-free RL algor…
Bayesian RL enhances LLMs to reflectively explore and correct errors.
problem LLMs trained via RL lack reflective behaviors like rethinking and error correction.
method Bayesian RL framework that optimizes expected return under posterior distribution over Markov decision processes.
result BARL algorithm improves LLM performance in reasoning tasks.
The paper introduces Bellman-consistent pessimism to improve offline reinforcement learning without overly pessimistic bias.
problem Offline reinforcement learning's challenge of discovering good policies without exhaustive exploration.
method Introduces Bellman-consistent pessimism for function approximation, improving sample complexity and adaptability.
result Improves sample complexity by in the action space finite case, and automatically adapts to bias-variance tradeoff.