On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

169,051 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “natural environment”

Natural Environment Benchmarks for Reinforcement Learningcs.LG

New natural environment benchmarks for RL to improve robustness.

problem Current RL benchmarks lack complexity and real-world applicability.

method Developed three new RL benchmark domains with natural complexity.

result Proposed benchmarks improve generalization and algorithm robustness.

reinforcement learning benchmarking natural environment robustness generalization

2018-11-14abs ↗pdf ↗

Using Natural Language for Reward Shaping in Reinforcement Learningcs.LG

Natural language instructions improve reinforcement learning efficiency.

problem Designing effective reward functions for reinforcement learning is difficult and time-consuming.

method Proposes LanguagE-Action Reward Network (LEARN) to map natural language instructions to intermediate rewards.

result Language-based rewards lead to successful task completion 60% more often than without language.

reinforcement learning reward shaping natural language intermediate rewards Montezuma's Revenge

2019-03-05abs ↗pdf ↗

Nature versus nurture in galaxy formation: the effect of environment on star formation with causal machine learningastro-ph.GA

New study finds environment significantly suppresses star formation in galaxies, contrary to previous beliefs.

problem Understanding the role of environment in galaxy formation and evolution.

method Applied causal inference framework to IllustrisTNG simulations.

result Environment suppresses star formation by a factor of ~100, contrary to previous beliefs.

galaxy formation star formation environmental effects causal inference IllustrisTNG

2024-12-03abs ↗pdf ↗

Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learningcs.LG

Deep RL for multi-agent autonomous driving in dynamic environments.

problem Adapting to dynamic, multi-agent driving environments.

method Formulated Partially Observable Markov Games (POSG) for multi-agent learning.

result Demonstrated successful training of multi-agent control policies.

Deep Reinforcement Learning Multi-Agent Systems Autonomous Driving Markov Games Simulation Platforms

2019-11-11abs ↗pdf ↗

Self-Supervised Policy Adaptation during Deploymentcs.LG

Self-supervised policy adapts after deployment without rewards.

problem Generalizing reinforcement learning policies across different environments.

method Uses self-supervision to train policies in new environments without reward signals.

result Significant improvements in generalization across diverse environments.

reinforcement learning policy adaptation self-supervision generalization deployment

2020-07-08abs ↗pdf ↗

Guiding Reinforcement Learning Exploration Using Natural Languagecs.AI

In this work we present a technique to use natural language to help reinforcement learning generalize to unseen environments. This technique uses neural machine translation, specifically the use of encoder-decoder networks, to learn associations between natural language behavior descriptions and state-action informatio…

2017-07-26abs ↗pdf ↗

PixL2R: Guiding Reinforcement Learning Using Natural Language by Mapping Pixels to Rewardscs.LG

PixL2R maps natural language to pixel-based rewards for RL, improving sample efficiency.

problem Sparse reward settings in RL limit applicability to complex problems.

method Directly maps natural language descriptions to pixel-based rewards for guiding RL.

result Language-based rewards significantly improve sample efficiency in policy learning.

Reinforcement Learning Natural Language Sparse Rewards Policy Learning Sample Efficiency

2020-07-30abs ↗pdf ↗

Zero-shot Policy Learning with Spatial Temporal RewardDecomposition on Contingency-aware Observationcs.LG

Paper tackles zero-shot policy learning for agents in new environments.

problem Learning in one environment and generalizing to unseen environments without additional data.

method Decomposes sparse rewards into spatial-temporal regions and uses Model Predictive Control (MPC).

result Method achieves natural generalization to new environments, demonstrated on various tasks.

zero-shot learning policy learning contingency-aware Model Predictive Control sparse rewards

2019-10-17abs ↗pdf ↗

CARL: Controllable Agent with Reinforcement Learning for Quadruped Locomotioncs.LG

CARL controls a quadruped to move naturally in complex environments.

problem Motion synthesis in dynamic environments with complex constraints.

method CARL uses GANs to adapt high-level controls to action distributions and deep reinforcement learning for dynamic recovery.

result CARL can be controlled with high-level directives and react naturally to dynamic environments.

quadruped locomotion motion synthesis reinforcement learning GANs navigation

2020-05-07abs ↗pdf ↗

Revisiting Non-Acyclic GFlowNets in Discrete Environmentscs.LG

We explore non-acyclic GFlowNets in discrete settings.

problem Training and understanding non-acyclic GFlowNets in discrete environments.

method Relaxing acyclicity assumption, simpler theoretical framework, novel theoretical insights, experimental validation.

result Theoretical and experimental validation of non-acyclic GFlowNets in discrete environments.

GFlowNets discrete environments non-acyclic theoretical insights experimental validation

2025-02-11abs ↗pdf ↗

Posterior Sampling for Continuing Environmentscs.LG

PSRL extension for continuing environments reduces regret.

problem Formalizing and analyzing resampling approach for reinforcement learning.

method Continuing PSRL maintains a model of the environment and replaces it with samples from the posterior distribution.

result Established an

ilde{O}(τS \sqrt{A T})

bound on Bayesian regret.

reinforcement learning PSRL continuing environments posterior sampling rewards

2022-11-29abs ↗pdf ↗

Learning Group Structure and Disentangled Representations of Dynamical Environmentscs.LG

A framework for learning disentangled representations of symmetric environments.

problem Discovering and modelling the underlying structure of environments.

method Group representation theory for disentangled representations of dynamical environments.

result Our method enables accurate long-horizon predictions and correlates with disentanglement quality.

group representation disentanglement symmetric environments dynamical systems long-horizon predictions

2020-02-17abs ↗pdf ↗

Enforcing Predictive Invariance across Structured Biomedical Domainscs.LG

New algorithm improves model generalization in structured biomedical domains.

problem Improving model generalization in structured biomedical domains.

method Proposes a new regret minimization (RGM) algorithm and its structured extension for better performance in diverse environments.

result Significantly outperforms previous state-of-the-art baselines on molecular property prediction, protein homology, and stability prediction.

predictive invariance regret minimization structured environments biomedical domains molecular property prediction

2020-06-06abs ↗pdf ↗

Been There, Done That: Meta-Learning with Episodic Recallstat.ML

Meta-learning agents excel at rapidly learning new tasks from open-ended task distributions; yet, they forget what they learn about each task as soon as the next begins. When tasks reoccur - as they do in natural environments - metalearning agents must explore again instead of immediately exploiting previously discover…

2018-05-24abs ↗pdf ↗

Improving Environment Novelty Quantification for Effective Unsupervised Environment Designcs.LG

This paper introduces CENIE to quantify environment novelty for better UED.

problem Challenges in measuring environment novelty for effective UED.

method CENIE framework using state-action space coverage and Gaussian Mixture Models.

result CENIE improves UED performance across multiple benchmarks.

Unsupervised Environment Design Novelty Quantification Gaussian Mixture Models Curriculum Design State-Action Space

2025-02-08abs ↗pdf ↗

Learning mirror maps in policy mirror descentstat.ML

New mirror maps improve PMD performance in reinforcement learning.

problem Limited exploration of PMD's full potential due to focus on negative entropy.

method Evolutionary strategies to identify and learn more efficient mirror maps.

result Learned mirror maps outperform negative entropy in various environments.

reinforcement learning Policy Mirror Descent mirror maps Natural Policy Gradient evolutionary strategies

2024-02-07abs ↗pdf ↗

When Is Generalizable Reinforcement Learning Tractable?cs.LG

Study shows tractable generalization in RL is impossible but possible with Strong Proximity.

problem RL agents struggle to generalize to new environments.

method Introduced Weak and Strong Proximity conditions to capture similarity between environments.

result Proved Strong Proximity is sufficient for efficient generalization.

Reinforcement Learning Generalization Query Complexity Weak Proximity Strong Proximity

2021-01-01abs ↗pdf ↗

Functional Natural Policy Gradientsstat.ML

Proposes a method to learn policies from offline data with reduced bias.

problem Learning policies from offline data with reduced bias and complexity constraints.

method Cross-fitted debiasing device for policy learning from offline data.

result Achieves

\sqrt N

regret for complex policy classes with a product-of-errors nuisance remainder.

policy learning offline data bias reduction regret bound policy complexity

2026-03-30abs ↗pdf ↗

From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Followingcs.LG

New method converts natural language commands into reward functions for robots.

problem Creating effective reward functions for autonomous machines.

method Language-conditioned reward learning (LC-RL) using inverse reinforcement learning.

result Model learns transferable rewards from natural language commands.

reinforcement learning inverse reinforcement learning natural language reward functions robotics

2019-02-20abs ↗pdf ↗

RecSim: A Configurable Simulation Platform for Recommender Systemscs.LG

RecSim creates customizable simulation environments for RSs.

problem Creating realistic simulation environments for RSs.

method Configurable platform for authoring simulation environments.

result Enables pushing the limits of RL and RS techniques.

recommender systems simulation platform reinforcement learning user behavior item familiarity

2019-09-11abs ↗pdf ↗

Reward Transfer from Inverse Reinforcement Learning: A Coupled Minimax Approachcs.LG

New approach transfers rewards learned in one environment to reinforcement learning in a new environment.

problem Transfer of rewards learned using inverse reinforcement learning from one environment to a new, different environment.

method Formulate the problem as a joint system of Bellman equations, develop minimax estimators for the target soft-

q

-function, solve the source and target system of equations jointly.

result The coupled approach removes the first-order influence of source Bellman residual error compared to the sequential approach.

reinforcement learning inverse reinforcement learning reward transfer minimax approach Bellman equations

2026-05-27abs ↗pdf ↗

Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agentscs.MA

A game environment simulates competition among many agents for resources.

problem Understanding large-scale multiagent interactions and resource competition.

method Developed a persistent, massively multiplayer AI environment.

result Population size affects the development of skillful behaviors and niche differentiation.

AI Multiagent Systems Resource Competition Persistent Environment MMO

2019-03-02abs ↗pdf ↗

Adversarial recovery of agent rewards from latent spaces of the limit order bookcs.LG

Adversarial RL recovers agent rewards from financial market data simulations.

problem Recovering agent rewards in volatile financial markets with unknown dynamics.

method Adversarial inverse reinforcement learning in latent space simulations.

result Adversarial RL can robustly recover agent rewards from latent space representations of real market data.

reinforcement learning inverse reinforcement learning adversarial learning financial markets latent space

2019-12-09abs ↗pdf ↗

INTAGS: Interactive Agent-Guided Simulationcs.LG

INTAGS uses interactive simulation to improve realism in multi-agent systems.

problem Challenges in developing realistic agent-based simulators for multi-agent systems.

method INTAGS introduces a novel metric to evaluate the difference between real and synthetic multi-agent systems, optimizing a stochastic policy in reinforcement learning to adapt to interactive sequential decision-making environments.

result INTAGS generates more realistic market data compared to state-of-the-art approaches.

multi-agent systems interactive simulation reinforcement learning agent-based modeling causal inference

2023-09-04abs ↗pdf ↗

Experience-Efficient Model-Free Deep Reinforcement Learning Using Pre-Trainingcs.LG

PPOPT uses pretraining to speed up reinforcement learning in physics simulations.

problem High computational costs and inefficiency in reinforcement learning with small training samples.

method A novel policy neural network architecture that combines pretraining and fully-connected networks.

result PPOPT outperforms classic PPO on small training samples in terms of rewards and stability.

reinforcement learning deep learning pretraining model-free physics simulation

2025-10-11abs ↗pdf ↗

Quick-Draw Bandits: Quickly Optimizing in Nonstationary Environments with Extremely Many Armscs.LG

A new method optimizes in nonstationary environments with many arms efficiently.

problem Optimizing in nonstationary environments with a large number of arms.

method Gaussian interpolation to learn continuous Lipschitz reward functions in nonstationary environments.

result Efficiently learns continuous Lipschitz reward functions with

\mathcal{O}^*(\sqrt{T})

cumulative regret.

bandits nonstationary continuous Gaussian interpolation Lipschitz

2025-05-30abs ↗pdf ↗

Out-of-Variable Generalization for Discriminative Modelscs.LG

The paper explores how agents can generalize to new environments with unseen variables.

problem Generalizing to new environments with unseen variables.

method Investigates and proposes a method for efficient re-use of past marginal information to achieve out-of-variable generalization.

result The residual distribution after fitting a classifier reveals partial derivatives of the true generating function with respect to unobserved causal parents.

out-of-variable generalization animate learning subsets of variables partial derivatives residual distribution

2023-04-16abs ↗pdf ↗

Meta-Gradient Reinforcement Learningcs.LG

The goal of reinforcement learning algorithms is to estimate and/or optimise the value function. However, unlike supervised learning, no teacher or oracle is available to provide the true value function. Instead, the majority of reinforcement learning algorithms estimate and/or optimise a proxy for the value function. …

2018-05-24abs ↗pdf ↗

Efficient Empowermentstat.ML

Empowerment quantifies the influence an agent has on its environment. This is formally achieved by the maximum of the expected KL-divergence between the distribution of the successor state conditioned on a specific action and a distribution where the actions are marginalised out. This is a natural candidate for an intr…

2015-09-28abs ↗pdf ↗

A Benchmarking Environment for Reinforcement Learning Based Task Oriented Dialogue Managementstat.ML

Dialogue assistants are rapidly becoming an indispensable daily aid. To avoid the significant effort needed to hand-craft the required dialogue flow, the Dialogue Management (DM) module can be cast as a continuous Markov Decision Process (MDP) and trained through Reinforcement Learning (RL). Several RL models have been…

2017-11-29abs ↗pdf ↗

Recursive Experts: An Efficient Optimal Mixture of Learning Systems in Dynamic Environmentscs.LG

Recursive experts improve dynamic sequential learning systems.

problem Adapt to changing environments in sequential learning.

method Design hyper-experts that recursively merge to achieve optimal performance.

result Achieve minimax optimal regret bounds up to constant factors.

sequential learning dynamic environments recursive experts optimal mixture minimax regret

2020-09-19abs ↗pdf ↗

Translating Natural Language Instructions for Behavioral Robot Navigation with a Multi-Head Attention Mechanismcs.LG

Improved robot navigation using multi-head attention for natural language instructions.

problem Improving robot navigation in unfamiliar environments.

method Proposes a multi-head attention mechanism blending layer in a neural network model.

result Significant performance gains in translating instructions for unseen environments.

robot navigation natural language processing multi-head attention behavioral language generalization

2020-06-01abs ↗pdf ↗

Correlation Priors for Reinforcement Learningcs.LG

Bayesian framework captures correlations in discrete environments for better decision-making.

problem Capturing correlations in discrete state-action domains for better decision-making.

method Bayesian learning framework based on Pólya-Gamma augmentation.

result Superior predictive performance compared to correlation-agnostic models.

Bayesian learning correlation capture discrete environments reinforcement learning Pólya-Gamma augmentation

2019-09-11abs ↗pdf ↗

Measurement-based adaptation protocol with quantum reinforcement learningquant-ph

Machine learning employs dynamical algorithms that mimic the human capacity to learn, where the reinforcement learning ones are among the most similar to humans in this respect. On the other hand, adaptability is an essential aspect to perform any task efficiently in a changing environment, and it is fundamental for ma…

2018-03-14abs ↗pdf ↗

FinRL-Meta: A Universe of Near-Real Market Environments for Data-Driven Deep Reinforcement Learning in Quantitative Financeq-fin.TR

FinRL-Meta creates diverse market environments for DRL in finance.

problem Inaccurate financial data and diverse market environments challenge DRL in finance.

method Open-source data processing tools, hundreds of market environments, and multiprocessing.

result FinRL-Meta improves DRL accuracy and speed in financial simulations.

Deep Reinforcement Learning Financial Markets Data Driven Market Simulations Quantitative Finance

2021-12-13abs ↗pdf ↗

Does Invariant Risk Minimization Capture Invariance?stat.ML

IRM fails to capture natural invariances on simple problems.

problem IRM fails to capture natural invariances on simple problems.

method IRM formulation and practical linear variant.

result IRM can lead to worse generalization than unconstrained ERM.

Invariant Risk Minimization invariance generalization fragility sampling

2021-01-04abs ↗pdf ↗

Beyond Sequential Prediction: Learning Financial Market Dynamics in Volatile and Non-Stationary Environments through Sentiment-Conditioned Generative Modellingq-fin.ST

Paper proposes a hybrid model for financial time series prediction using sentiment analysis.

problem Challenges in forecasting in non-stationary, complex environments with heterogeneous data.

method Hybrid model combining GANs with NLP-based sentiment analysis.

result Hybrid model enhances robustness in non-stationary environments.

financial time series sentiment analysis GANs NLP non-stationary environments

2026-04-13abs ↗pdf ↗

Learning piecewise Lipschitz functions in changing environmentscs.LG

New algorithms minimize regret in changing environments for piecewise Lipschitz functions.

problem Optimizing in unpredictable, time-varying conditions for piecewise Lipschitz functions.

method Introduced shifting regret as a metric, and provided bounds for

O(\sqrt{sdT\log T}+sT^{1-β})

result Improved bounds for

β

-dispersed functions, with empirical validation in online clustering.

piecewise Lipschitz changing environments shifting regret online optimization dispersed functions

2019-07-22abs ↗pdf ↗

Toward A Normative Theory of Normative Marketing Theoryq-fin.GN

We show how different approaches to developing marketing strategies depending on the type of environment a firm faces, where environments are distinguished in terms of their systems properties not their context. Particular emphasis is given to turbulent environments in which outcomes are not a priori predictable and ar…

2012-03-06abs ↗pdf ↗

Using Phone Sensors and an Artificial Neural Network to Detect Gait Changes During Drinking Episodes in the Natural Environmentcs.CY

Phone sensors could be useful in assessing changes in gait that occur with alcohol consumption. This study determined (1) feasibility of collecting gait-related data during drinking occasions in the natural environment, and (2) how gait-related features measured by phone sensors relate to estimated blood alcohol concen…

2017-11-09abs ↗pdf ↗

Accelerating Reinforcement Learning through GPU Atari Emulationcs.LG

CuLE accelerates reinforcement learning on GPUs.

problem Limited CPU-GPU communication bandwidth in Atari Learning Environment.

method CUDA port of ALE, leveraging GPU parallelization and direct frame rendering.

result Up to 155M frames per hour on a single GPU, accelerating reinforcement learning.

Reinforcement Learning Atari Learning Environment CUDA GPU Emulation Deep Learning

2019-07-19abs ↗pdf ↗

Natural Policy Gradient for Average Reward Non-Stationary RLcs.LG

New RL algorithm tackles non-stationary environments with flexible policy updates.

problem Non-stationary reinforcement learning with time-varying rewards and transition probabilities.

method Model-free policy-based algorithm NS-NAC with restart-based exploration and dynamic learning rates.

result Dynamic regret of

ilde{\mathscr O}(|S|^{1/2}|A|^{1/2}Δ_T^{1/6}T^{5/6})

for both algorithms.

reinforcement learning non-stationary policy gradient average reward dynamic regret

2025-04-23abs ↗pdf ↗

Graph Constrained Reinforcement Learning for Natural Language Action Spacescs.LG

KG-A2C agent learns natural language IF games by reasoning and constraining action spaces.

problem Challenges of natural language understanding, partial observability, and combinatorially large action spaces in IF games.

method Builds a dynamic knowledge graph while exploring, constraining actions using templates.

result Outperforms current IF agents across various games with larger action spaces.

Reinforcement Learning Natural Language Interactive Fiction Knowledge Graph Action Spaces

2020-01-23abs ↗pdf ↗

Robot See, Robot Do: Imitation Reward for Noisy Financial Environmentscs.LG

Paper introduces a novel reward function for noisy financial markets using imitation learning.

problem Noisy reward function in financial markets hinders RL agent performance.

method Integrates imitation learning feedback with reinforcement learning to improve reward function design.

result Improves financial performance metrics compared to traditional benchmarks and RL agents.

reinforcement learning financial markets imitation learning reward function noisy signals

2024-11-13abs ↗pdf ↗

Model Based Planning with Energy Based Modelscs.LG

EBMs improve sample efficiency and generalization in RL.

problem Improving sample efficiency and generalization in reinforcement learning.

method Developed an online algorithm to train EBMs for model-based planning, leveraging their ability to infer intermediate states.

result EBMs lead to significantly better online learning and state space planning compared to feed-forward networks.

reinforcement learning energy-based models model-based planning sample efficiency generalization

2019-09-15abs ↗pdf ↗

Asymptotic Optimal Portfolio in Fast Mean-reverting Stochastic Environmentsq-fin.MF

This paper studies the portfolio optimization problem when the investor's utility is general and the return and volatility of the risky asset are fast mean-reverting, which are important to capture the fast-time scale in the modeling of stock price volatility. Motivated by the heuristic derivation in [J.-P. Fouque, R. …

2018-03-21abs ↗pdf ↗

Best of Both Worlds Model Selectioncs.LG

Develops model selection for bandits balancing adversarial and stochastic guarantees.

problem Model selection in bandit scenarios with simultaneous adversarial and stochastic high-probability regret.

method Nested policy classes, balanced candidate regret bounds, mis-specification tests.

result Best of both world guarantees in linear bandits with simultaneous adversarial and stochastic environments.

model selection bandit algorithms nested policies regret bounds mis-specification tests

2022-06-29abs ↗pdf ↗

Rank-1 Approximation of Inverse Fisher for Natural Policy Gradients in Deep Reinforcement Learningcs.LG

Paper presents a rank-1 approximation method for natural policy gradients in deep RL.

problem Computing natural gradients requires inverting the Fisher Information Matrix, which is computationally expensive.

method Develops a rank-1 approximation to the inverse Fisher Information Matrix for efficient natural policy optimization.

result The rank-1 approximation converges faster and has similar sample complexity to stochastic policy gradient methods.

natural policy gradients deep reinforcement learning Fisher Information Matrix rank-1 approximation covariant weight updates

2026-01-26abs ↗pdf ↗