WILD-SCAV benchmarks AI in complex 3D FPS environments.
problem Lack of complexity and diversity in RL environments.
method Developed a 3D open-world FPS game environment.
result Demonstrates effectiveness in benchmarking RL algorithms.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “complex environments”
MiniHack simplifies creation of complex RL environments.
problem Limited availability of challenging RL benchmarks.
method Develops a sandbox framework for easy RL environment design.
result MiniHack enables rapid creation of diverse RL testbeds.
Building agents that can explore their environments intelligently is a challenging open problem. In this paper, we make a step towards understanding how a hierarchical design of the agent's policy can affect its exploration capabilities. First, we design EscapeRoom environments, where the agent must figure out how to n…
NetHack Learning Environment (NLE) tests RL algorithms, offering scalable, complex, and challenging gameplay.
problem Challenging environments for testing RL algorithms.
method Procedurally generated, stochastic, rich, and complex NetHack environment.
result Demonstrates empirical success for early stages of NetHack using RL.
Study shows tractable generalization in RL is impossible but possible with Strong Proximity.
problem RL agents struggle to generalize to new environments.
method Introduced Weak and Strong Proximity conditions to capture similarity between environments.
result Proved Strong Proximity is sufficient for efficient generalization.
We present MDP Playground, a testbed for Reinforcement Learning (RL) agents with dimensions of hardness that can be controlled independently to challenge agents in different ways and obtain varying degrees of hardness in toy and complex RL environments. We consider and allow control over a wide variety of dimensions, i…
Proposes a method to learn policies from offline data with reduced bias.
problem Learning policies from offline data with reduced bias and complexity constraints.
method Cross-fitted debiasing device for policy learning from offline data.
result Achieves regret for complex policy classes with a product-of-errors nuisance remainder.
New algorithm improves model generalization in structured biomedical domains.
problem Improving model generalization in structured biomedical domains.
method Proposes a new regret minimization (RGM) algorithm and its structured extension for better performance in diverse environments.
result Significantly outperforms previous state-of-the-art baselines on molecular property prediction, protein homology, and stability prediction.
New model-based methods adapt pre-trained policies to unseen environments efficiently.
problem High sample complexity in reinforcement learning limits practical applications.
method Combines online learning and adaptive control to adapt policies in unseen environments.
result Proves policies can quickly recover trajectories from source to target environments.
LEADS improves model generalization across different environments.
problem Modeling dynamical systems from varied environments leads to biased or scarce solutions.
method LEADS learns a shared model capturing common dynamics and additional terms for environment-specific dynamics.
result LEADS improves model generalization for both known and novel environments.
This paper introduces CENIE to quantify environment novelty for better UED.
problem Challenges in measuring environment novelty for effective UED.
method CENIE framework using state-action space coverage and Gaussian Mixture Models.
result CENIE improves UED performance across multiple benchmarks.
When agents interact with a complex environment, they must form and maintain beliefs about the relevant aspects of that environment. We propose a way to efficiently train expressive generative models in complex environments. We show that a predictive algorithm with an expressive generative model can form stable belief-…
This paper introduces Dex, a reinforcement learning environment toolkit specialized for training and evaluation of continual learning methods as well as general reinforcement learning problems. We also present the novel continual learning method of incremental learning, where a challenging environment is solved using o…
In reinforcement learning algorithms, it is a common practice to account for only a single view of the environment to make the desired decisions; however, utilizing multiple views of the environment can help to promote the learning of complicated policies. Since the views may frequently suffer from partial observabilit…
DR-MCTS improves decision quality and sample efficiency in complex environments.
problem Improving decision quality and sample efficiency in complex environments.
method Integrates Doubly Robust off-policy estimation into Monte Carlo Tree Search (MCTS).
result DR-MCTS achieves superior performance in Tic-Tac-Toe and VirtualHome tasks.
For sophisticated reinforcement learning (RL) systems to interact usefully with real-world environments, we need to communicate complex goals to these systems. In this work, we explore goals defined in terms of (non-expert) human preferences between pairs of trajectory segments. We show that this approach can effective…
Recent advances in artificial intelligence have been driven by the presence of increasingly realistic and complex simulated environments. However, many of the existing environments provide either unrealistic visuals, inaccurate physics, low task complexity, restricted agent perspective, or a limited capacity for intera…
Deep Deterministic Policy Gradient (DDPG) has been proved to be a successful reinforcement learning (RL) algorithm for continuous control tasks. However, DDPG still suffers from data insufficiency and training inefficiency, especially in computationally complex environments. In this paper, we propose Asynchronous Episo…
Reinforcement learning usually uses the feedback rewards of environmental to train agents. But the rewards in the actual environment are sparse, and even some environments will not rewards. Most of the current methods are difficult to get good performance in sparse reward or non-reward environments. Although using shap…
New algorithm improves RL performance across different environments.
problem Improving reinforcement learning performance across various environments.
method Designing a fully model-free DRRL algorithm that learns from a single trajectory.
result Demonstrates superior robustness and sample efficiency compared to existing methods.
In this paper, we train a recurrent neural network to learn dynamics of a chaotic road environment and to project the future of the environment on an image. Future projection can be used to anticipate an unseen environment for example, in autonomous driving. Road environment is highly dynamic and complex due to the int…
Intrinsically motivated goal exploration processes enable agents to autonomously sample goals to explore efficiently complex environments with high-dimensional continuous actions. They have been applied successfully to real world robots to discover repertoires of policies producing a wide diversity of effects. Often th…
Investigates Q value evolution in Stable Baselines for DQL in simple vs complex environments.
problem DQL in Stable Baselines struggles with simple non-game environments.
method Comparison of TrafficLight and FrozenLake environments; Q value decomposition analysis.
result Q values meander far from optimal in complex relationships between states.
Exploration Unboundcs.LG
An agent explores indefinitely in an environment with unlimited rewards.
problem Balancing exploration and exploitation in environments with unlimited rewards.
method Simple example of an environment with unbounded rewards and optimal agent behavior.
result An optimal agent always explores to maximize rewards, regardless of accumulated knowledge.
PSRL extension for continuing environments reduces regret.
problem Formalizing and analyzing resampling approach for reinforcement learning.
method Continuing PSRL maintains a model of the environment and replaces it with samples from the posterior distribution.
result Established an bound on Bayesian regret.
In reinforcement learning algorithms, leveraging multiple views of the environment can improve the learning of complicated policies. In multi-view environments, due to the fact that the views may frequently suffer from partial observability, their level of importance are often different. In this paper, we propose a dee…
Improved RL algorithm for robustness against parameter mismatches.
problem Learning robust control policies against parameter mismatches between training and testing environments.
method Formulated as DR-RL problem, proposed RPVL algorithm for tabular episodic learning with four divergences.
result Achieved sample complexity uniformly better than existing results.
AceIRL learns expert reward from active exploration.
problem Learning reward function from expert demonstrations in unknown environments.
method Active exploration to infer reward function and identify good policy.
result First approach to active IRL with sample-complexity bounds that doesn't require a generative model.
CARL controls a quadruped to move naturally in complex environments.
problem Motion synthesis in dynamic environments with complex constraints.
method CARL uses GANs to adapt high-level controls to action distributions and deep reinforcement learning for dynamic recovery.
result CARL can be controlled with high-level directives and react naturally to dynamic environments.
Assisted by neural networks, reinforcement learning agents have been able to solve increasingly complex tasks over the last years. The simulation environment in which the agents interact is an essential component in any reinforcement learning problem. The environment simulates the dynamics of the agents' world and henc…
Improved reinforcement learning for environments with distributional shifts.
problem Learning optimal policies in environments with distributional shifts.
method Distributionally robust Q-learning with multi-level Monte Carlo estimator.
result Proved upper bound on sample complexity for robust RL.
SMEs provide a transparent testbed for RL evaluation.
problem Lack of precise, white-box diagnostics in RL environments.
method Synthetic Monitoring Environments (SMEs) with fully configurable task characteristics and known optimal policies.
result SMEs allow for precise evaluation of RL algorithms, revealing the impact of specific environmental properties.
Models that can simulate how environments change in response to actions can be used by agents to plan and act efficiently. We improve on previous environment simulators from high-dimensional pixel observations by introducing recurrent neural networks that are able to make temporally and spatially coherent predictions f…
Efficient exploration in complex environments remains a major challenge for reinforcement learning. We propose bootstrapped DQN, a simple algorithm that explores in a computationally and statistically efficient manner through use of randomized value functions. Unlike dithering strategies such as epsilon-greedy explorat…
FEPS models agents to learn and act in complex environments without deep neural networks.
problem Modeling complex adaptive systems and understanding self-organizing behavior.
method Introducing Free Energy Projective Simulation (FEPS) within the constraints of the free energy principle and active inference.
result FEPS agents resolve ambiguity and infer optimal policies in partially observable environments.
In this paper, we consider the source of Deep Reinforcement Learning (DRL)'s sample complexity, asking how much derives from the requirement of learning useful representations of environment states and how much is due to the sample complexity of learning a policy. While for DRL agents, the distinction between represent…
Deep Recurrent Q-Network improves autonomous driving in urban areas with pedestrians.
problem Challenges in urban autonomous driving due to complex road structures and unpredictable pedestrian behavior.
method Combines Deep Q-Network with LSTM for long-term memory, designed a 3-D state representation, and uses a reward function.
result The proposed DRQN-based approach outperforms rule-based methods in dense urban scenarios.
Dual behavior policy improves reinforcement learning across various environments.
problem Challenges in reinforcement learning with inconsistent or unknown environments.
method Dual, advantage-based behavior policy using counterfactual regret minimization.
result Demonstrated improved performance over baseline models in diverse environments.
Inverse reinforcement learning has proved its ability to explain state-action trajectories of expert agents by recovering their underlying reward functions in increasingly challenging environments. Recent advances in adversarial learning have allowed extending inverse RL to applications with non-stationary environment …
New insights into RL with weak function approximation.
problem Statistical complexity of RL with function approximation in large state spaces.
method Agnostic policy learning framework, exploring environment access, coverage, and representational conditions.
result Characterization of fundamental performance bounds and statistical separations.
The study questions the necessity of accurate models in MBRL.
problem The difficulty of achieving and utilizing accurate models in complex or changing environments.
method Introduces a meta-learning algorithm focused on model usefulness rather than accuracy.
result Meta-learning models enable faster learning than accurate models built with domain-specific knowledge.
Environment Reconstruction with Hidden Confounders for Reinforcement Learning based Recommendationcs.LG
Reinforcement learning aims at searching the best policy model for decision making, and has been shown powerful for sequential recommendations. The training of the policy by reinforcement learning, however, is placed in an environment. In many real-world applications, however, the policy training in the real environmen…
New algorithm learns optimal policies with just 1 episode, settling horizon-dependence in RL.
problem Understanding the sample complexity of reinforcement learning with horizon length.
method Developed an algorithm using only episodes to achieve PAC guarantee, leveraging connections between value functions in discounted and finite-horizon MDPs and novel perturbation analysis.
result Achieved the same PAC guarantee with only episodes of environment interactions, completely settling horizon-dependence in RL.
Toward a Metrology for Artificial Intelligence: Hidden-Rule Environments and Reinforcement Learningcs.LG
Study explores reinforcement learning in a complex game environment, analyzing rule inference and policy learning.
problem Learning optimal policies in environments with hidden rules.
method Investigated using the Game Of Hidden Rules (GOHR) environment, employing Feature-Centric and Object-Centric state representations with a Transformer-based A2C algorithm.
result Transformer-based A2C models outperform traditional methods in GOHR, demonstrating the effectiveness of representation strategies.
Predictive sampling improves on Thompson sampling for non-stationary bandit environments.
problem Thompson sampling fails in non-stationary bandit environments.
method Proposes predictive sampling, which deprioritizes actions based on information loss rate.
result Predictive sampling outperforms Thompson sampling in all tested non-stationary environments.
Recent advances in reinforcement learning have proved that given an environment we can learn to perform a task in that environment if we have access to some form of a reward function (dense, sparse or derived from IRL). But most of the algorithms focus on learning a single best policy to perform a given set of tasks. I…
HySRL improves RL sample efficiency with shifted-dynamics data.
problem Leveraging historical data with shifted dynamics to improve sample efficiency in RL.
method HySRL, a hybrid transfer RL algorithm that uses prior information on dynamics shift to achieve better sample complexity.
result HySRL achieves problem-dependent sample complexity and outperforms pure online RL.
New RL method tackles sim-to-real gap using interactive data collection.
problem Sim-to-real gap in reinforcement learning.
method Distributionally robust reinforcement learning with interactive data collection.
result Proves sample-efficient learning is impossible without additional assumptions.