Two algorithms improve Federated RL in diverse environments.
problem Collaborative learning in environments with varying dynamics.
method Proposed two federated RL algorithms, QAvg and PAvg, and a personalization heuristic.
result Achieved better performance and generalization in diverse environments.
Research shows collective learning across diverse environments is hard due to privacy and security concerns.
problem Privacy, security, and equity concerns restrict information sharing in diverse AI environments.
method Characterized learning algorithms as choice correspondences, provided minimum requirements for rational learning algorithms.
result The only rational learning algorithm in heterogeneous environments is unilaterally learning from a single environment without information sharing.
UAED discovers adaptive environments for robust learning.
problem Avoiding spurious correlations in data.
method Unified framework that learns a distribution over data transformations.
result Improves worst-case accuracy on standard benchmarks.
This paper introduces Dex, a reinforcement learning environment toolkit specialized for training and evaluation of continual learning methods as well as general reinforcement learning problems. We also present the novel continual learning method of incremental learning, where a challenging environment is solved using o…
In reinforcement learning algorithms, it is a common practice to account for only a single view of the environment to make the desired decisions; however, utilizing multiple views of the environment can help to promote the learning of complicated policies. Since the views may frequently suffer from partial observabilit…
We consider apprenticeship learning, i.e., having an agent learn a task by observing an expert demonstrating the task in a partially observable environment when the model of the environment is uncertain. This setting is useful in applications where the explicit modeling of the environment is difficult, such as a dialog…
New method learns robust representations by modeling environment variation.
problem Learning invariant representations across varying environments.
method Explicitly modeling variation across environments and marginalizing it out.
result Proposed method outperforms invariant-learning methods in various settings.
GALA framework learns invariant graph representations via environment augmentation with minimal assumptions.
problem Learning invariant graph representations from different environments without additional assumptions.
method Developed GALA framework with minimal assumptions of variation sufficiency and consistency. Uses an assistant model to differentiate graph environment changes.
result Extracting maximally invariant subgraphs to proxy predictions identifies underlying invariant subgraphs for successful out-of-distribution generalization.
Researchers use DT to transfer policies from one environment to another using causal reasoning.
problem Adapting to changes in environmental dynamics in reinforcement learning.
method Applying causal counterfactual reasoning to Decision Transformer (DT) architecture for policy transfer.
result DT successfully transfers a learned policy to new environments while retaining most of the reward.
A new method shapes reinforcement learning environments by abstracting large state spaces.
problem Learning in large, noisy environments with sparse feedback.
method Environment shaping using state abstraction.
result Agent's policy in shaped environment preserves near-optimal behavior in original environment.
Paper tackles efficient navigation in constrained environments using supervised and reinforcement learning.
problem Building learning agents that efficiently navigate in obstacle-cluttered environments.
method Synergistic use of supervised learning for path prediction and reinforcement learning for path following.
result Proposed method achieves good generalization and faster learning compared to existing work.
Reinforcement learning aims at searching the best policy model for decision making, and has been shown powerful for sequential recommendations. The training of the policy by reinforcement learning, however, is placed in an environment. In many real-world applications, however, the policy training in the real environmen…
Paper uses RL for high-level character control in 3D environments.
problem Creating intelligent characters with generalizable behavior.
method Combines traditional animations, heuristics, and reinforcement learning.
result Demonstrates learning of complex behaviors in a 3D environment.
New model-based methods adapt pre-trained policies to unseen environments efficiently.
problem High sample complexity in reinforcement learning limits practical applications.
method Combines online learning and adaptive control to adapt policies in unseen environments.
result Proves policies can quickly recover trajectories from source to target environments.
Learning in a non-stationary environment is an inevitable problem when applying machine learning algorithm to real world environment. Learning new tasks without forgetting the previous knowledge is a challenge issue in machine learning. We propose a Kalman Filter based modifier to maintain the performance of Neural Net…
In reinforcement learning algorithms, leveraging multiple views of the environment can improve the learning of complicated policies. In multi-view environments, due to the fact that the views may frequently suffer from partial observability, their level of importance are often different. In this paper, we propose a dee…
WILD-SCAV benchmarks AI in complex 3D FPS environments.
problem Lack of complexity and diversity in RL environments.
method Developed a 3D open-world FPS game environment.
result Demonstrates effectiveness in benchmarking RL algorithms.
OBSER framework infers sub-environments from objects, outperforming scene-based methods.
problem Zero-shot recognition of environments from object distributions.
method Bayesian framework using metric and self-supervised learning models to estimate object distributions in latent space.
result OBSER framework reliably performs inference in open-world and photorealistic environments, outperforming scene-based methods.
NetHack Learning Environment (NLE) tests RL algorithms, offering scalable, complex, and challenging gameplay.
problem Challenging environments for testing RL algorithms.
method Procedurally generated, stochastic, rich, and complex NetHack environment.
result Demonstrates empirical success for early stages of NetHack using RL.
Analyzes non-Markovian environments in stochastic approximation.
problem Understanding learning mechanisms in non-ergodic, non-Markovian settings.
method Analytic framework for transformer learning and continual learning.
result Proposes a new approach to transformer and continual learning.
LEADS improves model generalization across different environments.
problem Modeling dynamical systems from varied environments leads to biased or scarce solutions.
method LEADS learns a shared model capturing common dynamics and additional terms for environment-specific dynamics.
result LEADS improves model generalization for both known and novel environments.
MiniHack simplifies creation of complex RL environments.
problem Limited availability of challenging RL benchmarks.
method Develops a sandbox framework for easy RL environment design.
result MiniHack enables rapid creation of diverse RL testbeds.
The study questions the necessity of accurate models in MBRL.
problem The difficulty of achieving and utilizing accurate models in complex or changing environments.
method Introduces a meta-learning algorithm focused on model usefulness rather than accuracy.
result Meta-learning models enable faster learning than accurate models built with domain-specific knowledge.
This paper explores how environmental properties can simplify reinforcement learning in non-episodic settings.
problem Challenges in reinforcement learning with continuous interaction and sparse delayed rewards.
method Analysis of environment shaping and dynamism properties to simplify learning.
result Properties like environment shaping and dynamism can significantly ease learning in non-episodic, sparse reward settings.
New method improves RL/IL agents' adaptability to unseen environments.
problem Current RL/IL techniques struggle with generalizing to unseen environments.
method Zero-shot compositional policy learning with multi-modal fusion and attention mechanism.
result Language grounding enhances generalization across varied environments.
Algorithm learns from offline data to improve performance in target environment.
problem Learning from offline data in a target environment with unknown shifts.
method Adaptive algorithm that uses offline data to improve performance when informative.
result Algorithm provably improves performance over purely online learning when offline data are informative.
A framework for learning disentangled representations of symmetric environments.
problem Discovering and modelling the underlying structure of environments.
method Group representation theory for disentangled representations of dynamical environments.
result Our method enables accurate long-horizon predictions and correlates with disentanglement quality.
Proposes a method to improve few-shot transfer in off-dynamics RL.
problem Traditional RL struggles with transferring policies between environments with different dynamics.
method Introduces a penalty to regulate source-trained policies in target environments with limited data.
result Improves performance in various off-dynamics RL scenarios compared to existing methods.
We discuss a variant of Thompson sampling for nonparametric reinforcement learning in a countable classes of general stochastic environments. These environments can be non-Markov, non-ergodic, and partially observable. We show that Thompson sampling learns the environment class in the sense that (1) asymptotically its …
Deep learning agent improves pedestrian navigation in urban environments.
problem Autonomous driving among pedestrians in urban areas.
method Multi-objective deep reinforcement learning using a deep Q-learning variant.
result The multi-objective DQN agent outperforms single-objective DQN in various environments.
Object-based approaches for learning action-conditioned dynamics has demonstrated promise for generalization and interpretability. However, existing approaches suffer from structural limitations and optimization difficulties for common environments with multiple dynamic objects. In this paper, we present a novel self-s…
Self-supervised policy adapts after deployment without rewards.
problem Generalizing reinforcement learning policies across different environments.
method Uses self-supervision to train policies in new environments without reward signals.
result Significant improvements in generalization across diverse environments.
Equivariant CNNs improve RL performance in symmetric environments.
problem Learning equivariant representations for RL in symmetric environments.
method Proposed and studied equivariant CNNs for RL.
result Equivariant CNNs enhance RL performance and sample efficiency.
Proposes method to learn state abstractions that generalize across environments.
problem Learning abstractions that generalize in block MDPs.
method Invariant causal prediction to learn model-irrelevant state abstractions (MISA).
result Proves high probability of outputting a state abstraction corresponding to causal feature set for return.
Evaluation of deep reinforcement learning (RL) is inherently challenging. In particular, learned policies are largely opaque, and hypotheses about the behavior of deep RL agents are difficult to test in black-box environments. Considerable effort has gone into addressing opacity, but almost no effort has been devoted t…
We consider the problem of how a teacher algorithm can enable an unknown Deep Reinforcement Learning (DRL) student to become good at a skill over a wide range of diverse environments. To do so, we study how a teacher algorithm can learn to generate a learning curriculum, whereby it sequentially samples parameters contr…
Building models, or maps, of robot environments is a highly active research area; however, most existing techniques construct unstructured maps and assume static environments. In this paper, we present an algorithm for learning object models of non-stationary objects found in office-type environments. Our algorithm exp…
New approach transfers rewards learned in one environment to reinforcement learning in a new environment.
problem Transfer of rewards learned using inverse reinforcement learning from one environment to a new, different environment.
method Formulate the problem as a joint system of Bellman equations, develop minimax estimators for the target soft-q-function, solve the source and target system of equations jointly. result The coupled approach removes the first-order influence of source Bellman residual error compared to the sequential approach.
New reward function improves GAIL performance in task-based environments.
problem Reward bias in adversarial imitation learning.
method Proposed a new reward function to overcome existing biases.
result New reward function outperforms existing methods in task-based environments.
New method learns optimal environment and goal difficulty for reinforcement learning.
problem Lack of robust transfer in reinforcement learning.
method Self-Supervised Active Domain Randomization (SS-ADR).
result Optimal domain randomization strategy improves transfer in goal-directed tasks.
Bayesian model for multi-environment prediction with latent variable changes.
problem Prediction in environments with changing latent variable distributions.
method Bayesian model with empirical Bayes prior and amortized variational algorithm.
result Method outperforms previous approaches in new environments.
Reinforcement learning (RL) methods learn optimal decisions in the presence of a stationary environment. However, the stationary assumption on the environment is very restrictive. In many real world problems like traffic signal control, robotic applications, one often encounters situations with non-stationary environme…
ICIL learns policies invariant to multiple environments, improving generalization.
problem Learning policies from multiple environments leads to spurious correlations.
method ICIL learns invariant feature representations and a matching imitation policy.
result ICIL policies generalize better to unseen environments.
Assisted by neural networks, reinforcement learning agents have been able to solve increasingly complex tasks over the last years. The simulation environment in which the agents interact is an essential component in any reinforcement learning problem. The environment simulates the dynamics of the agents' world and henc…
Asynchronous Advantage Actor Critic (A3C) is an effective Reinforcement Learning (RL) algorithm for a wide range of tasks, such as Atari games and robot control. The agent learns policies and value function through trial-and-error interactions with the environment until converging to an optimal policy. Robustness and s…
PPOPT uses pretraining to speed up reinforcement learning in physics simulations.
problem High computational costs and inefficiency in reinforcement learning with small training samples.
method A novel policy neural network architecture that combines pretraining and fully-connected networks.
result PPOPT outperforms classic PPO on small training samples in terms of rewards and stability.
Complex environments and tasks pose a difficult problem for holistic end-to-end learning approaches. Decomposition of an environment into interacting controllable and non-controllable objects allows supervised learning for non-controllable objects and universal value function approximator learning for controllable obje…
Paper argues context equals environment, improving AI generalization.
problem AI models struggle to generalize in new environments.
method In-Context Risk Minimization (ICRM) algorithm.
result ICRM leads to significant out-of-distribution performance improvements.