LEADS improves model generalization across different environments.
problem Modeling dynamical systems from varied environments leads to biased or scarce solutions.
method LEADS learns a shared model capturing common dynamics and additional terms for environment-specific dynamics.
result LEADS improves model generalization for both known and novel environments.
Self-supervised policy adapts after deployment without rewards.
problem Generalizing reinforcement learning policies across different environments.
method Uses self-supervision to train policies in new environments without reward signals.
result Significant improvements in generalization across diverse environments.
UAED discovers adaptive environments for robust learning.
problem Avoiding spurious correlations in data.
method Unified framework that learns a distribution over data transformations.
result Improves worst-case accuracy on standard benchmarks.
New algorithm guarantees domain generalization with few environments.
problem Performing well on unseen environments with limited training data.
method Iterative feature matching algorithm with theoretical guarantees.
result Guaranteed domain generalization with logarithmic environments.
This paper introduces CENIE to quantify environment novelty for better UED.
problem Challenges in measuring environment novelty for effective UED.
method CENIE framework using state-action space coverage and Gaussian Mixture Models.
result CENIE improves UED performance across multiple benchmarks.
Proposes method to learn state abstractions that generalize across environments.
problem Learning abstractions that generalize in block MDPs.
method Invariant causal prediction to learn model-irrelevant state abstractions (MISA).
result Proves high probability of outputting a state abstraction corresponding to causal feature set for return.
Study shows tractable generalization in RL is impossible but possible with Strong Proximity.
problem RL agents struggle to generalize to new environments.
method Introduced Weak and Strong Proximity conditions to capture similarity between environments.
result Proved Strong Proximity is sufficient for efficient generalization.
We discuss a variant of Thompson sampling for nonparametric reinforcement learning in a countable classes of general stochastic environments. These environments can be non-Markov, non-ergodic, and partially observable. We show that Thompson sampling learns the environment class in the sense that (1) asymptotically its …
XRM discovers environments without human annotations for OOD methods.
problem Costly and biased manual annotations limit OOD methods.
method XRM trains twin networks to mimic mistakes, eliminating hyper-parameters.
result XRM achieves oracle worst-group-accuracy for OOD methods.
Paper argues context equals environment, improving AI generalization.
problem AI models struggle to generalize in new environments.
method In-Context Risk Minimization (ICRM) algorithm.
result ICRM leads to significant out-of-distribution performance improvements.
New method learns robust representations by modeling environment variation.
problem Learning invariant representations across varying environments.
method Explicitly modeling variation across environments and marginalizing it out.
result Proposed method outperforms invariant-learning methods in various settings.
Two algorithms improve Federated RL in diverse environments.
problem Collaborative learning in environments with varying dynamics.
method Proposed two federated RL algorithms, QAvg and PAvg, and a personalization heuristic.
result Achieved better performance and generalization in diverse environments.
GALA framework learns invariant graph representations via environment augmentation with minimal assumptions.
problem Learning invariant graph representations from different environments without additional assumptions.
method Developed GALA framework with minimal assumptions of variation sufficiency and consistency. Uses an assistant model to differentiate graph environment changes.
result Extracting maximally invariant subgraphs to proxy predictions identifies underlying invariant subgraphs for successful out-of-distribution generalization.
In model-based reinforcement learning, generative and temporal models of environments can be leveraged to boost agent performance, either by tuning the agent's representations during training or via use as part of an explicit planning mechanism. However, their application in practice has been limited to simplistic envi…
New method uses unlabeled data to improve model robustness across different environments.
problem Learning robust models for new, unseen environments when labeled data are scarce.
method Regularizes model sensitivity to perturbations in covariate means and covariances without requiring labels.
result Empirically validated on physical and physiological datasets, demonstrating improved robustness.
Paper uses RL for high-level character control in 3D environments.
problem Creating intelligent characters with generalizable behavior.
method Combines traditional animations, heuristics, and reinforcement learning.
result Demonstrates learning of complex behaviors in a 3D environment.
The paper formalizes robot environments using topological concepts.
problem Determining indistinguishable environments for robots.
method Formalizing robot systems as topological dynamical systems, using covering maps and bisimulation.
result Covering maps can witness indistinguishability of environments under general conditions.
When agents interact with a complex environment, they must form and maintain beliefs about the relevant aspects of that environment. We propose a way to efficiently train expressive generative models in complex environments. We show that a predictive algorithm with an expressive generative model can form stable belief-…
New method recovers causal DAGs from general environments without strict assumptions.
problem Recovering causal DAGs from real-world data with varying distributions.
method Formalizes desiderata for causal representation learning in general environments, leveraging sufficient change conditions up to third-order derivatives.
result Fully recovers latent DAG and identifies latent variables up to minor indeterminacies under nonparametric mixing.
MiniHack simplifies creation of complex RL environments.
problem Limited availability of challenging RL benchmarks.
method Develops a sandbox framework for easy RL environment design.
result MiniHack enables rapid creation of diverse RL testbeds.
A generative recurrent neural network is quickly trained in an unsupervised manner to model popular reinforcement learning environments through compressed spatio-temporal representations. The world model's extracted features are fed into compact and simple policies trained by evolution, achieving state of the art resul…
The paper explores how agents can generalize to new environments with unseen variables.
problem Generalizing to new environments with unseen variables.
method Investigates and proposes a method for efficient re-use of past marginal information to achieve out-of-variable generalization.
result The residual distribution after fitting a classifier reveals partial derivatives of the true generating function with respect to unobserved causal parents.
We demonstrate the use of conditional autoregressive generative models (van den Oord et al., 2016a) over a discrete latent space (van den Oord et al., 2017b) for forward planning with MCTS. In order to test this method, we introduce a new environment featuring varying difficulty levels, along with moving goals and obst…
New algorithm improves model generalization in structured biomedical domains.
problem Improving model generalization in structured biomedical domains.
method Proposes a new regret minimization (RGM) algorithm and its structured extension for better performance in diverse environments.
result Significantly outperforms previous state-of-the-art baselines on molecular property prediction, protein homology, and stability prediction.
Framework tackles OOD challenges in molecule property prediction by modeling environments.
problem Challenges in modeling OOD samples for molecule property prediction.
method Soft causal learning framework incorporating chemistry theories and cross-attention mechanisms.
result Demonstrates well generalization ability on seven datasets.
Bayesian model for multi-environment prediction with latent variable changes.
problem Prediction in environments with changing latent variable distributions.
method Bayesian model with empirical Bayes prior and amortized variational algorithm.
result Method outperforms previous approaches in new environments.
New method improves RL/IL agents' adaptability to unseen environments.
problem Current RL/IL techniques struggle with generalizing to unseen environments.
method Zero-shot compositional policy learning with multi-modal fusion and attention mechanism.
result Language grounding enhances generalization across varied environments.
Mixreg improves RL generalization by mixing diverse training environments.
problem Overfitting in RL agents trained in limited environments.
method Trains on a mixture of diverse observations and imposes linearity constraints.
result Mixreg outperforms baselines on unseen testing environments.
NetHack Learning Environment (NLE) tests RL algorithms, offering scalable, complex, and challenging gameplay.
problem Challenging environments for testing RL algorithms.
method Procedurally generated, stochastic, rich, and complex NetHack environment.
result Demonstrates empirical success for early stages of NetHack using RL.
GIB improves neural network generalization by dynamically selecting task-relevant features across different sequential environments.
problem Poor generalization of deep neural networks to unseen environments.
method Proposes a gated information bottleneck (GIB) approach that dynamically drops spurious correlations and selects task-relevant features.
result GIB outperforms other IB approaches in adversarial robustness and OOD detection.
New algorithms identify invariant features for domain generalization.
problem Achieving robust models across unseen environments.
method Invariant-Feature Subspace Recovery (ISR) algorithms.
result ISR algorithms achieve provable domain generalization with fewer training environments.
Research shows collective learning across diverse environments is hard due to privacy and security concerns.
problem Privacy, security, and equity concerns restrict information sharing in diverse AI environments.
method Characterized learning algorithms as choice correspondences, provided minimum requirements for rational learning algorithms.
result The only rational learning algorithm in heterogeneous environments is unilaterally learning from a single environment without information sharing.
This paper introduces Dex, a reinforcement learning environment toolkit specialized for training and evaluation of continual learning methods as well as general reinforcement learning problems. We also present the novel continual learning method of incremental learning, where a challenging environment is solved using o…
ICIL learns policies invariant to multiple environments, improving generalization.
problem Learning policies from multiple environments leads to spurious correlations.
method ICIL learns invariant feature representations and a matching imitation policy.
result ICIL policies generalize better to unseen environments.
It is a long-standing challenge to enable an intelligent agent to learn in one environment and generalize to an unseen environment without further data collection and finetuning. In this paper, we consider a zero shot generalization problem setup that complies with biological intelligent agents' learning and generaliza…
New method for ancestral inference in branching processes with random environments.
problem Determining ancestor distribution parameters in branching processes with random environments.
method Generalized method of moments for ancestral inference.
result Limiting distribution of ancestor and offspring estimators decouple and converge to independent Gaussian variables under certain conditions.
In reinforcement learning algorithms, leveraging multiple views of the environment can improve the learning of complicated policies. In multi-view environments, due to the fact that the views may frequently suffer from partial observability, their level of importance are often different. In this paper, we propose a dee…
WILD-SCAV benchmarks AI in complex 3D FPS environments.
problem Lack of complexity and diversity in RL environments.
method Developed a 3D open-world FPS game environment.
result Demonstrates effectiveness in benchmarking RL algorithms.
Object-based approaches for learning action-conditioned dynamics has demonstrated promise for generalization and interpretability. However, existing approaches suffer from structural limitations and optimization difficulties for common environments with multiple dynamic objects. In this paper, we present a novel self-s…
Dual behavior policy improves reinforcement learning across various environments.
problem Challenges in reinforcement learning with inconsistent or unknown environments.
method Dual, advantage-based behavior policy using counterfactual regret minimization.
result Demonstrated improved performance over baseline models in diverse environments.
We introduce Procgen Benchmark, a suite of 16 procedurally generated game-like environments designed to benchmark both sample efficiency and generalization in reinforcement learning. We believe that the community will benefit from increased access to high quality training environments, and we provide detailed experimen…
Reinforcement learning aims at searching the best policy model for decision making, and has been shown powerful for sequential recommendations. The training of the policy by reinforcement learning, however, is placed in an environment. In many real-world applications, however, the policy training in the real environmen…
New method ISR improves domain generalization with provable guarantees.
problem Achieving reliable performance across unseen environments.
method Invariant-feature Subspace Recovery (ISR) algorithms.
result ISR can achieve provable domain generalization with fewer training environments.
In this paper, we consider the problem of building learning agents that can efficiently learn to navigate in constrained environments. The main goal is to design agents that can efficiently learn to understand and generalize to different environments using high-dimensional inputs (a 2D map), while following feasible pa…
New model-based methods adapt pre-trained policies to unseen environments efficiently.
problem High sample complexity in reinforcement learning limits practical applications.
method Combines online learning and adaptive control to adapt policies in unseen environments.
result Proves policies can quickly recover trajectories from source to target environments.
We consider the problem of how a teacher algorithm can enable an unknown Deep Reinforcement Learning (DRL) student to become good at a skill over a wide range of diverse environments. To do so, we study how a teacher algorithm can learn to generate a learning curriculum, whereby it sequentially samples parameters contr…
New methods improve prediction intervals across multiple environments.
problem Valid confidence intervals and sets in multi-environment prediction.
method Extended jackknife and split-conformal methods, with resizing for problem difficulty.
result Distribution-free coverage achieved in non-traditional data scenarios.
Agents struggle with solving tasks in new environments, but new models improve performance.
problem Rapid task-solving in novel environments.
method Developed EPNs to enable deep RL agents to plan over gathered knowledge.
result EPNs enable deep RL agents to excel at RTS, outperforming baselines by factors of 2-3.