A new method shapes reinforcement learning environments by abstracting large state spaces.
problem Learning in large, noisy environments with sparse feedback.
method Environment shaping using state abstraction.
result Agent's policy in shaped environment preserves near-optimal behavior in original environment.
Curiosity-driven exploration learns disentangled goal spaces for efficient complex environments.
problem Efficient exploration in complex environments with high-dimensional continuous actions.
method Learned disentangled goal spaces to reflect the environment's structure and maximize learning progress.
result Disentangled goal spaces lead to better exploration performances than entangled goal spaces.
OBSER framework infers sub-environments from objects, outperforming scene-based methods.
problem Zero-shot recognition of environments from object distributions.
method Bayesian framework using metric and self-supervised learning models to estimate object distributions in latent space.
result OBSER framework reliably performs inference in open-world and photorealistic environments, outperforming scene-based methods.
This paper introduces CENIE to quantify environment novelty for better UED.
problem Challenges in measuring environment novelty for effective UED.
method CENIE framework using state-action space coverage and Gaussian Mixture Models.
result CENIE improves UED performance across multiple benchmarks.
Adversarial RL recovers agent rewards from financial market data simulations.
problem Recovering agent rewards in volatile financial markets with unknown dynamics.
method Adversarial inverse reinforcement learning in latent space simulations.
result Adversarial RL can robustly recover agent rewards from latent space representations of real market data.
The paper formalizes robot environments using topological concepts.
problem Determining indistinguishable environments for robots.
method Formalizing robot systems as topological dynamical systems, using covering maps and bisimulation.
result Covering maps can witness indistinguishability of environments under general conditions.
Proposes method to learn state abstractions that generalize across environments.
problem Learning abstractions that generalize in block MDPs.
method Invariant causal prediction to learn model-irrelevant state abstractions (MISA).
result Proves high probability of outputting a state abstraction corresponding to causal feature set for return.
Introduces Conditional Action Trees to simplify RL action spaces.
problem Challenges in RL with large, complex action spaces.
method Structures action spaces and reduces complexity through Conditional Action Trees.
result Demonstrates effectiveness in reducing action space and improving decision making.
New method uses autoregressive models for dynamic planning.
problem Planning in dynamic environments with moving obstacles and goals.
method Conditional autoregressive generative models in a discrete latent space.
result Method nearly matches true environment performance for planning.
Study on variance of policy gradient in simple RL environments.
problem Understanding variance of policy gradient estimators in continuous RL.
method Analyzes REINFORCE estimator in linear-quadratic environments with Gaussian noise.
result Derives and validates bounds on estimator variance empirically.
Generative model learns spatial memory for predicting agent's future in complex environments.
problem Training generative and temporal models in partially observed, 3D environments is challenging.
method Action-conditioned generative model with non-parametric spatial memory and state-space model for dynamics.
result Scalable architecture capable of coherent predictions over hundreds of time steps in 2D and 3D environments.
Research shows collective learning across diverse environments is hard due to privacy and security concerns.
problem Privacy, security, and equity concerns restrict information sharing in diverse AI environments.
method Characterized learning algorithms as choice correspondences, provided minimum requirements for rational learning algorithms.
result The only rational learning algorithm in heterogeneous environments is unilaterally learning from a single environment without information sharing.
SMEs provide a transparent testbed for RL evaluation.
problem Lack of precise, white-box diagnostics in RL environments.
method Synthetic Monitoring Environments (SMEs) with fully configurable task characteristics and known optimal policies.
result SMEs allow for precise evaluation of RL algorithms, revealing the impact of specific environmental properties.
PackIt creates a virtual space for testing geometric planning skills.
problem Evaluating geometric planning abilities in virtual environments.
method Developed a virtual environment, PackIt, for geometric planning tasks.
result Demonstrated the effectiveness of various methods for geometric planning.
A framework for learning disentangled representations of symmetric environments.
problem Discovering and modelling the underlying structure of environments.
method Group representation theory for disentangled representations of dynamical environments.
result Our method enables accurate long-horizon predictions and correlates with disentanglement quality.
A new method optimizes in nonstationary environments with many arms efficiently.
problem Optimizing in nonstationary environments with a large number of arms.
method Gaussian interpolation to learn continuous Lipschitz reward functions in nonstationary environments.
result Efficiently learns continuous Lipschitz reward functions with O ∗ ( T ) \mathcal{O}^*(\sqrt{T}) O ∗ ( T ) cumulative regret. New algorithm guarantees domain generalization with few environments.
problem Performing well on unseen environments with limited training data.
method Iterative feature matching algorithm with theoretical guarantees.
result Guaranteed domain generalization with logarithmic environments.
Proposes a method for model-based RL in complex environments without perfect simulators.
problem Lack of cheap and perfect simulators in real-world tasks.
method Induces a world program by learning dynamics and actions in graph-based environments.
result World program enables complex planning tasks in environments without perfect simulators.
The paper uses Bayesian Surprise to identify unexpected structures in indoor environments.
problem Identifying unexpected structures in indoor environments.
method Bayesian Surprise applied to Isovist Analysis of 2D floor plans.
result Surprise regions in indoor environments can be used to focus on important areas in LBS.
Paper improves satellite collision prediction using ML.
problem Current orbit predictions fail due to lack of space data.
method Integrates physics-based models with ML for improved accuracy.
result ML approach reduces prediction errors and improves generalization.
A method for self-supervised representation learning in partially observable environments.
problem Sparse rewards and stochasticity in partially observable environments.
method World model in latent space for estimating missing information.
result Significant improvement in exploration compared to prior work.
BYOL-Explore learns to explore visually-rich environments by predicting world dynamics.
problem Exploration in visually complex environments.
method Optimizes a single prediction loss in latent space to learn world representation, dynamics, and exploration policy.
result Achieves superhuman performance on Atari games with simpler design.
Teacher algorithm helps DRL learn diverse environments efficiently.
problem Teach DRL to learn in various, unknown environments efficiently.
method Transformed into a bandit problem, learns to sample environments.
result ALP-GMM models learning progress, improving curriculum design.
Proves limiting distributions for Markov chains in random environments.
problem Analyzing Markov chains in random environments.
method Proves existence of limiting distributions using drift and minorization conditions.
result Law of large numbers holds for bounded functionals of the process.
A new exploration method for RL using parameter space noise.
problem Improving exploration in deep reinforcement learning.
method Switching isotropic and directional exploration in parameter space with parameter space noise.
result The proposed method achieves competitive results and better performance in sparse reward environments.
Deep belief networks improve Dyna-style planning in large state spaces.
problem Lack of real data and difficulty in learning a good generative model for large state spaces.
method Used deep belief networks to learn an environment model for Dyna-style planning.
result Deep belief networks significantly outperform linear expectation models in empirical validation.
New framework for PMD convergence in non-tabular environments.
problem Applying PMD to general policy classes with weak closure conditions.
method Develops a theoretical framework with a novel smoothness notion.
result Obtains upper bounds on convergence rate for non-tabular environments.
Paper introduces CRLMaze, a new benchmark for continual reinforcement learning in 3D non-stationary environments.
problem Challenges of training reinforcement learning agents in high-dimensional, always-changing environments.
method End-to-end model-free continual reinforcement learning strategy.
result Competitive results in a complex 3D non-stationary task, outperforming four baselines.
DeepMDP simplifies complex observations into continuous latent states.
problem Learning from high-dimensional observations in reinforcement learning.
method Trains a DeepMDP model that predicts rewards and next latent states.
result Optimization of DeepMDP objectives ensures quality of latent space and environment model.
Paper tackles efficient exploration of unseen graph-structured environments.
problem Efficient exploration of unseen graph-structured environments.
method Learning to explore framework using reinforcement learning and graph-structured memory.
result Approach outperforms hand-engineered methods in software testing and real-world applications.
Curiosity-driven exploration improves learning in sparse reward environments.
problem Sparse rewards or lack thereof in real-world scenarios.
method Formulate curiosity as prediction error in self-supervised feature space.
result Curiosity enables efficient exploration and skill learning in complex environments.
Paper explains failures of DDPG in simple deterministic environments.
problem DDPG's failures in trivial environments with sparse rewards.
method Formal explanation through a simple control problem and general analysis.
result DDPG can get stuck in poor solutions in deterministic environments.
The paper tackles causal discovery and forecasting in nonstationary environments using state-space models.
problem Challenges in identifying causal relations and forecasting in nonstationary time series.
method Exploiting a particular type of state-space model to represent nonstationary processes, allowing changes in causal strengths and noise variances.
result Nonstationarity helps identify causal structure and improves forecasting.
Combines NES and PPO to enhance exploration in various environments.
problem Improving exploration in reinforcement learning environments.
method Parameter transfer and parameter space noise methods for combining NES and PPO.
result PPO benefits from both NES methods in discrete and continuous control tasks.
RL agents optimize only specified features; this project infers unmentioned preferences from the state of the environment.
problem RL agents are indifferent to features not specified in a reward function, leading to unconsidered preferences.
method Developed an algorithm based on Maximum Causal Entropy IRL to infer preferences and side effects from the state of the environment.
result Information from the initial state can infer both side effects to avoid and preferences for environment organization.
Method adapts policies for new environments efficiently.
problem Difficulties in transferring reinforcement learning policies to new environments.
method Variational Policy Embedding (VPE) learns latent variables and a master policy.
result Policies can quickly adapt to new environments in latent space.
This work uses reinforcement learning to optimize task scheduling and execution in a dynamic multi-agent warehouse environment.
problem Optimizing task scheduling and execution in a dynamic multi-agent warehouse environment with limited observability.
method Deep reinforcement learning to solve both high-level scheduling and low-level multi-agent execution problems.
result Demonstrates the effectiveness of reinforcement learning in optimizing task scheduling and execution in a dynamic multi-agent environment.
POPLIN improves model-based planning in complex environments.
problem Efficient planning in complex high-dimensional environments.
method Combines policy networks with online planning, optimizing parameters directly.
result POPLIN achieves state-of-the-art performance in MuJoCo benchmarks, 3x more sample efficient.
VaST maps complex environments to simpler models for faster learning.
problem Sample inefficiency in deep reinforcement learning.
method Variational State Tabulation (VaST) and prioritized sweeping planning.
result VaST enables rapid learning and adaptation in complex tasks.
CEA augments reinforcement learning by generating counterfactual experiences.
problem Challenges in reinforcement learning, especially out-of-distribution and inefficient exploration.
method CEA uses variational autoencoders to model state transitions and introduces randomness for non-stationarity. It expands learning data through counterfactual inference.
result CEA outperforms SOTA algorithms in diverse environments.
HashReward improves imitation learning in high-dimensional environments by balancing reward generation and dimensionality reduction.
problem Making policies generalize well in high-dimensional state-action spaces, especially in game playing with raw pixel inputs.
method HashReward uses supervised hashing to balance reward generation and dimensionality reduction.
result HashReward outperforms state-of-the-art methods in high-dimensional environments.
KRCD detects unobserved confounders in nonlinear observational data.
problem Detecting unobserved confounders in nonlinear observational studies.
method Kernel Regression Confounder Detection (KRCD) using reproducing kernel Hilbert spaces.
result KRCD outperforms existing methods and achieves superior computational efficiency.
Diffusion models mimic human actions in sequential tasks.
problem Cloning human behavior in dynamic environments is challenging.
method Adapting diffusion models to handle stochastic, multimodal, and correlated actions.
result Diffusion models closely replicate human behavior in robotic and gaming tasks.
Develops HMRL for sparse reward RL problems, improving meta policy efficiency and transferability.
problem Difficulty in learning meta policies for sparse reward RL problems.
method Hyper-Meta RL framework with cross-environment meta state embedding and shaped meta reward.
result Improves meta policy generalization and efficiency for sparse reward RL problems.
Deep RL agent performs well in Doom, a complex FPS game.
problem Addressing complex environments with sparse rewards and large state spaces.
method Divide and conquer approach using unsupervised auxiliary tasks.
result Our agent performs better in unknown environments than state-of-the-art algorithms.
Robots learn spatial perception from sensorimotor invariants.
problem Developing autonomous robots that perceive space without human intuition.
method Study how a robot's motor commands relate to changes in exteroceptive inputs to deduce its spatial configuration.
result Robots can learn the configuration space of their sensors, revealing a planar position and orientation.
Quantum algorithms speed up reinforcement learning policies in large state-action spaces.
problem Limitations of quantum access in training reinforcement learning policies.
method Designing quantum algorithms to train reinforcement learning policies.
result Quantum algorithms offer full quadratic speed-ups in sample complexity for well-behaved policies.
HOMER learns latent states to explore rich environments efficiently.
problem Exploration in rich observation environments with unknown latent states.
method Interleaves representation learning and strategic exploration to identify kinematic states.
result Provably efficient exploration with polynomial sample complexity in latent states and time horizon.