Improves reinforcement learning extrapolation in Gridworlds.
problem Generalizing to unseen states in reinforcement learning.
method Avoiding deterministic action choice, using ego-centric representation, incorporating symmetry, and adding an entropy term.
result Significant improvement in extrapolation performance in Gridworld environments.
New method detects dangerous states in gridworlds using geometric defects.
problem Detecting dangerous states in gridworlds for AI safety.
method Modified state complexes and Gromov's Link Condition.
result Geometric defects indicate dangerous states in gridworlds.
New method tackles multi-task IRL in gridworlds with fewer demonstrations.
problem Inferring multiple reward functions from expert demonstrations in complex environments.
method Maximum Entropy IRL framework, leveraging meta-learning for function approximators.
result Approach solves multi-task IRL in gridworlds with fewer demonstrations than prior methods.
Designs a framework to transfer causal models between similar environments.
problem Transferability of causal models between different but similar environments.
method Object-oriented representations and continuous optimization for structure learning.
result Demonstrates advantages in gridworld settings using reinforcement learning.
Algorithm learns actions from past states in complex tasks.
problem Learning policies from human feedback is expensive.
method Combining learned feature encoder with inverse models to simulate past actions.
result Algorithm can infer specific skills from single state.
New methods for evaluating and optimizing policies in offline RL with unobserved confounders.
problem Evaluating and optimizing policies in the presence of unobserved confounders.
method Characterized settings and algorithms for consistent value estimates and lower bounds, with sample complexity guarantees.
result Proved local convergence guarantees for offline policy improvement.
Direct and indirect RL methods classified and compared.
problem Classifying RL methods for sequential decision making.
method Direct RL solves optimal policy directly, indirect RL solves Bellman equation.
result Direct and indirect RL methods are equivalent and can be unified.
A new RL framework helps align RL solutions with existing practices.
problem Discrepancies between RL solutions and existing procedures.
method Student-teacher RL mechanism with a constraint on policy difference.
result Proves asymptotic optimality and effectiveness in multiple scenarios.
ExTra uses past experience to guide exploration in reinforcement learning.
problem Navigating new tasks with limited prior knowledge.
method Transfer-guided exploration using bisimulation distances and Softmax sampling.
result ExTra outperforms traditional exploration methods in gridworld environments.
Agents learn to communicate and solve navigation tasks efficiently.
problem Understanding and developing communication in artificial agents.
method Multi-agent navigation tasks in gridworld environments.
result Agents develop an interpretable communication protocol that optimally solves tasks.
Bayesian approach models policy distribution for faster exploration and transfer learning.
problem Challenges in exploration and adaptation to new tasks in transfer learning.
method Modeling a distribution over policies in a Bayesian deep reinforcement learning setup.
result Favorable experimental results on GridWorld and MiniGrid environments.
Adapts auxiliary losses using gradient similarity to improve neural network performance.
problem Statistical inefficiency in neural networks and difficulty in selecting helpful auxiliary tasks.
method Uses cosine similarity between gradients of tasks to adaptively weight auxiliary losses.
result Guaranteed convergence to critical points of the main task and practical usefulness across domains.
DBS improves convergence in reinforcement learning.
problem Softmax operator convergence issues in reinforcement learning.
method Dynamic Boltzmann Softmax (DBS) updates value function.
result DBS enables better value function estimation and convergence.
Parenting algorithm improves AI safety by learning from human input.
problem Safety concerns in reinforcement learning, especially reward hacking and unsafe exploration.
method Inspired by parenting, a precise framework for learning from human input.
result Parenting algorithm solves safety problems in AI Safety gridworlds.
Detects spiky corruption in CRMDPs to learn optimal policies.
problem Learning optimal policies in environments with imperfect reward functions.
method Characterized spiky reward corruption, introduced algorithm to detect corrupt states.
result Algorithm can detect corrupt states and learn optimal policies.
PCHID improves sample efficiency in reinforcement learning tasks.
problem Sparse rewards make learning policies difficult in reinforcement learning.
method Hindsight Inverse Dynamics with Hindsight Experience Replay and Policy Continuation.
result PCHID significantly improves sample efficiency and final performance on multi-goal tasks.
Risk-aware active learning reduces generalization error.
problem Learning policies with minimal performance risk.
method Risk-aware active inverse reinforcement learning algorithm.
result Risk-aware active learning outperforms standard approaches.
In this paper we study how to learn stochastic, multimodal transition dynamics in reinforcement learning (RL) tasks. We focus on evaluating transition function estimation, while we defer planning over this model to future work. Stochasticity is a fundamental property of many task environments. However, discriminative f…
Active IRL selects optimal human demonstrations for learning AI preferences.
problem Costly human demonstrations in IRL for autonomous systems.
method Information-theoretic acquisition function for selecting informative human demonstrations.
result Efficiently reduces human effort in learning AI preferences.
UTE improves reinforcement learning by measuring action uncertainty, enhancing policy learning efficiency.
problem Degrading performance of action repetition in reinforcement learning, especially with sub-optimal actions.
method UTE uses ensemble methods to measure uncertainty during action extension, allowing strategic exploration or certainty.
result UTE outperforms existing action repetition algorithms, significantly enhancing policy learning efficiency.
Algorithm learns mixtures of Markov chains and MDPs from short trajectories.
problem Learning mixtures of Markov chains and MDPs from short unlabeled trajectories.
method Subspace estimation, spectral clustering, EM algorithm, model estimation, classification.
result 96.6% average accuracy on a mixture of two MDPs in gridworld, outperforming EM algorithm with random initialization.
Goals for reinforcement learning problems are typically defined through hand-specified rewards. To design such problems, developers of learning algorithms must inherently be aware of what the task goals are, yet we often require agents to discover them on their own without any supervision beyond these sparse rewards. W…
A new method for learning policies in multiple environments.
problem Learning policies that work in different but related environments.
method Decentralized entropy-regularized policy gradient method.
result The method can learn effective policies in multiple environments.
Eigenoptions improve credit assignment in reinforcement learning.
problem Improving credit assignment in reinforcement learning models.
method Investigated eigenoptions for credit assignment in model-free RL, comparing pre-specified and online discovery methods.
result Pre-specified eigenoptions aid exploration and credit assignment, while online discovery can hinder learning.
New algorithm infers reward function from agent's learning trajectories.
problem Inferring reward function from agent's learning data.
method Gradient-based approach to recover reward function.
result Improved performance compared to state-of-the-art methods.
This work improves policy optimization by maximizing entropy of state distribution, leading to better exploration.
problem Lack of exploration in state space when maximizing policy entropy.
method Proposes maximizing the entropy of a lower bound approximation to the state weighting distribution, based on latent space representation.
result Entropy regularization based on marginal state distribution achieves superior state space coverage and better performance in various domains.
CoCoRL learns safe constraints from demonstrations with unknown rewards.
problem Learning safe constraints from demonstrations with different unknown rewards.
method Convex Constraint Learning for Reinforcement Learning (CoCoRL) constructs a convex safe set based on demonstrations.
result CoCoRL learns constraints that lead to safe driving behavior and can safely transfer to different tasks and environments.
Directed exploration improves reinforcement learning efficiency and robustness.
problem Achieving good sample efficiency in reinforcement learning with efficient exploration.
method Directed exploration through goal-conditioned policies that are independent of uncertainty.
result Directed exploration is more efficient and robust to uncertainty than reward bonuses.
NOHD optimizes multi-agent systems by decomposing dynamics into irrotational and solenoidal components.
problem Non-stationarity and conflicting interests in multi-agent learning problems.
method NOHD (Newton Optimization on Helmholtz Decomposition) decomposes system dynamics into irrotational and solenoidal components.
result NOHD ensures quadratic convergence in purely irrotational and solenoidal systems and attracts to stable fixed points in general multi-agent systems.
New approach to avoid bad incentives in reinforcement learning agents.
problem Designing safe reinforcement learning agents that avoid unnecessary disruptions.
method Break down side effects penalties into baseline state and deviation measure; introduce new stepwise inaction baseline and relative reachability deviation measure.
result Combination of new design choices avoids undesirable incentives, while simpler alternatives fail.
USFs capture dynamics for faster RL task transfer.
problem Applying knowledge from one task to another.
method Proposed Universal Successor Features (USFs) for RL.
result USFs accelerate training and transfer knowledge.
New neural network learns to transfer across environments and tasks from sparse data.
problem Learning transfer across environments and tasks with limited data.
method Compositional neural network architecture and novel training methods to learn embeddings.
result High success rates on all (ENV, TASK) pairs after learning from only 40% of them.
The paper investigates how to ensure safety-critical reinforcement learning agents perform well in novel situations.
problem Ensuring safety-critical reinforcement learning agents perform well in unseen test environments.
method Investigates safety and generalization from a limited number of training environments, using gridworld and CoinRun settings.
result Simple modifications like ensemble model averaging and blocking classifier can reduce catastrophes in gridworld, but not in CoinRun.
Combines BC and GAIL for efficient imitation learning.
problem Efficient imitation learning without reward signals.
method Integrates Behavior Cloning and Generative Adversarial Imitation Learning.
result Combination leads to stable and sample-efficient learning.
ISA learns subgoals for reinforcement learning agents.
problem Learning subgoals for efficient reinforcement learning.
method Induces a subgoal automaton from observation traces using inductive logic programming.
result ISA learns subgoals that improve RL performance and convergence.
ADVISOR dynamically balances imitation and reinforcement learning to overcome the imitation gap.
problem The gap between imitation learning and reinforcement learning when teaching agents have privileged information.
method Adaptive Insubordination (ADVISOR) dynamically weights imitation and reward-based reinforcement learning losses.
result On-the-fly switching with ADVISOR outperforms pure imitation, pure reinforcement learning, and their combinations.
This paper considers Safe Policy Improvement (SPI) in Batch Reinforcement Learning (Batch RL): from a fixed dataset and without direct access to the true environment, train a policy that is guaranteed to perform at least as well as the baseline policy used to collect the data. Our approach, called SPI with Baseline Boo…
OtoWorld helps agents learn to navigate by listening in interactive environments.
problem Training agents to navigate using auditory information.
method Interactive environment with agents learning to listen and navigate.
result Agents can win at OtoWorld, a navigation game with sound sources.
This paper introduces sample-averaged Q-learning for better RL performance.
problem Improving reinforcement learning algorithms by managing uncertainty.
method Integrates statistical inference into Q-learning through sample averaging and functional central limit theorem.
result Establishes a unified theoretical foundation for sample-averaged Q-learning.
New method learns high-quality Laplacian representations for reinforcement learning.
problem Lack of accurate Laplacian representations in large or continuous state spaces.
method Reformulated spectral graph drawing objective to have eigenvectors as unique global minimizer.
result Learned Laplacian representations more faithfully approximate the ground truth.
Paper proposes a novel RRL framework that learns from images and incorporates expert knowledge.
problem Lack of effective methods to incorporate expert background knowledge and learn from non-relational data in RRL.
method Differentiable Inductive Logic Programming (ILP) for learning relational information from images and incorporating expert knowledge.
result Efficacy demonstrated on various environments and datasets, showing improved learning and generalization.
The paper introduces a method to learn Markov state abstractions for reinforcement learning.
problem Learning Markov state representations in complex environments.
method The paper introduces a novel set of conditions and a training procedure combining inverse model estimation and temporal contrastive learning.
result The approach learns representations that capture the underlying structure of the domain and improve sample efficiency.
The paper adapts step sizes in TD learning to identify relevant features.
problem Identifying which features are relevant for temporal-difference learning.
method Adapting step sizes in stochastic gradient descent for feature relevance in TD learning.
result TD IDBD effectively distinguishes relevant features in gridworld and robotic tasks.
This project proposes using reinforcement learning to train spiking neural networks.
problem Training spiking neural networks using traditional methods is challenging due to the discrete nature of spikes.
method The project investigates two approaches: 1) treating each neuron as an RL agent, 2) applying the reparameterization trick.
result The project demonstrates that reinforcement learning can be applied to train spiking neural networks.
Proves error bounds for state representation in RL using graph spectral features.
problem Addressing the curse of dimensionality in RL with unknown transition graphs.
method Proves upper bounds on approximation error of linear value function approximation using learned spectral features of the state-graph.
result Error bounds scale with algebraic connectivity and eigenvector estimation error.
Platform combines RL and language models to study narrative influence on AI decisions.
problem Understanding how narrative elements shape AI decision-making.
method Dual-system architecture with reinforcement learning and language model integration.
result Initial experiments show narrative frameworks can influence AI decision-making.
A new method for student-initiated action advice using novelty detection.
problem Exploration and sample inefficiency in RL, especially with teacher absence.
method Random Network Distillation (RND) to measure advice novelty, updates only for advised states.
result Significant performance improvement over state-of-the-art methods, especially in challenging scenarios.
This paper interpolates reward functions to predict optimal value functions in MORL.
problem Finding optimal value functions in MORL requires recomputing for each set of weights.
method Interpolating reward function weights to smooth value function transformations.
result Smooth interpolation of optimal value functions over reward function weights.