GENE tackles sparse reward in RL by generating states to explore and exploit.
problem Sparse reward in reinforcement learning.
method Generative Exploration and Exploitation (GENE) method.
result GENE significantly outperforms existing methods in tasks with binary rewards.
HRM-Agent learns to navigate dynamic mazes using reinforcement learning.
problem Training HRM in dynamic, uncertain, partially observable environments.
method Reinforcement learning to train HRM-Agent.
result HRM-Agent successfully learns to navigate dynamic mazes.
AutoDIME automates design of multi-agent environments for RL.
problem Designing multi-agent environments for reinforcement learning is challenging.
method Developed intrinsic teacher rewards for multi-agent settings and evaluated them in various tasks.
result Value disagreement was found to be most consistent and effective across tasks.
Enactive learning shows agents can learn from their environment, but limited by action choices.
problem Learning and interaction of autonomous agents in complex environments.
method Simulation of artificial agents in maze environments, comparing enactive learning to classical reinforcement learning.
result Enactive agents can learn to avoid unfavorable interactions but performance is limited by action choices.
This paper presents a problem of model learning for the purpose of learning how to navigate a ball to a goal state in a circular maze environment with two degrees of freedom. The motion of the ball in the maze environment is influenced by several non-linear effects such as dry friction and contacts, which are difficult…
PBCS combines RL and motion planning for better exploration.
problem RL algorithms struggle with versatile exploration in complex environments.
method PBCS uses motion planning to find a good trajectory, then trains RL on a curriculum derived from it.
result PBCS outperforms state-of-the-art RL algorithms in 2D maze environments.
This paper introduces a new approach to active inference using constrained Bethe Free Energy.
problem Tackling the limitations of existing epistemic behavior models in active inference.
method Introducing a constrained Bethe Free Energy (CBFE) perspective to optimize epistemic behavior in generative models.
result CBFE optimization leads to more robust and flexible epistemic behavior compared to existing methods.
A graph abstraction speeds up reinforcement learning in complex environments.
problem Learning hierarchical reinforcement learning tasks in complex environments.
method Jointly trains a latent pivotal state model and a curiosity-driven policy. Uses a world graph to guide high-level and low-level agents.
result Significant performance and efficiency improvements over baseline methods.
The ability to use a 2D map to navigate a complex 3D environment is quite remarkable, and even difficult for many humans. Localization and navigation is also an important problem in domains such as robotics, and has recently become a focus of the deep reinforcement learning community. In this paper we teach a reinforce…
Models that can simulate how environments change in response to actions can be used by agents to plan and act efficiently. We improve on previous environment simulators from high-dimensional pixel observations by introducing recurrent neural networks that are able to make temporally and spatially coherent predictions f…
A simple DQN-based multi-agent RL system for binary actions.
problem Complexity and training issues in multi-agent reinforcement learning.
method Shared state and rewards, agent-specific actions, experience replay pool.
result Better performance and faster convergence compared to conventional methods.
MB-DQN uses different backup lengths for improved reinforcement learning.
problem Improving reinforcement learning with multi-step returns.
method Integrates multi-step returns into bootstrapped DQN with different backup lengths.
result MB-DQN maintains advantages of different backup lengths.
Value Iteration Networks (VINs) are effective differentiable path planning modules that can be used by agents to perform navigation while still maintaining end-to-end differentiability of the entire architecture. Despite their effectiveness, they suffer from several disadvantages including training instability, random …
Surprise describes a range of phenomena from unexpected events to behavioral responses. We propose a measure of surprise and use it for surprise-driven learning. Our surprise measure takes into account data likelihood as well as the degree of commitment to a belief via the entropy of the belief distribution. We find th…
New algorithm minimizes FE objectives for synthetic AIF agents.
problem Minimizing Free Energy for synthetic active inference agents.
method Variational message passing on Constrained FFGs.
result Message passing algorithms induce epistemic behavior in AIF agents.
Deep network solves maze path planning without training.
problem Efficient path planning in large mazes with obstacles.
method Max pooling layers without training.
result Solves mazes with over half a billion nodes in short time.
MAZE attacks a model without data, using synthetic inputs.
problem Model Stealing attacks that require data to replicate a target model.
method MAZE uses zeroth-order gradient estimation to create synthetic inputs.
result MAZE achieves high clone accuracy without data.
We propose an expert-augmented actor-critic algorithm, which we evaluate on two environments with sparse rewards: Montezumas Revenge and a demanding maze from the ViZDoom suite. In the case of Montezumas Revenge, an agent trained with our method achieves very good results consistently scoring above 27,000 points (in ma…
Recent developments in deep reinforcement learning have enabled the creation of agents for solving a large variety of games given a visual input. These methods have been proven successful for 2D games, like the Atari games, or for simple tasks, like navigating in mazes. It is still an open question, how to address more…
Trajectory-based learning speeds up deep RL in Ball-in-Maze games.
problem Lack of sample efficiency in deep reinforcement learning.
method Use of trajectories generated by a simulator to improve learning speed.
result Achieved a 2-3x speed-up in learning process for Ball-in-Maze games.
Convolutional networks improve reinforcement learning for navigating all goals.
problem Expensive parallel updates limit reinforcement learning to small tabular cases.
method Use convolutional neural networks to generate Q-values and updates for all goals simultaneously.
result Demonstrated improved accuracy and generalization on various environments.
SMiRL learns to minimize surprise in unstable environments, improving agent performance.
problem Learning useful behaviors in unpredictable, unstable environments.
method Alternates between learning a density model and improving policy to seek more predictable stimuli.
result SMiRL agents can play games, control robots, and navigate mazes without task-specific rewards.
Neural architectures learn belief representations for partially observable environments.
problem Learning belief states in partially observable domains.
method One-step frame prediction and contrastive predictive coding (CPC) as objective functions.
result Neural architectures can learn belief representations, encoding both state information and uncertainty.
Director learns hierarchical behaviors from pixels, outperforming exploration methods.
problem Challenges in learning long-term hierarchical behaviors from pixels.
method Directly learns hierarchical behaviors from pixels using a latent space world model.
result Director outperforms exploration methods on various tasks.
Improves deep RL agents' generalization to unseen environments.
problem Deep RL agents struggle to adapt to new environments.
method Information Bottleneck regularization and annealing-based optimization.
result Agents can generalize to test parameters more than 10 standard deviations away from training.
Logical neural networks solve mazes by filling dead ends, but not all methods generalize well.
problem Understanding how logical neural networks extrapolate solutions to mazes.
method Examined recurrent and implicit neural networks trained on maze-solving tasks.
result Models fail to generalize well to diverse maze sizes, suggesting limitations in learning scalable algorithms.
GNMC reduces XCSF population size while preserving function approximation and policy accuracy.
problem Population bloat in XCSF compaction.
method Introduced GNMC, a novel compaction algorithm.
result GNMC reduces population size significantly without compromising function approximation or policy accuracy.
Paper tackles continual reinforcement learning challenges with diversity exploration and adversarial self-correction.
problem Challenges in learning new tasks sequentially due to catastrophic forgetting.
method Develops an end-to-end framework (CDAN) combining unsupervised diversity exploration and adversarial self-correction.
result Final result outperforms baseline by 18.35% in NSD and 0.61 in average reward.
Equivariant MuZero improves generalization in procedurally generated environments.
problem Leveraging world models for robust generalization in real-world scenarios.
method Incorporates symmetries of the environment into MuZero's world-model architecture, making MuZero equivariant to certain groups of transformations.
result Improves data efficiency and generalization capabilities, demonstrated on procedurally generated MiniPacman and Chaser.
The paper proposes a structure learning model for efficient reinforcement learning.
problem Efficient sample-based decision making in complex environments.
method Temporal Gaussian Mixture Model with structure learning.
result The model discovers the number of states and transition probabilities.
In this paper we approach two relevant deep learning topics: i) tackling of graph structured input data and ii) a better understanding and analysis of deep networks and related learning algorithms. With this in mind we focus on the topological classification of reachability in a particular subset of planar graphs (Maze…
Reinforcement Learning (RL) is a research area that has blossomed tremendously in recent years and has shown remarkable potential for artificial intelligence based opponents in computer games. This success is primarily due to vast capabilities of Convolutional Neural Networks (ConvNet), enabling algorithms to extract u…
Novelty search in low-dimensional space improves sample efficiency in exploration tasks.
problem Efficient exploration in complex environments with sparse rewards.
method Combines model-based and model-free objectives to learn a low-dimensional representation. Uses intrinsic novelty rewards based on nearest neighbor distances in this space.
result Our approach achieves more sample-efficient exploration compared to strong baselines on various tasks.
New RL approach builds short ancestral recombination graphs.
problem Building short ancestral recombination graphs (ARGs).
method Reinforcement Learning applied to genetic sequences.
result RL can build ARGs as short as heuristic algorithms.
CSAC enables cooperative reinforcement learning for multi-stage tasks.
problem Coordinating consecutive reinforcement learning agents for long-term multi-stage tasks.
method CSAC modifies each agent's policy to maximize both current and next agent's critic.
result CSAC outperforms uncooperative policies and single-agent training in multi-room maze domain.
Bayesian framework inserts prior knowledge in RL for faster task solving.
problem Faster transfer learning across reinforcement learning tasks.
method Bayesian posterior distribution combining task-specific and prior knowledge.
result Significant speed ups achieved in maze solving.
This paper proposes a new method to automatically learn optimal return functions in reinforcement learning.
problem Learning optimal policies in reinforcement learning can be slow and inefficient.
method The authors propose a general mathematical form for the return function and use meta-learning to automatically learn the optimal form.
result Their method significantly speeds up the learning of optimal policies in reinforcement learning.
ERM uses energy-based selection to improve recursive reasoning.
problem Lack of principled inference mechanism in recursive models.
method Energy-guided Recursive Model (ERM) introduces Hopfield energies for trajectory selection.
result ERM achieves optimal solutions on various puzzles.
The Morris Water Maze is commonly used in behavioural neuroscience for the study of spatial learning with rodents. Over the years, various methods of analysing rodent data collected in this task have been proposed. These methods span from classical performance measurements (e.g. escape latency, rodent speed, quadrant p…
A method using competitive experience replay enhances learning from sparse rewards.
problem Learning from sparse rewards in reinforcement learning.
method Competitive experience replay method that augments sparse rewards through an exploration competition between agents.
result The method leads to faster convergence and improved task performance.
Learning robot tasks or controllers using deep reinforcement learning has been proven effective in simulations. Learning in simulation has several advantages. For example, one can fully control the simulated environment, including halting motions while performing computations. Another advantage when robots are involved…
A new neural model evolves to learn at the synaptic level.
problem Lack of biologically realistic neural models in deep learning.
method Evolve individual neuron and synaptic models using ENUs.
result Evolved neural network learns complex tasks like a T-maze.
Wave network efficiently propagates long-range information in undirected graphs.
problem Efficiently propagating long-range information in undirected graphs.
method Propagates information in waves of nonlinear computation.
result Wave outperforms graph convolution on three graph-based tasks.
Abstract perspective on quadratic programming for optimal portfolio allocation.
problem Optimal allocation problems in long portfolio theory.
method Using maximum principles and distinguished boundaries in reproducing kernel Hilbert spaces.
result Support of an optimal distribution lies in a variety intersecting a distinguished boundary.
We study the problem of learning a good search policy for combinatorial search spaces. We propose retrospective imitation learning, which, after initial training by an expert, improves itself by learning from \textit{retrospective inspections} of its own roll-outs. That is, when the policy eventually reaches a feasible…
Switching linear dynamics improves model-based reinforcement learning and system identification.
problem Complex and nonlinear systems can be approximated by linear dynamical systems.
method Bayesian inference, Variational Autoencoders, Concrete relaxations.
result Improved accuracy in learning dynamics from partial and high-dimensional observations.
Neural network uses local credit assignment to learn distant cause-effect relationships.
problem Learning distant cause-effect relationships in sequential data.
method Sparse coding in a recurrent neural network memory with local and immediate credit assignment.
result Network can predict partially-observable higher-order sequences and navigate mazes.
This paper improves reinforcement learning policies in a scalable way.
problem Ensuring monotonic policy improvement in entropy-regularized RL.
method Derives an entropy-aware lower bound and proposes a novel RL algorithm.
result Demonstrates effectiveness in continuous-state tasks using a linear function approximator.