Graph-based state representation improves deep RL performance.
problem High sample-complexity and starting with a good input representation in deep RL.
method Exploiting the graph structure of MDPs for effective state representation learning.
result Graph-based node representation methods outperform matrix-based methods in grid-world navigation tasks.
The paper studies how grid cell patterns emerge in neural networks.
problem Understanding how grid cells in the brain form hexagonal firing patterns.
method Training recurrent neural networks with conformal normalization of velocity inputs.
result Conformal normalization is crucial for the emergence of hexagonal grid patterns in neural networks.
HRL4IN tackles interactive navigation tasks with mobile manipulators, improving efficiency and performance.
problem Interactive Navigation tasks require mobile manipulators to perform various actions, but choosing the right part of the embodiment is inefficient.
method HRL4IN uses a hierarchical reinforcement learning architecture to handle heterogeneous phases of navigation and manipulation, selecting the appropriate part of the embodiment for each phase.
result HRL4IN significantly outperforms flat PPO and HAC in terms of task performance and energy efficiency.
Paper proposes PRR network for better experience reuse in reinforcement learning.
problem Efficient experience reuse in reinforcement learning across multiple granularities.
method Proposes PRR network trained on multi-level architecture to extract and store experience.
result PRR network leads to better experience reuse and improved performance.
Decades of research on the neural code underlying spatial navigation have revealed a diverse set of neural response properties. The Entorhinal Cortex (EC) of the mammalian brain contains a rich set of spatial correlates, including grid cells which encode space using tessellating patterns. However, the mechanisms and fu…
This paper proposes grid cells encode position via a conformal isometric embedding of 2D physical space.
problem Hexagonal grid firing patterns in grid cells.
method Learning a distance-preserving position embedding in neural space using a recurrent neural network.
result The conformal isometric embedding of 2D physical space into neural space explains hexagonal grid firing patterns.
Develops a new neural spike train decoding framework using topological data.
problem Decoding neural spike trains from head direction and grid cells.
method Combines simplicial complex discovery with deep learning to capture higher-order connectivity.
result Demonstrates effectiveness on head direction and trajectory prediction datasets.
The options framework in reinforcement learning models the notion of a skill or a temporally extended sequence of actions. The discovery of a reusable set of skills has typically entailed building options, that navigate to bottleneck states. This work adopts a complementary approach, where we attempt to discover option…
Framework discovers sub-goals for better exploration in RL tasks.
problem Improving exploration in sequential decision making under partial observability.
method Variational intrinsic control framework with information theoretic regularizer.
result Sub-goals discovered without explicit supervision lead to better exploration and sample efficiency.
Machine learning has been widely applied to various applications, some of which involve training with privacy-sensitive data. A modest number of data breaches have been studied, including credit card information in natural language data and identities from face dataset. However, most of these studies focus on supervise…
Projective simulation (PS) is a model for intelligent agents with a deliberation capacity that is based on episodic memory. The model has been shown to provide a flexible framework for constructing reinforcement-learning agents, and it allows for quantum mechanical generalization, which leads to a speed-up in deliberat…
Learning in sparse reward settings remains a challenge in Reinforcement Learning, which is often addressed by using intrinsic rewards. One promising strategy is inspired by human curiosity, requiring the agent to learn to predict the future. In this paper a curiosity-driven agent is extended to use these predictions di…
A new algorithm finds optimal solutions for constrained decision processes.
problem Optimizing state-value functions with constraints in CMDPs.
method Gradient-Aware Search (GAS) exploiting PWLC structure.
result GAS converges faster and more reliably than existing methods.
ReCoRe learns invariant features for world navigation using contrastive learning and regularizers.
problem Limited sample efficiency and overfitting to training scenarios in RL for visual navigation.
method Contrastive unsupervised learning and intervention-invariant regularizer.
result Significantly improves sample efficiency and generalization in out-of-distribution point navigation tasks.
A new method learns continuous occupancy fields efficiently using the Ising model.
problem Efficiently learning continuous occupancy fields for robot navigation.
method Generalizes the Ising model to model continuous occupancy fields, using a unique kernel for range measurements.
result The method is computationally efficient and requires only a small number of hyperparameters.
Consider an assistive system that guides visually impaired users through speech and haptic feedback to their destination. Existing robotic and ubiquitous navigation technologies (e.g., portable, ground, or wearable systems) often operate in a generic, user-agnostic manner. However, to minimize confusion and navigation …
SOM-VQ tokenizes discrete models with semantic structure and navigable topology.
problem Lack of semantic structure in vector quantized representations limits interpretable human control.
method Combines vector quantization with Self-Organizing Maps to learn discrete codebooks with explicit topology.
result SOM-VQ produces more learnable token sequences and provides an explicit navigable geometry in code space.
Jelly Bean World tests never-ending learning in complex environments.
problem Current machine learning systems are specialized and limited in adaptability.
method Developed Jelly Bean World testbed for complex, non-stationary environments.
result Demonstrates the need for more adaptable machine learning systems.
Bayesian model for energy consumption helps electric vehicles navigate efficiently.
problem Limited battery capacity in electric vehicles makes energy efficient navigation challenging.
method Developed an online learning framework using Bayesian models and exploration strategies like Thompson Sampling.
result Established rigorous regret bounds for Thompson Sampling in both single-agent and multi-agent settings.
A novel method visualizes higher-dimensional spaces using hyperbolic geometry.
problem Challenges in visualizing higher-dimensional spaces.
method Interactive visualization of higher-dimensional grids based on hyperbolic geometry.
result Our method shows the whole higher-dimensional space at once and avoids disadvantages of previous methods.
Bayesian model for energy-efficient EV navigation.
problem Limited battery capacity in electric vehicles.
method Bayesian modeling and online learning framework with exploration strategies.
result Established rigorous regret bounds for Thompson Sampling.
SLAM-net learns to navigate visually in challenging indoor environments.
problem Challenges in SLAM for visual robot navigation, especially in noisy conditions.
method Differentiable SLAM Network (SLAM-net) that encodes a particle filter SLAM algorithm in a differentiable graph and learns components through backpropagation.
result Significantly outperforms ORB-SLAM in noisy conditions and improves the Habitat Challenge 2020 PointNav task.
We investigate the multi-step prediction of the drivable space, represented by Occupancy Grid Maps (OGMs), for autonomous vehicles. Our motivation is that accurate multi-step prediction of the drivable space can efficiently improve path planning and navigation resulting in safe, comfortable and optimum paths in autonom…
OtoWorld helps agents learn to navigate by listening in interactive environments.
problem Training agents to navigate using auditory information.
method Interactive environment with agents learning to listen and navigate.
result Agents can win at OtoWorld, a navigation game with sound sources.
Capsule Networks improve autonomous navigation in sparse environments.
problem Challenges in reinforcement learning for sparse reward environments.
method Caps-EM pairs CapsNets with Advantage Actor Critic, using fewer parameters.
result Caps-EM achieves significant time improvements and fewer parameters compared to competing methods.
Paper introduces timing-based adversarial attacks on DRL-based navigation systems.
problem Vulnerability of DRL-based navigation systems to adversarial attacks.
method Timing-based adversarial strategies using physical noise patterns.
result Adversarial timing attacks significantly degrade DRL-based navigation performance.
Improves AI agents' 3D navigation by learning from failures and 3D spatial relationships.
problem Challenges in data efficiency, obstacle avoidance, and generalization in 3D visual navigation.
method Incorporates attention on 3D spatial relationships and a target skill extension module into DRL framework.
result Significantly improves navigation performance and generalization across targets and scenes.
This paper improves MADDPG's performance in discrete grid-world scenarios.
problem MADDPG struggles with discrete action spaces in grid-world tasks.
method Integrates alternative Gumbel-Softmax estimators into MADDPG.
result One alternative estimator significantly improves performance in grid-world tasks.
New algorithms solve time-dependent navigation problems, including zig-zag paths.
problem Finding optimal paths in moving water with varying speed profiles.
method Lorentz-Finsler geometry, novel computational algorithms.
result Optimal paths may involve zig-zag trajectories due to tacking behavior.
H-STGCN predicts traffic using navigation data and improves accuracy.
problem Limited accuracy in traffic forecasting due to lack of contextual information.
method Proposes H-STGCN, a hybrid spatio-temporal graph convolutional network.
result H-STGCN outperforms state-of-the-art methods in various metrics, especially for non-recurring congestion.
Power grids are one of the most important components of infrastructure in today's world. Every nation is dependent on the security and stability of its own power grid to provide electricity to the households and industries. A malfunction of even a small part of a power grid can cause loss of productivity, revenue and i…
Prototype real-world RL environment for robotics training.
problem Lack of common real-world RL benchmark.
method Developed OffWorld Gym with open access to real-world robotics environments.
result Baseline results in navigation task on real and simulated terrain.
AC-State discovers minimal latent state for control.
problem Discover minimal latent state from sensory information.
method Multi-step inverse model with information bottleneck.
result Guaranteed discovery of control-endogenous latent states.
Deep learning has revolutionized the ability to learn "end-to-end" autonomous vehicle control directly from raw sensory data. While there have been recent extensions to handle forms of navigation instruction, these works are unable to capture the full distribution of possible actions that could be taken and to reason a…
Transfer learning significantly accelerates the reinforcement learning process by exploiting relevant knowledge from previous experiences. The problem of optimally selecting source policies during the learning process is of great importance yet challenging. There has been little theoretical analysis of this problem. In…
WebGUM learns web navigation from multimodal data, outperforming previous methods.
problem Limited generalization from domain-specific models in web navigation.
method Instruction-following multimodal agent trained on vision-language foundation models.
result Significant improvement in web navigation performance on benchmarks.
Proposes a new convolutional neural network for non-grid data.
problem Limited applicability of standard CNNs to non-grid structured data.
method Introduces Parametric Continuous Convolution (PCC) with learnable kernel functions.
result Significant improvement in point cloud segmentation and lidar motion estimation.
For low-dimensional data sets with a large amount of data points, standard kernel methods are usually not feasible for regression anymore. Besides simple linear models or involved heuristic deep learning models, grid-based discretizations of larger (kernel) model classes lead to algorithms, which naturally scale linear…
Power system studies require the topological structures of real-world power networks; however, such data is confidential due to important security concerns. Thus, power grid synthesis (PGS), i.e., creating realistic power grids that imitate actual power networks, has gained significant attention. In this letter, we cas…
ProbETA models travel time correlations between trips for better navigation.
problem Trip correlations not captured by existing methods.
method Deep hierarchical joint probabilistic model with learnable link representations.
result ProbETA outperforms state-of-the-art methods with 12.60% MAPE reduction.
We present Vision-based Navigation with Language-based Assistance (VNLA), a grounded vision-language task where an agent with visual perception is guided via language to find objects in photorealistic indoor environments. The task emulates a real-world scenario in that (a) the requester may not know how to navigate to …
Paper proposes a method to locate power grid recordings using ENF sequences.
problem Locating power grid recordings without concurrent power signals.
method Extract ENF sequences from power and audio recordings, develop multi-class SVM model.
result Validation of location authenticity of recordings using ENF sequences.
HRM-Agent learns to navigate dynamic mazes using reinforcement learning.
problem Training HRM in dynamic, uncertain, partially observable environments.
method Reinforcement learning to train HRM-Agent.
result HRM-Agent successfully learns to navigate dynamic mazes.
Helps visually impaired users make better decisions by adjusting their observations.
problem Systematic biases in users' perception and processing of visual information.
method Synthesizes new observations based on true observations to correct user biases.
result Significant improvement in task performance for users with various biases.
Deep Sets improve reinforcement learning agent's object-centered navigation and generalization.
problem Improving reinforcement learning agents' ability to generalize to unseen objects and goals.
method Combining object-wise permutation invariant networks (Deep Sets) and gated-attention mechanisms.
result Agent demonstrates strong generalization to out-of-distribution goals in a procedurally-generated 2D world.
Curiosity-Critic improves world model training by focusing on cumulative prediction error.
problem Training world models with intrinsic rewards that consider cumulative prediction error.
method Curiosity-Critic uses a surrogate reward based on the difference between current and asymptotic prediction errors, estimated online by a co-trained critic.
result Curiosity-Critic outperforms other methods in training speed and final world model accuracy.
Improves RL planning by proposing sub-goals hierarchically.
problem Sequential planning assumption in RL.
method Divide-and-Conquer Monte Carlo Tree Search (DC-MCTS).
result Improves navigation and control tasks.
A new RL model ensures safe learning in uncertain environments.
problem Safe reinforcement learning in uncertain, partially observable environments.
method Lyapunov-based uncertainty quantification and Transformers for memory.
result Significant improvement in safety and optimality in grid-world tasks.