Graph-based state representation improves deep RL performance.
problem High sample-complexity and starting with a good input representation in deep RL.
method Exploiting the graph structure of MDPs for effective state representation learning.
result Graph-based node representation methods outperform matrix-based methods in grid-world navigation tasks.
Most common navigation tasks in human environments require auxiliary arm interactions, e.g. opening doors, pressing buttons and pushing obstacles away. This type of navigation tasks, which we call Interactive Navigation, requires the use of mobile manipulators: mobile bases with manipulation capabilities. Interactive N…
We propose a novel framework to identify sub-goals useful for exploration in sequential decision making tasks under partial observability. We utilize the variational intrinsic control framework (Gregor et.al., 2016) which maximizes empowerment -- the ability to reliably reach a diverse set of states and show how to ide…
The options framework in reinforcement learning models the notion of a skill or a temporally extended sequence of actions. The discovery of a reusable set of skills has typically entailed building options, that navigate to bottleneck states. This work adopts a complementary approach, where we attempt to discover option…
Projective simulation (PS) is a model for intelligent agents with a deliberation capacity that is based on episodic memory. The model has been shown to provide a flexible framework for constructing reinforcement-learning agents, and it allows for quantum mechanical generalization, which leads to a speed-up in deliberat…
Learning in sparse reward settings remains a challenge in Reinforcement Learning, which is often addressed by using intrinsic rewards. One promising strategy is inspired by human curiosity, requiring the agent to learn to predict the future. In this paper a curiosity-driven agent is extended to use these predictions di…
A new algorithm finds optimal solutions for constrained decision processes.
problem Optimizing state-value functions with constraints in CMDPs.
method Gradient-Aware Search (GAS) exploiting PWLC structure.
result GAS converges faster and more reliably than existing methods.
Machine learning has been widely applied to various applications, some of which involve training with privacy-sensitive data. A modest number of data breaches have been studied, including credit card information in natural language data and identities from face dataset. However, most of these studies focus on supervise…
This paper improves MADDPG's performance in discrete grid-world scenarios.
problem MADDPG struggles with discrete action spaces in grid-world tasks.
method Integrates alternative Gumbel-Softmax estimators into MADDPG.
result One alternative estimator significantly improves performance in grid-world tasks.
Experience reuse is key to sample-efficient reinforcement learning. One of the critical issues is how the experience is represented and stored. Previously, the experience can be stored in the forms of features, individual models, and the average model, each lying at a different granularity. However, new tasks may requi…
Improves RL planning by proposing sub-goals hierarchically.
problem Sequential planning assumption in RL.
method Divide-and-Conquer Monte Carlo Tree Search (DC-MCTS).
result Improves navigation and control tasks.
Jelly Bean World tests never-ending learning in complex environments.
problem Current machine learning systems are specialized and limited in adaptability.
method Developed Jelly Bean World testbed for complex, non-stationary environments.
result Demonstrates the need for more adaptable machine learning systems.
Paper tackles reinforcement learning generalization through invariant policy optimization.
problem Learning policies that generalize beyond training domains.
method Invariant policy optimization principle and novel learning algorithm IPO.
result Significant improvements in generalization performance on unseen domains.
The paper introduces MDP homomorphic networks for faster reinforcement learning.
problem Current reinforcement learning approaches do not exploit symmetries in the joint state-action space.
method Equivariant neural networks with group-structured symmetries (reflections, rotations).
result MDP homomorphic networks converge faster than unstructured baselines on various tasks.
Introduces a natural parallel translation for navigation data.
problem Navigation data geometric representation and parallelism.
method Introduces a natural parallel translation using Riemannian parallelism.
result The natural parallel translation preserves the Randers norm and has a finite-dimensional holonomy group.
We designed a grid world task to study human planning and re-planning behavior in an unknown stochastic environment. In our grid world, participants were asked to travel from a random starting point to a random goal position while maximizing their reward. Because they were not familiar with the environment, they needed…
Resolves spurious correlations in causal models via intervention design.
problem Spurious correlations lead to incorrect causal models in reinforcement learning environments.
method Proposes a method to design interventions that improve causal models by incentivizing agents to find errors.
result Experimental results show improved causal models compared to baselines.
Challenge to separate Earth's magnetic field from vehicle's magnetic field for accurate navigation.
problem Separate Earth's magnetic field from vehicle's magnetic field for accurate magnetic navigation.
method Use machine learning (ML) and integrate physics of magnetic navigation (SciML) to remove aircraft magnetic field from total magnetic field.
result A model can be constructed to effectively remove aircraft magnetic field from the dataset.
Navigation in Lorentz Finsler geometry induces isoparametric hypersurfaces.
problem Defining and analyzing isoparametric hypersurfaces in Lorentz Finsler geometry.
method Using a navigation process with a Finsler metric and a tangent vector field, isoparametric functions and hypersurfaces are defined and analyzed.
result Local correspondences between isoparametric functions and hypersurfaces are established.
The paper solves navigation problems on conic Kropina manifolds and establishes curvature relationships.
problem Navigation problems on conic Kropina manifolds.
method Analyzes the solution of navigation problems and establishes curvature relationships.
result The solution to navigation problems on conic Kropina manifolds must be either a Randers metric or a Kropina metric.
Robotic navigation improves with RL and ultrasound images.
problem Improving accuracy in robotic navigation for ultrasound-guided procedures.
method Combines deep Q-networks with ultrasound images and binary classifier.
result Significantly improved navigation accuracy compared to pure RL and SL.
Improved robot navigation using multi-head attention for natural language instructions.
problem Improving robot navigation in unfamiliar environments.
method Proposes a multi-head attention mechanism blending layer in a neural network model.
result Significant performance gains in translating instructions for unseen environments.
The problem of pursuing a moving target is always one of the main topics in navigation. In the literatures, there are two well-known algorithms called Pure Pursuit and Pure Rendezvous navigation in the 3-dimensional space R3. In this paper, these two methods are combined to introduce a novel family of pursu…
Mobile robot navigation in complex and dynamic environments is a challenging but important problem. Reinforcement learning approaches fail to solve these tasks efficiently due to reward sparsities, temporal complexities and high-dimensionality of sensorimotor spaces which are inherent in such problems. We present a nov…
SLAM-net learns to navigate visually in challenging indoor environments.
problem Challenges in SLAM for visual robot navigation, especially in noisy conditions.
method Differentiable SLAM Network (SLAM-net) that encodes a particle filter SLAM algorithm in a differentiable graph and learns components through backpropagation.
result Significantly outperforms ORB-SLAM in noisy conditions and improves the Habitat Challenge 2020 PointNav task.
Improves AI agents' 3D navigation by learning from failures and 3D spatial relationships.
problem Challenges in data efficiency, obstacle avoidance, and generalization in 3D visual navigation.
method Incorporates attention on 3D spatial relationships and a target skill extension module into DRL framework.
result Significantly improves navigation performance and generalization across targets and scenes.
Bayesian model for energy consumption helps electric vehicles navigate efficiently.
problem Limited battery capacity in electric vehicles makes energy efficient navigation challenging.
method Developed an online learning framework using Bayesian models and exploration strategies like Thompson Sampling.
result Established rigorous regret bounds for Thompson Sampling in both single-agent and multi-agent settings.
This work analyzes minimum-time navigation on Riemannian manifolds using Finsler geometry.
problem Minimum-time navigation on Riemannian manifolds.
method Finsler geometry, introducing superwind concept, extending (α,β)-metrics. result Time-minimizing geodesics and necessary/sufficient conditions for strong convexity.
A new RL model ensures safe learning in uncertain environments.
problem Safe reinforcement learning in uncertain, partially observable environments.
method Lyapunov-based uncertainty quantification and Transformers for memory.
result Significant improvement in safety and optimality in grid-world tasks.
Deep learning agent improves pedestrian navigation in urban environments.
problem Autonomous driving among pedestrians in urban areas.
method Multi-objective deep reinforcement learning using a deep Q-learning variant.
result The multi-objective DQN agent outperforms single-objective DQN in various environments.
Bayesian model for energy-efficient EV navigation.
problem Limited battery capacity in electric vehicles.
method Bayesian modeling and online learning framework with exploration strategies.
result Established rigorous regret bounds for Thompson Sampling.
Solves time-minimizing navigation on a mountain slope using Riemann-Finsler geometry.
problem Time-minimizing navigation on a mountain slope under gravity.
method Riemann-Finsler geometry, Zermelo navigation problem, anisotropic deformation of the background Riemannian metric, rescaled gravitational wind.
result A new Finsler metric for optimal navigation on slippery mountain slopes.
Deep reinforcement learning (RL) has been successfully applied to a variety of game-like environments. However, the application of deep RL to visual navigation with realistic environments is a challenging task. We propose a novel learning architecture capable of navigating an agent, e.g. a mobile robot, to a target giv…
The paper develops a method to learn navigation costs from expert demonstrations in partially observable environments.
problem Learning navigation costs from expert demonstrations in partially observable environments.
method Develops a cost function representation composed of a probabilistic occupancy encoder and a cost encoder, optimized by differentiating the error between demonstrated controls and a control policy computed from the cost encoder.
result The method outperforms baseline IRL algorithms in robot navigation tasks, improving both training and test-time efficiency.
New algorithms solve time-dependent navigation problems, including zig-zag paths.
problem Finding optimal paths in moving water with varying speed profiles.
method Lorentz-Finsler geometry, novel computational algorithms.
result Optimal paths may involve zig-zag trajectories due to tacking behavior.
VALAN is a lightweight and scalable software framework for deep reinforcement learning based on the SEED RL architecture. The framework facilitates the development and evaluation of embodied agents for solving grounded language understanding tasks, such as Vision-and-Language Navigation and Vision-and-Dialog Navigation…
OtoWorld helps agents learn to navigate by listening in interactive environments.
problem Training agents to navigate using auditory information.
method Interactive environment with agents learning to listen and navigate.
result Agents can win at OtoWorld, a navigation game with sound sources.
Paper introduces timing-based adversarial attacks on DRL-based navigation systems.
problem Vulnerability of DRL-based navigation systems to adversarial attacks.
method Timing-based adversarial strategies using physical noise patterns.
result Adversarial timing attacks significantly degrade DRL-based navigation performance.
We consider a setting in which the objective is to learn to navigate in a controlled Markov process (CMP) where transition probabilities may abruptly change. For this setting, we propose a performance measure called exploration steps which counts the time steps at which the learner lacks sufficient knowledge to navigat…
Curiosity-Critic improves world model training by focusing on cumulative prediction error.
problem Training world models with intrinsic rewards that consider cumulative prediction error.
method Curiosity-Critic uses a surrogate reward based on the difference between current and asymptotic prediction errors, estimated online by a co-trained critic.
result Curiosity-Critic outperforms other methods in training speed and final world model accuracy.
UAV uses RL to navigate, map, and detect targets in unknown environments.
problem Optimizing UAV trajectory for accurate mapping and target detection in unknown environments.
method Formulated as an MDP, UAV uses RL to infer navigation policy.
result UAV autonomously explores high target detection areas while reconstructing the environment.
In this paper, we study Zermelo navigation on Riemannian manifolds and use that to solve a long standing problem in Finsler geometry. Namely, the complete classification of strongly convex Randers metrics of constant flag curvature.
This research introduces an autonomous robot navigation method using reinforcement learning.
problem Improving robot navigation in complex environments.
method Deep Q Network (DQN) and Proximal Policy Optimization (PPO) models for path planning and decision-making.
result The models enhance robot navigation ability and adaptive learning in unknown environments.
Capsule Networks improve autonomous navigation in sparse environments.
problem Challenges in reinforcement learning for sparse reward environments.
method Caps-EM pairs CapsNets with Advantage Actor Critic, using fewer parameters.
result Caps-EM achieves significant time improvements and fewer parameters compared to competing methods.
Agents learn to communicate and solve navigation tasks efficiently.
problem Understanding and developing communication in artificial agents.
method Multi-agent navigation tasks in gridworld environments.
result Agents develop an interpretable communication protocol that optimally solves tasks.
We present a novel human-aware navigation approach, where the robot learns to mimic humans to navigate safely in crowds. The presented model, referred to as DeepMoTIon, is trained with pedestrian surveillance data to predict human velocity in the environment. The robot processes LiDAR scans via the trained network to n…
Consider an assistive system that guides visually impaired users through speech and haptic feedback to their destination. Existing robotic and ubiquitous navigation technologies (e.g., portable, ground, or wearable systems) often operate in a generic, user-agnostic manner. However, to minimize confusion and navigation …
New method identifies critical states to improve RL agent explainability and speed.
problem Challenges in RL agent explainability and action selection timing.
method Identify critical states based on action-based variance in Q-function, prioritize exploitation on these states.
result Identified critical states accelerate RL in grid worlds and deep RL tasks.