Top 8 robotic vision systems tackled lifelong object recognition challenges.
problem Lifelong learning in robotic vision for varied, dynamic environments.
method Design of a dataset with diverse conditions and rules for evaluation.
result Robotic vision systems improved over time with dynamic object appearances.
New dataset and benchmarks for lifelong robotic vision tasks.
problem Challenges in applying computer vision to robots, especially lifelong learning.
method Provided a new lifelong robotic vision dataset and benchmarks.
result Demonstrated the complexity and bottlenecks in lifelong object recognition.
Detects anomalies in autonomous mobile robots using vision.
problem Anomaly detection for autonomous mobile robots.
method Unsupervised deep learning methods and a novel dataset.
result State-of-the-art approach tested on a new dataset.
Systematic review of machine vision in robotics, highlighting challenges and improvements.
problem Challenges in machine vision for robotics, especially occlusion and lighting variance.
method Systematic literature review of 172 papers from four databases, selecting 52 relevant papers.
result Robustness and computation time improvements, but occlusion and lighting variance remain major issues.
Project aims to improve robot vision by adapting to new environments.
problem Robots struggle to recognize objects in new settings.
method Selected and evaluated state-of-the-art DA methods on various datasets.
result Training a good object classifier remains challenging.
Method trains vision and control policies on real robots quickly.
problem Training vision-based control policies on real robots efficiently.
method Multi-task Reinforcement Learning with auxiliary tasks.
result Significant learning speed-ups and task learning from-scratch.
Diffusion models enhance robotic manipulation through probabilistic multi-modal learning.
problem Enhancing robotic manipulation through robust and multi-modal learning.
method Probabilistic diffusion models integrating imitation and reinforcement learning.
result Diffusion models improve grasp learning, trajectory planning, and data augmentation in robotics.
New method improves robot learning from vision with better sample efficiency.
problem Scaling reinforcement learning to real robots from vision.
method State representation learning to extract relevant features.
result Improved sample efficiency and robustness to hyper-parameters.
This work proposes a RL approach to learn versatile robotic manipulation tasks.
problem Challenging manipulation tasks in robotics and vision.
method Reinforcement learning (RL) to combine primitive skills, no intermediate rewards, few demonstrations, and efficient skill learning.
result Versatile robotic manipulation in challenging settings with temporary occlusions and dynamic scene changes.
Q2-Opt improves robot grasping success and efficiency.
problem Improving robot grasping success and efficiency in vision-based tasks.
method Quantile QT-Opt, a distributional variant of Q-learning for continuous domains.
result Q2-Opt achieves superior grasping success and is more sample efficient.
In this paper, we study the problem of learning vision-based dynamic manipulation skills using a scalable reinforcement learning approach. We study this problem in the context of grasping, a longstanding challenge in robotic manipulation. In contrast to static learning behaviors that choose a grasp point and then execu…
TACTO simulates high-resolution touch sensing for robotics.
problem Accurate simulation of touch sensing in robotics.
method Fast, flexible, open-source simulator for vision-based tactile sensors.
result Demonstrated TACTO's effectiveness in grasping stability prediction and marble manipulation control.
Enhances neural rendering with geometry-aware attention.
problem Efficiently modeling complex 3D scenes.
method Introduces Epipolar Cross Attention (ECA) for non-local operations.
result Significant improvement in Generative Query Networks (GQN) performance.
Robotic grasping system learns to target objects from a single image.
problem Robotic grasping in unstructured environments.
method Distributed reinforcement learning, active vision, synchronous SGD.
result System learns to grasp unseen objects without retraining.
STRING improves 2D and 3D position encodings for better performance.
problem Efficient and accurate position encoding for 2D and 3D applications.
method STRING extends Rotary Position Encodings with a unifying theoretical framework, maintaining translation invariance and low computational cost.
result STRING shows substantial gains in open-vocabulary object detection and robotics.
Hierarchical Foresight improves robot vision tasks by planning long-term goals.
problem Compounding uncertainty and scalability issues in long horizon video prediction.
method Subgoal generation and planning using hierarchical visual foresight (HVF).
result Achieves nearly 200% performance improvement in vision-based manipulation tasks.
Paper introduces rehearsal-free continual learning for small, non-i.i.d. batches in robotic vision.
problem Learning new objects and improving recognition in a changing robotic environment.
method Two rehearsal-free continual learning techniques (CWR* and AR1*) for small, non-i.i.d. batches.
result AR1* outperforms other techniques by more than 15% in some cases.
Paper presents a method for efficient robot adaptation using fine-tuning.
problem Continuous adaptation of robot learning systems in real-world scenarios.
method Fine-tuning previously learned policies using off-policy reinforcement learning.
result Fine-tuning leads to substantial performance gains and adaptation to new conditions.
Study detects anomalies in robot vision data to predict hazards.
problem Detecting unexpected hazards in robot exploration data.
method Anomaly detection using autoencoders at different scales.
result Autoencoders improve anomaly detection performance on diverse robot scenarios.
In this paper, we explore deep reinforcement learning algorithms for vision-based robotic grasping. Model-free deep reinforcement learning (RL) has been successfully applied to a range of challenging environments, but the proliferation of algorithms makes it difficult to discern which particular approach would be best …
Method learns multi-stage tasks from single video, overcoming challenges of raw pixel learning and insufficient demonstrations.
problem Learning multi-stage vision-based tasks from a single video of a human performing the task.
method Learn primitive behaviors from video demonstrations and dynamically compose them to perform multi-stage tasks.
result Demonstrated learning of various tasks on real robots using raw pixel inputs and minimal demonstrations.
Robot solves Rubik's cube using simulation-trained models.
problem Solving Rubik's cube with a robot hand.
method Automatic domain randomization (ADR) and a custom robot platform.
result Control policies and vision state estimators trained with ADR exhibit improved sim2real transfer.
Deep neural networks have been widely adopted in recent years, exhibiting impressive performances in several application domains. It has however been shown that they can be fooled by adversarial examples, i.e., images altered by a barely-perceivable adversarial noise, carefully crafted to mislead classification. In thi…
Robotic clothing manipulation improved with fashion image analysis techniques.
problem Automated identification of clothing categories and landmarks for robotic tasks.
method Training data augmentation methods and rotation invariant convolutions.
result Our approach outperforms state-of-the-art models on unseen datasets.
Deep RL improves robot navigation in images.
problem Applying deep RL to visual navigation in realistic environments.
method Extended A2C algorithm with auxiliary tasks for segmentation, depth prediction, and target prediction.
result Method outperforms state-of-the-art visual navigation methods.
DIGIT is a low-cost tactile sensor for in-hand manipulation.
problem Difficulty in sensing contact forces limits robotic manipulation.
method DIGIT miniaturizes and improves a vision-based tactile sensor.
result DIGIT enables better control of interactions with the environment.
Deep learning is an established framework for learning hierarchical data representations. While compute power is in abundance, one of the main challenges in applying this framework to robotic grasping has been obtaining the amount of data needed to learn these representations, and structuring the data to the task at ha…
Hybrid RL combines simulated and real data for robust autonomous flight.
problem Challenges in training deep RL models for real-world robotic tasks.
method Combines real-world and simulated data to improve generalization.
result Quadrotor avoids collisions using only a monocular camera.
This work tackles real-world robotic reinforcement learning challenges.
problem Limited success of reinforcement learning in real-world robotics.
method Proposes a system for autonomous real-world learning without instrumentation.
result Demonstrates a complete system that learns without human intervention.
Proposes Gaussian Processes for more accurate time-correlated measurement noise in robotics.
problem Time-correlated measurement noise in robotics applications.
method Gaussian Processes as a non-parametric model for correlated measurement noise.
result Improved performance of Kalman filtering with Gaussian Processes.
Enhances robotic grasping efficiency with learning-adaptive imagination.
problem Improving sample efficiency and performance in robotic grasping tasks.
method Learning-adaptive imagination approach using ensemble of local dynamics models in latent space.
result Significantly improves sample efficiency and achieves near-optimal performance.
RL learns dexterous object reorientation on a physical robot.
problem Learning complex in-hand manipulation tasks on physical robots.
method Reinforcement learning in a simulated environment, transfer learning.
result RL policies transfer from simulation to physical robot.
This paper bridges outlier-robust estimation in robotics and computer vision with robust statistics.
problem Outlier-robust estimation for geometric perception in robotics and computer vision.
method Adapting and extending robust linear regression and list-decodable regression to non-convex domains and vector-valued measurements.
result Performance guarantees for modern estimation algorithms in the presence of outliers.
Robots learn diverse behaviors to adapt to changing environments.
problem Robots struggle to adapt to new environments with unexpected changes.
method Generative adversarial policy networks to learn and sample a diverse set of behaviors.
result Robots can hit targets more often in changing environments.
Equivariant networks improve geometric prediction without scalar approximations.
problem Efficiently predicting geometric tensors in real-world scenarios.
method Equivariant networks for geometric prediction.
result Equivariant networks can generalize to unseen systems for geometric prediction.
Reliable and effective multi-task learning is a prerequisite for the development of robotic agents that can quickly learn to accomplish related, everyday tasks. However, in the reinforcement learning domain, multi-task learning has not exhibited the same level of success as in other domains, such as computer vision. In…
This paper improves robot grasping by integrating meta-control and latent-space imagination.
problem Dual-system approaches fail to consider the reliability of the learned model when making multiple-step predictions.
method A meta-controller arbitrates between model-based and model-free decisions based on local reliability, encouraging actions that improve the model and generating imagined experiences for additional training.
result Our approach learns near-optimal grasping policies in dense- and sparse-reward environments, outperforming baseline and state-of-the-art methods.
Y-GAN uses multi-camera data to estimate depth maps without expensive hardware.
problem Depth perception for autonomous systems requires accurate 3D spatial information.
method Proposes Y-GAN, a deep convolutional generative adversarial network.
result Y-GAN estimates depth maps from multi-camera stereo images without ground truth data.
Optimizes synthetic image augmentation for sim2real policy transfer in robotics.
problem Difficulty in transferring learned policies from simulated to real environments.
method Optimizes random transformations to augment synthetic images, enabling policy learning without real data.
result Significant improvement in policy accuracy on real robots for three manipulation tasks.
Survey on deep neural networks for vision and speech.
problem Improving intelligent vision and speech systems.
method Review of deep learning models and challenges.
result Emerging technologies show promise for future systems.
Improved RL for grasping in cluttered scenes using state representation learning.
problem Poor performance of RL methods in grasping diverse objects from raw images.
method Employed state representation learning (SRL) with disentanglement of raw input images.
result Deep RL can learn grasping skills from varied visual inputs.
V-SysId identifies keypoints and 3D system from unlabeled videos.
problem Identifying keypoints and 3D system from unlabeled videos.
method Alternates between parameter estimation and extrinsic camera calibration, using motion equations as weak supervision.
result Utility of the approach demonstrated across various settings.
A framework disentangles controllable objects from visual signals for improved RL.
problem Improving sample efficiency and game performance in vision-based RL.
method Action-conditioned video prediction to disentangle controllable objects.
result Improved sample efficiency and game performance in Atari games.
Paper presents a semi-supervised grasp detection method using VQ-VAE.
problem Robotic grasp detection difficulty due to insufficient labelled data.
method Semi-supervised learning with VQ-VAE in a latent space.
result Model performs better than existing approaches using unlabelled images.
We present an approach for building an active agent that learns to segment its visual observations into individual objects by interacting with its environment in a completely self-supervised manner. The agent uses its current segmentation model to infer pixels that constitute objects and refines the segmentation model …
Overview of deep reinforcement learning, including elements and applications.
problem Optimizing decision-making in complex environments.
method Combining deep learning with reinforcement learning algorithms.
result Effective in diverse applications like games, robotics, and NLP.
New method converts natural language commands into reward functions for robots.
problem Creating effective reward functions for autonomous machines.
method Language-conditioned reward learning (LC-RL) using inverse reinforcement learning.
result Model learns transferable rewards from natural language commands.
xYOLO speeds up object detection on low-end hardware for humanoid soccer robots.
problem Real-time object detection on resource-constrained devices like Raspberry Pi.
method Adaptation of YOLO CNN model to achieve faster inference speed.
result Achieved 9.66 FPS on Raspberry Pi 3 B, 70x faster than Tiny-YOLO.