Large datasets enable training of convolutional networks for optical flow, disparity, and scene flow.
problem Training convolutional networks for optical flow, disparity, and scene flow.
method Proposed three synthetic stereo video datasets for disparity, optical flow, and scene flow estimation.
result First large-scale datasets for training and evaluating scene flow methods.
Novel video prediction method for complex urban scenes using optical flow.
problem Making accurate future frame predictions in complex urban scenes.
method Optical flow conditioned method using video sequences and optical flow sequences.
result Empirical evaluations show the effectiveness of the method on KITTI and Cityscapes datasets.
This paper uses deep learning to estimate flow fields from OCT images for laser ablation control.
problem Automatic control of laser bone ablation using 4D OCT images.
method Semi-supervised convolutional neural network for 2.5D scene flow estimation.
result Scene flow estimation enables markerless tracking and automated laser ablation control.
Scene text magnifier enhances readability for visually impaired.
problem Helps visually impaired read natural scene text.
method Four CNN-based networks: character erasing, extraction, magnify, synthesis.
result Effective text magnification without background alteration.
Recent progress on automatic generation of image captions has shown that it is possible to describe the most salient information conveyed by images with accurate and meaningful sentences. In this paper, we propose an image caption system that exploits the parallel structures between images and sentences. In our model, …
Scene parsing is an important and challenging prob- lem in computer vision. It requires labeling each pixel in an image with the category it belongs to. Tradition- ally, it has been approached with hand-engineered features from color information in images. Recently convolutional neural networks (CNNs), which automatica…
A neural scene representation framework enforcing 3D transformations.
problem Learning 3D scene representations from images without 3D supervision.
method Introducing a loss enforcing equivariance of the scene representation with 3D transformations.
result Real-time neural rendering with comparable results to models requiring minutes for inference.
This paper introduces the notions of vector field and flow on a general differentiable stack. Our main theorem states that the flow of a vector field on a compact proper differentiable stack exists and is unique up to a uniquely determined 2-cell. This extends the usual result on the existence and uniqueness of flows o…
GENESIS generates and samples 3D scenes by capturing object interactions.
problem Lack of models that explicitly capture object interactions in scene generation.
method Object-centric latent variables, spatial GMM, amortized inference, autoregressive prior.
result First object-centric generative model of 3D visual scenes.
Efficient model for foggy scene understanding in vehicles.
problem Challenging scene understanding and segmentation under foggy conditions.
method Domain adaptation and illumination-invariant image transformation.
result Outperforms state-of-the-art models in foggy scene understanding.
Bayesian framework refines scene interpretation with prior and posterior models.
problem Challenges in incorporating contextual relationships into scene recognition.
method Combines prior scene geometry and object spatial arrangement with classifier outputs.
result Progressive refinement of scene interpretation through mutual information maximization.
New model separates objects in scenes, enabling novel arrangements and depth.
problem Lack of modular, compositional scene modeling in generative models.
method Ensemble of generative models (experts) compete for explaining different parts of a scene.
result Model generates scenes with novel object arrangement and depth ordering.
ROOTS learns to represent and render 3D scenes with object-centric models.
problem Learning to represent and render 3D scenes with object-centric compositionality.
method Probabilistic generative model for learning object representations and scene rendering from partial observations.
result The model can infer 3D object representations and render scenes from arbitrary viewpoints.
MONet learns to decompose scenes into meaningful components without supervision.
problem Learning meaningful scene decompositions without labeled data.
method MONet combines a VAE and recurrent attention network to learn decompositions of 3D scenes.
result MONet can learn to represent 3D scenes into meaningful components like objects and background.
SPACE models complex scenes by decomposing objects and backgrounds.
problem Scalability and unsupervised object-oriented scene representation learning.
method Generative latent variable model combining spatial-attention and scene-mixture approaches.
result SPACE achieves factorized object representations and decomposes complex scenes.
NeRF-VAE generates 3D scenes with geometric structure from few images.
problem Generating 3D scenes from few images with geometric consistency.
method Combines NeRF and VAE, incorporating shared geometric structure.
result NeRF-VAE can infer and render geometrically-consistent scenes from unseen environments.
Enhances sound texture in CNN for better acoustic scene classification.
problem Limited understanding of how CNNs perceive audio scenes.
method Used Class Activation Mapping (CAM) to analyze log-Mel features and proposed edge enhancement using DoG and Sobel operators.
result Edge-enhanced log-Mel features improve CNN performance in acoustic scene classification.
Systematic approach to generating diverse 3D scenes and images for training and benchmarking.
problem Training and benchmarking machine learning algorithms for scene understanding.
method Stochastic grammar-based pipeline for generating and rendering photorealistic 3D scenes with detailed ground truth.
result Improves performance in scene understanding tasks and provides controllable benchmarks for model diagnostics.
Improved acoustic scene classification with factorized CNN.
problem Acoustic scene classification in varying environments.
method Large-margin factorized CNN with triplet loss.
result Improved performance and better generalization on unseen data.
A new method for estimating joint value functions in multi-scene reinforcement learning.
problem High variance in samples for policy gradient computations in multi-scene environments.
method Sparse attention mechanism over multiple value function hypotheses to approximate the true joint value function.
result Significant improvements in reward scores and enhanced navigation efficiency across OpenAI ProcGen environments.
Generative models learn from unlabeled videos via object segmentation and scene modeling.
problem Learning generative models from unlabelled videos.
method Decomposed into three subtasks: motion segmentation, background and foreground modeling, and scene sampling.
result Approach allows learning models that generalize beyond occlusions and represent scenes in a modular fashion.
Pix2Shape learns 3D scene representations from single images without supervision.
problem Learning 3D scene information from a single image without supervision.
method Pix2Shape uses an encoder, decoder, and critic network to generate 2.5D surfel-based reconstructions.
result Pix2Shape can generate complex 3D scenes from a single image, scaling with on-screen resolution.
Generative Multisensory Network learns 3D scene representations from multiple modalities.
problem Learning robust 3D scene representations from multiple sensory modalities.
method Amortized Product-of-Experts for efficient inference and cross-modal generation.
result The model can infer modality-invariant 3D scene representations efficiently from various sensory modalities.
Model removes objects from general scenes using weak supervision.
problem Automatic object removal from general scene images with weak supervision.
method Two-stage editor architecture with mask generator and image in-painter; novel GAN prior for mask generator.
result Effectively removes a wide variety of objects from general scenes using weak supervision.
Paper improves reinforcement learning in multi-scene tasks.
problem Reducing sample variance in multi-scene reinforcement learning.
method Sparse dynamic value estimation using Gaussian mixture models.
result Significant improvements in reward scores and navigation efficiency.
RICH models scenes as hierarchical tree to learn and generate complex compositions.
problem Learning compositional structures between parts and objects in natural scenes.
method RICH uses a latent scene graph to organize entities into a tree structure and employs a top-down inference approach.
result RICH learns and generates complex scene hierarchies from unlabeled data.
Improves reinforcement learning agent's scene-specific value function.
problem High variance in samples for policy gradient computations in multi-scene environments.
method Proposes dynamic value estimation (DVE) for multiple MDPs, clustering value functions across scenes.
result Lower sample variance and more accurate scene-specific value function estimates.
Proposes a deep structured generative model for complex scenes.
problem Difficulty in generating images with complicated structures.
method Boosts GANs with structure information encoded by a stochastic and-or graph (sAOG).
result Successfully captures and generates images with complex structures.
A novel method for visual question answering using scene graphs and reinforcement learning.
problem Answering free-form questions about images with deep linguistic and visual understanding.
method Context-driven, sequential reasoning based on scene graphs and reinforcement learning.
result Our method almost reaches human performance on the GQA dataset.
A new method to decompose audio scenes using deep learning.
problem Understanding audio scenes with random microphone arrangements.
method Formulated a neural network for nonnegative tensor factorization.
result Learned sources' individual spectral dictionaries and activation patterns.
Generative model for set-valued data using permutation invariant flows.
problem Modeling set-valued data with conditional generative models.
method Conditional generative probabilistic model using continuous normalizing flows with permutation equivariant dynamics.
result Significantly outperforms non-permutation invariant baselines in log likelihood and domain-specific metrics.
Generative model creates realistic scenes from pixel-wise labels.
problem Creating photo-realistic scenes from pixel-wise labels.
method Semantic bottleneck GAN model combining conditional and unconditional generation networks.
result Model outperforms state-of-the-art models in unsupervised image synthesis.
P3I learns holistic scene representations from a single image.
problem Inferring camera poses, object locations, and global scene structures from a single image.
method Combines search-based and gradient-based algorithms.
result P3I outperforms baselines on various image manipulation tasks.
Generates coherent 3D scenes from monocular videos without supervision.
problem Lack of 3D scene modeling in video generation models.
method Trains a model to generate 3D scenes with moving objects and a background from monocular videos.
result Trained model generates coherent 3D scenes with multiple moving objects and a background.
New model improves image scene graph labeling.
problem Understanding complex scenes with multiple inter-related objects.
method Permutation-invariant structured prediction using deep learning.
result Achieves state-of-the-art results on Visual Genome benchmark.
SCALOR learns scalable object representations for crowded scenes.
problem Scalability in scenes with many objects.
method Spatially-parallel attention and proposal-rejection mechanisms.
result SCALOR can handle up to a hundred objects in crowded scenes.
Improved 3D scene understanding from partial point sets using multiview fusion.
problem Challenging task of 3D scene semantic understanding from partial point clouds.
method Multiview representation of 360° point clouds and fusion with original data.
result Overall increase of 31.9% and 4.3% in segmentation accuracy for partial and complete scenes.
Improved object segmentation and tracking in video using optical flow and initial state conditioning.
problem Challenges in fully unsupervised object-centric learning from video data.
method Weakly-supervised approach using optical flow and initial state conditioning.
result Conditioning the model on simple object location cues significantly improves instance segmentation in realistic synthetic data.
New dataset tests mental rotation from single images, improving model understanding of 3D scenes.
problem Understanding how a scene looks from a different viewpoint using a single image.
method Created CLEVR-MRT dataset, explored neural architectures for volumetric scene representations.
result Demonstrated the effectiveness of volumetric representations in answering mental rotation questions.
Proposes Deep Scenes for interaction-aware scene understanding in reinforcement learning for autonomous driving.
problem Leveraging deep reinforcement learning for high-level decision making in autonomous driving requires handling variable-length sequences of different object types and interactions.
method Introduces Deep Scenes architecture, an extension of Deep Sets or Graph Convolutional Networks, to learn complex interaction-aware scene representations.
result Graph-Q and DeepScene-Q algorithms outperform state-of-the-art methods in evaluations with SUMO.
Deep network predicts action sequences for complex tasks from a scene image.
problem Scalable task and motion planning from initial scene images.
method Deep convolutional recurrent neural network that predicts action sequences.
result Predicts promising action sequences, reducing motion planning problems.
CNNs accurately model retinal responses to natural scenes.
problem Understanding neural computations in retinal responses to natural stimuli.
method Deep convolutional neural networks (CNNs) were used to model retinal responses to natural scenes.
result CNNs are more accurate than linear models in predicting retinal responses to natural scenes.
New neural network outperforms existing methods in scene matching.
problem Automated scene matching with high accuracy and low false positives.
method Convolutional hashing using a new loss function and training scheme.
result Significantly higher true positive rate and 100-fold reduction in false positives.
Generative model improves scene graph prediction for rare compositions.
problem Challenges in inferring unseen or rare scene graph compositions.
method Synthesizing rare scene graphs by perturbing real ones, using conditional GANs.
result Marginal but consistent improvements in zero- and few-shot metrics.
Scene understanding remains a significant challenge in the computer vision community. The visual psychophysics literature has demonstrated the importance of interdependence among parts of the scene. Yet, the majority of methods in computer vision remain local. Pictorial structures have arisen as a fundamental parts-bas…
ObSuRF converts a single image into a 3D model with NeRFs.
problem Creating a 3D model from a single image with object segmentation.
method Unsupervised volume segmentation using Neural Radiance Fields (NeRFs).
result ObSuRF can segment a 3D scene into objects from a single image.
Paper proposes a voting method to improve acoustic scene classification.
problem Improving acoustic scene classification accuracy.
method Punishment voting algorithm based on super categories construction.
result Punishment voting significantly improves classification performance.
A multi-head attention network improves ASC by recognizing overlapping sound patterns.
problem Challenging ASC due to overlapping sound patterns and complex event mixtures.
method Proposes a multi-head attention network to model complex temporal input structures.
result Achieved competitive performance on DCASE 2018 Task 5 dataset.