Proposes a framework for predicting traffic situations involving multiple interacting agents.
problem Predicting the behavior of multiple interacting agents in traffic scenarios.
method Two-layer Hidden Markov Model (TLHMM) for situation distribution and learning-based dynamic scene evolution model for trajectory sampling.
result Demonstrates effectiveness and accuracy in highway ramp merging scenario.
Paper improves reinforcement learning in multi-scene tasks.
problem Reducing sample variance in multi-scene reinforcement learning.
method Sparse dynamic value estimation using Gaussian mixture models.
result Significant improvements in reward scores and navigation efficiency.
A new method for estimating joint value functions in multi-scene reinforcement learning.
problem High variance in samples for policy gradient computations in multi-scene environments.
method Sparse attention mechanism over multiple value function hypotheses to approximate the true joint value function.
result Significant improvements in reward scores and enhanced navigation efficiency across OpenAI ProcGen environments.
SNP extends Neural Processes to handle temporal dependencies in sequences.
problem Handling temporal dependencies in sequences of stochastic processes.
method Integrates a temporal state-transition model into Neural Processes.
result First 4D model capable of dynamic 3D scene modeling.
SCALOR learns scalable object representations for crowded scenes.
problem Scalability in scenes with many objects.
method Spatially-parallel attention and proposal-rejection mechanisms.
result SCALOR can handle up to a hundred objects in crowded scenes.
Improves reinforcement learning agent's scene-specific value function.
problem High variance in samples for policy gradient computations in multi-scene environments.
method Proposes dynamic value estimation (DVE) for multiple MDPs, clustering value functions across scenes.
result Lower sample variance and more accurate scene-specific value function estimates.
Light-based attacks can misclassify images without altering physical objects.
problem Physical attacks on deep learning classifiers are limited by the ability to modify inputs directly.
method Constructs an experimental setup with a light projection source, object, and camera. Uses differential evolution to select light patterns.
result Projected light can degrade classification accuracy from 98% to 22% for 2D objects and from 89% to 43% for 3D objects.
Proposes local coordinate frames for improving model performance in complex dynamical systems.
problem Improving model performance in complex, non-linear, and time-dependent dynamical systems.
method Introduces roto-translation invariant local coordinate frames for geometric graphs.
result The approach outperforms state-of-the-art models in various complex scenarios.
Paper learns particle dynamics for versatile object manipulation.
problem Challenges in traditional rigid-body physics engines for complex scenes.
method Combines learning with particle-based systems for versatile object manipulation.
result Robots achieve complex manipulation tasks using the learned simulator.
Modular method predicts motion in crowded scenes using learned environment models.
problem Predicting motion in dynamic, crowded environments.
method Modular model of spatial and dynamic aspects, unsupervised adaptation to new tasks.
result Comparable performance to state-of-the-art, transferable across tasks.
Proposes Deep Scenes for interaction-aware scene understanding in reinforcement learning for autonomous driving.
problem Leveraging deep reinforcement learning for high-level decision making in autonomous driving requires handling variable-length sequences of different object types and interactions.
method Introduces Deep Scenes architecture, an extension of Deep Sets or Graph Convolutional Networks, to learn complex interaction-aware scene representations.
result Graph-Q and DeepScene-Q algorithms outperform state-of-the-art methods in evaluations with SUMO.
A model learns object representations for physical scene understanding without direct supervision.
problem Learning object-centric representations without direct supervision of object properties.
method Object-Oriented Prediction and Planning (O2P2) model that learns perception, physics interaction, and rendering functions.
result The model can predict physical interactions and build block towers more complex than those seen during training.
Deep neural networks decode natural visual scenes from neural spikes.
problem Decoding visual scenes from neural spikes for brain-machine interfaces.
method Developed a novel spike-image decoder (SID) using deep neural networks.
result SID reconstructs natural visual scenes from neural spikes with high accuracy.
We propose an automatic method to infer high dynamic range illumination from a single, limited field-of-view, low dynamic range photograph of an indoor scene. In contrast to previous work that relies on specialized image capture, user input, and/or simple scene models, we train an end-to-end deep neural network that di…
A new neural network learns from acoustic scenes by suppressing irrelevant patterns.
problem Acoustic scenes are rich and redundant, making classification challenging.
method Spatio-temporal attention pooling layer coupled with a convolutional recurrent neural network.
result The method outperforms a strong convolutional neural network baseline and sets new state-of-the-art performance.
Soft geometric bias improves physical dynamics predictions.
problem Learning physical dynamics with exact group equivariance can degrade performance.
method Object-centric world models using geometric algebra neural networks.
result Soft geometric inductive bias leads to better physical fidelity predictions.
A lightweight network and NAS method improve ASC tasks.
problem Heavy computational burden in acoustic scene classification.
method Inspired by MobileNetV2, unidirectional convolutions; dynamic NAS with evolutionary algorithm.
result 90.3% F1-score on DCASE2018 task 5, 25% fewer FLOPs.
This work presents a novel approach for robust PCA with total variation regularization for foreground-background separation and denoising on noisy, moving camera video. Our proposed algorithm registers the raw (possibly corrupted) frames of a video and then jointly processes the registered frames to produce a decomposi…
Abstract reviews symmetry and reduction in dynamical systems.
problem Understanding symmetries and reductions in dynamical systems.
method Algebraic formulation for dynamics of physical systems.
result Describes a reduction procedure for classical and quantum evolutions.
A neural scene representation framework enforcing 3D transformations.
problem Learning 3D scene representations from images without 3D supervision.
method Introducing a loss enforcing equivariance of the scene representation with 3D transformations.
result Real-time neural rendering with comparable results to models requiring minutes for inference.
A central challenge in neuroscience is to understand neural computations and circuit mechanisms that underlie the encoding of ethologically relevant, natural stimuli. In multilayered neural circuits, nonlinear processes such as synaptic transmission and spiking dynamics present a significant obstacle to the creation of…
EvoNet predicts the evolution of dynamic graphs using a graph neural network and recurrent architecture.
problem Predicting the evolution of dynamic graphs is challenging and underexplored.
method EvoNet uses a graph neural network and recurrent architecture to predict the evolution of dynamic graphs.
result EvoNet effectively predicts the evolution of dynamic graphs on both artificial and real-world datasets.
New approach uses dynamic programming to efficiently discover failures in autonomous vehicle simulations.
problem Efficiently discovering rare failure events in autonomous vehicle simulations.
method Approximate dynamic programming and scene decomposition to estimate failure distribution.
result Increased number of failures discovered compared to baseline approaches.
KeyIn discovers keyframes to simplify video analysis.
problem Extracting essential dynamics from videos interleaved with predictable details.
method A hierarchical Keyframe-Inpainter (KeyIn) model learns keyframes and fills in the rest.
result KeyIn finds informative keyframes across various video dynamics.
Study the geometry and dynamics of skew evolutes and involutes, related to bicycle kinematics.
problem Understanding the geometry and dynamics of skew evolutes and involutes.
method Investigate the skew evolute and involute maps, comparing them to bicycle kinematics.
result The skew evolute and involute maps have properties analogous to bicycle kinematics.
GENESIS generates and samples 3D scenes by capturing object interactions.
problem Lack of models that explicitly capture object interactions in scene generation.
method Object-centric latent variables, spatial GMM, amortized inference, autoregressive prior.
result First object-centric generative model of 3D visual scenes.
Efficient model for foggy scene understanding in vehicles.
problem Challenging scene understanding and segmentation under foggy conditions.
method Domain adaptation and illumination-invariant image transformation.
result Outperforms state-of-the-art models in foggy scene understanding.
Scene text magnifier enhances readability for visually impaired.
problem Helps visually impaired read natural scene text.
method Four CNN-based networks: character erasing, extraction, magnify, synthesis.
result Effective text magnification without background alteration.
New model separates objects in scenes, enabling novel arrangements and depth.
problem Lack of modular, compositional scene modeling in generative models.
method Ensemble of generative models (experts) compete for explaining different parts of a scene.
result Model generates scenes with novel object arrangement and depth ordering.
ROOTS learns to represent and render 3D scenes with object-centric models.
problem Learning to represent and render 3D scenes with object-centric compositionality.
method Probabilistic generative model for learning object representations and scene rendering from partial observations.
result The model can infer 3D object representations and render scenes from arbitrary viewpoints.
This paper studies the critical dynamics of random surfaces, focusing on area and genus evolution.
problem Understanding the time evolution of random surfaces and their genus.
method Analyzes the dynamics of area and genus using Cox-Ingersoll-Ross process and critical phenomena.
result The genus of surfaces evolves into two phases: planar surfaces and foamy surfaces.
Framework learns dynamic graph attributes and links co-evolution.
problem Forecasting change of node attributes and link formation in dynamic graphs.
method CoEvoGNN framework with temporal self-attention and joint optimization.
result Framework outperforms baselines on predicting unseen graph snapshots.
MONet learns to decompose scenes into meaningful components without supervision.
problem Learning meaningful scene decompositions without labeled data.
method MONet combines a VAE and recurrent attention network to learn decompositions of 3D scenes.
result MONet can learn to represent 3D scenes into meaningful components like objects and background.
Despite enormous progress in object detection and classification, the problem of incorporating expected contextual relationships among object instances into modern recognition systems remains a key challenge. In this work we propose Information Pursuit, a Bayesian framework for scene parsing that combines prior models …
SPACE models complex scenes by decomposing objects and backgrounds.
problem Scalability and unsupervised object-oriented scene representation learning.
method Generative latent variable model combining spatial-attention and scene-mixture approaches.
result SPACE achieves factorized object representations and decomposes complex scenes.
NeRF-VAE generates 3D scenes with geometric structure from few images.
problem Generating 3D scenes from few images with geometric consistency.
method Combines NeRF and VAE, incorporating shared geometric structure.
result NeRF-VAE can infer and render geometrically-consistent scenes from unseen environments.
Enhances sound texture in CNN for better acoustic scene classification.
problem Limited understanding of how CNNs perceive audio scenes.
method Used Class Activation Mapping (CAM) to analyze log-Mel features and proposed edge enhancement using DoG and Sobel operators.
result Edge-enhanced log-Mel features improve CNN performance in acoustic scene classification.
Learning to control robots directly based on images is a primary challenge in robotics. However, many existing reinforcement learning approaches require iteratively obtaining millions of robot samples to learn a policy, which can take significant time. In this paper, we focus on learning a realistic world model capturi…
Improved acoustic scene classification with factorized CNN.
problem Acoustic scene classification in varying environments.
method Large-margin factorized CNN with triplet loss.
result Improved performance and better generalization on unseen data.
Generative models learn from unlabeled videos via object segmentation and scene modeling.
problem Learning generative models from unlabelled videos.
method Decomposed into three subtasks: motion segmentation, background and foreground modeling, and scene sampling.
result Approach allows learning models that generalize beyond occlusions and represent scenes in a modular fashion.
Generative Multisensory Network learns 3D scene representations from multiple modalities.
problem Learning robust 3D scene representations from multiple sensory modalities.
method Amortized Product-of-Experts for efficient inference and cross-modal generation.
result The model can infer modality-invariant 3D scene representations efficiently from various sensory modalities.
Pix2Shape learns 3D scene representations from single images without supervision.
problem Learning 3D scene information from a single image without supervision.
method Pix2Shape uses an encoder, decoder, and critic network to generate 2.5D surfel-based reconstructions.
result Pix2Shape can generate complex 3D scenes from a single image, scaling with on-screen resolution.
RICH models scenes as hierarchical tree to learn and generate complex compositions.
problem Learning compositional structures between parts and objects in natural scenes.
method RICH uses a latent scene graph to organize entities into a tree structure and employs a top-down inference approach.
result RICH learns and generates complex scene hierarchies from unlabeled data.
A novel method for visual question answering using scene graphs and reinforcement learning.
problem Answering free-form questions about images with deep linguistic and visual understanding.
method Context-driven, sequential reasoning based on scene graphs and reinforcement learning.
result Our method almost reaches human performance on the GQA dataset.
A new method to decompose audio scenes using deep learning.
problem Understanding audio scenes with random microphone arrangements.
method Formulated a neural network for nonnegative tensor factorization.
result Learned sources' individual spectral dictionaries and activation patterns.
We propose a systematic learning-based approach to the generation of massive quantities of synthetic 3D scenes and arbitrary numbers of photorealistic 2D images thereof, with associated ground truth information, for the purposes of training, benchmarking, and diagnosing learning-based computer vision and robotics algor…
AR model forecasts partially observed dynamical time series by estimating evolution function and imputing missing variables.
problem Forecasting dynamical time series with missing variables.
method Autoregressive with slack time series (ARS) model.
result ARS model forecasts future time series with time-invariant and linear assumptions.
Generative model creates realistic scenes from pixel-wise labels.
problem Creating photo-realistic scenes from pixel-wise labels.
method Semantic bottleneck GAN model combining conditional and unconditional generation networks.
result Model outperforms state-of-the-art models in unsupervised image synthesis.