We create real-time geodesic rendering for non-isotropic geometries.
problem Challenging visualization of non-isotropic geometries.
method Novel methods for real-time native geodesic rendering.
result Methods can be applied to visualization, machine learning, and video games.
Paper shows adversarial attacks are possible in real-time video classification systems.
problem Adversarial perturbations in real-time video classification systems.
method Exploits GAN architectures to generate stealthy adversarial samples.
result Generative adversarial networks can cause misclassification rates of over 80% in real-time video classification.
Automated video conferencing system improves user experience with ASD and VC.
problem Improve remote video conferencing experience through automated speaker detection and virtual cinematography.
method Uses 4K wide-FOV camera, depth camera, and microphone array to extract features and train machine learning models for ASD and VC.
result System performs within 0.3 MOS of an expert cinematographer, as rated by users.
AMS improves video inference on edge devices by adapting a small model with online knowledge distillation.
problem High computation cost of Deep Neural Networks for real-time video inference on edge devices.
method AMS uses a remote server to continually train and adapt a small model on edge devices, using online knowledge distillation from a large model.
result 0.4--17.8 percent mean Intersection-over-Union improvement in video semantic segmentation.
A deep learning approach for efficient power control in wireless video transmissions.
problem Optimizing power control for real-time wireless video transmissions with quality constraints.
method Proposes a learning-based approach using a deep neural network to solve the non-convex power control problem.
result The deep neural network can quickly provide optimal power levels for given channel conditions.
Paper proposes a GAN-based approach for RTLMP prediction.
problem Predicting real-time locational marginal prices (RTLMPs) in power markets.
method GAN-based video prediction model for spatio-temporal correlations.
result Proposed method accurately predicts RTLMPs without confidential information.
This paper optimizes object tracking on edge devices with small matrices.
problem Efficiently tracking objects in video sequences on edge devices with small matrices.
method Parallelized a Simple Online and Real-time Tracking (SORT) application on shared-memory multicores.
result Throughput-based parallelization technique outperforms multi-threading for small matrices.
IrisNet tracks tongue contours in ultrasound videos using deep learning.
problem Automatic and real-time tracking of tongue contours in ultrasound videos.
method Convolutional neural network mimicking peripheral vision.
result Outstanding generalization on various ultrasound tongue datasets.
Edge filters reduce video data transmission to datacenters.
problem Strain on wide area network infrastructure due to video camera deployments.
method FilterForward system with lightweight edge filters and microclassifiers.
result Reduces bandwidth use by an order of magnitude.
Improved animated faces using audiovisual and modality dropout.
problem Creating realistic animated faces using speech and visual cues.
method Training a deep learning model with modality dropout to balance audio and visual inputs.
result Modality dropout improves viewer preference for audiovisual-driven animation.
New RL framework improves real-time control performance.
problem Real-time RL systems assume static states, leading to suboptimal outcomes.
method Introduces a new real-time RL framework where states and actions evolve simultaneously.
result RTAC algorithm outperforms existing state-of-the-art algorithms in real-time and non-real-time settings.
Recently, several models based on deep neural networks have achieved great success in terms of both reconstruction accuracy and computational performance for single image super-resolution. In these methods, the low resolution (LR) input image is upscaled to the high resolution (HR) space using a single filter, commonly…
The waggle dance that honeybees perform is an astonishing way of communicating the location of food source. After over 60 years of its discovery, researchers still use manual labeling by watching hours of dance videos to detect different transitions between dance components thus extracting information regarding the dis…
New approach uses synthetic labels to train models on scarce annotated data for surgical phase recognition.
problem Learning surgical phase recognition from limited annotated data.
method Teacher/Student approach with a CNN-biLSTM-CRF teacher generating synthetic labels for a CNN-LSTM student.
result Improved surgical phase recognition performance with fewer annotated videos.
Real-time spell checker adapts to new languages.
problem No real-time, language-adaptable spell checkers for non-English languages.
method Used Wikipedia and subtitles data to generate dictionaries, created noisy channel datasets, compared with industry tools.
result System performs well across 24 languages, outperforming existing tools.
CADNN optimizes DNN execution on smartphones for real-time inference.
problem Executing Deep Neural Networks on mobile devices with low latency and high accuracy.
method Advanced model compression and architecture-aware optimization.
result CADNN outperforms state-of-the-art frameworks in DNN execution on mobile devices.
Lipper synthesizes speech from silent videos, improving over single-view methods.
problem Lipreading as text classification is limited; multi-view approach needed.
method Multi-view lipreading as a regression task, producing speech from silent videos.
result Improvement in speech reconstruction with multi-view silent videos.
This paper presents GRASTA (Grassmannian Robust Adaptive Subspace Tracking Algorithm), an efficient and robust online algorithm for tracking subspaces from highly incomplete information. The algorithm uses a robust l1-norm cost function in order to estimate and track non-stationary subspaces when the streaming data …
Paper develops real-time through-wall human pose imaging using RF signals.
problem Developing real-time vision through opaque objects.
method Many-to-Many Encoder/Decoder Paradigm, student/teacher learning, Residual CNN, RPN, LSTM.
result Accurately predicts human pose through visual occlusion using RF signals.
Real-time scene understanding solved using Approximate Bayesian Computation.
problem Predicting human actions, object poses, and pedestrian crossings from depth images.
method Bayesian error model, neural surrogates, and adaptive discretization.
result Real-time inference on real-world problems is feasible.
Dual frame-rate system improves real-time perception for autonomous vehicles.
problem Conflicting requirement of safety and efficiency in real-time perception systems.
method Proposes a dual frame-rate system with a modulator stream for robust features and a prediction stream for transient signals.
result Consistent improvements across various backbone architectures and input resolutions.
Real-time fetal abdominal aorta measurement from ultrasound images.
problem Automating the challenging task of measuring fetal abdominal aorta diameter from ultrasound images.
method Proposes a neural network architecture with three blocks: convolutional layer, Convolution Gated Recurrent Unit (C-GRU), and CyclicLoss.
result Significantly improved accuracy and real-time execution speed compared to previous methods.
Automated graphics testing detects novel corruptions without manual labeling.
problem Detecting novel visual corruptions in graphics unit testing without manual labeling.
method Reproduces driver bugs to generate corruptions and uses Multiple Instance Learning (MIL) methods.
result Significantly outperforms unsupervised methods and discovers novel corruptions.
Unsupervised fire and smoke segmentation from IR videos.
problem Early detection of fire from infrared videos.
method Spatial, temporal, and motion information fusion using optical flow, divergence, and intensity values.
result Markov Random Field outperforms other clustering algorithms in fire and smoke segmentation.
Dreamer 4 learns Minecraft tasks from videos alone.
problem Accurately predicting object interactions in complex environments.
method Reinforcement learning inside a fast, accurate world model.
result Dreamer 4 outperforms previous models in Minecraft, learning from only offline data.
We present a novel algorithm, called Links, designed to perform online clustering on unit vectors in a high-dimensional Euclidean space. The algorithm is appropriate when it is necessary to cluster data efficiently as it streams in, and is to be contrasted with traditional batch clustering algorithms that have access t…
SA-ABR uses UAV sensor data to optimize video streaming quality.
problem Dynamic UAV flight states cause fluctuating video streaming quality.
method SA-ABR integrates sensor data with network observations to train a DRL model.
result SA-ABR outperforms existing ABR algorithms by 21.4% in QoE.
The paper improves safety in autonomous systems using adversarial learning.
problem Ensuring safety in real-time control systems of autonomous vehicles.
method The paper introduces a dual anomaly detection framework (CFAM and SFAM) using generative adversarial networks (GANs) and video prediction.
result Demonstrated effectiveness on both indoor and outdoor autonomous ground vehicles.
Model-based RL learns Atari games faster with fewer interactions.
problem Efficiently learning Atari games with minimal interactions.
method Simulated Policy Learning (SimPLe) using video prediction models.
result SimPLe outperforms model-free methods in most Atari games with fewer interactions.
Aerial robot estimates human pose and path using dynamic classifier selection.
problem Estimating human pose and trajectory from aerial video.
method Dynamic classifier selection architecture; perspective correction; HOG and CNN features; 64 pose-viewpoint classes.
result Dynamic classifier selection improves efficiency and accuracy.
Learning-based methods for visual segmentation have made progress on particular types of segmentation tasks, but are limited by the necessary supervision, the narrow definitions of fixed tasks, and the lack of control during inference for correcting errors. To remedy the rigidity and annotation burden of standard appro…
Robust PCA methods are typically batch algorithms which requires loading all observations into memory before processing. This makes them inefficient to process big data. In this paper, we develop an efficient online robust principal component methods, namely online moving window robust principal component analysis (OMW…
System guides freehand obstetric ultrasound probe movements.
problem Reducing operator expertise for standard plane acquisition.
method Artificial neural network trained on real scans.
result 88.8% accuracy for goal prediction, 90.9% for action prediction.
Latent Replay reduces continual learning computation and storage needs.
problem Catastrophic forgetting in continual learning.
method Store activations volumes at intermediate layers instead of past data, slow down learning below the latent replay layer.
result Latent Replay achieves state-of-the-art performance on complex video benchmarks.
MobileNet-v2 improves re-ID on edge devices with mixed precision.
problem Real-time person re-identification in edge devices with limited resources.
method Mixed precision training (FP32 and FP16) for MobileNet-v2 on edge nodes.
result Mixed precision training improves inference throughput and reduces power consumption.
Ground-A-Video edits videos without training, preserving intended changes.
problem Complex multi-attribute video editing with omitted or wrong changes.
method Grounding-guided video-to-video translation with Cross-Frame Gated Attention.
result Zero-shot multi-attribute video editing with improved accuracy and frame consistency.
Data-driven control of robotic systems using Koopman operators with error bounds.
problem Real-time control of nonlinear robotic systems with unknown dynamics.
method Constructing a Koopman operator-based linear representation using higher-order derivatives of nonlinear dynamics, with error bounds derived from Taylor series accuracy analysis.
result The Koopman model provides marginally better performance than competing nonlinear modeling methods and can be efficiently controlled using linear control design tools.
CB-GLNs learn video data's complex dependencies via graph representation.
problem Capturing complex dependency structures in sequential data like videos.
method Represent video data as a graph, find compositional dependencies via graph-cut and message passing.
result CB-GLNs efficiently learn video data's semantic compositional structure.
Recurrent CNNs improve image classification in low light conditions.
problem Poor performance of CNNs in noisy images.
method Added recurrent connections to CNN layers to enhance robustness.
result gruCNNs outperform cCNNs in low signal-to-noise ratio images.
Recent work has shown that optical flow estimation can be formulated as a supervised learning task and can be successfully solved with convolutional networks. Training of the so-called FlowNet was enabled by a large synthetically generated dataset. The present paper extends the concept of optical flow estimation via co…
RaMViD uses diffusion models for video prediction and infilling.
problem Predicting and infilling missing information in videos.
method Extends image diffusion models to videos using 3D convolutions and a new conditioning technique.
result Achieves state-of-the-art results on video prediction benchmarks.
Current deep learning results on video generation are limited while there are only a few first results on video prediction and no relevant significant results on video completion. This is due to the severe ill-posedness inherent in these three problems. In this paper, we focus on human action videos, and propose a gene…
New metric for video generation models, addressing quality, coherence, and diversity.
problem Lack of metrics for evaluating generative models of video.
method Proposed Fréchet Video Distance (FVD) and StarCraft 2 Videos (SCV) benchmark.
result FVD correlates well with human judgment of generated videos.
LumièreNet creates lecture videos from audio narration.
problem Creating high-quality lecture videos from audio narration.
method Modular deep-learning architecture that learns from audio to video.
result Synthesizes high-quality lecture videos from any length of audio.
Paper defends against adversarial videos by detecting and reducing imperceptible perturbations.
problem Adversarial videos can fool well-trained video classification models.
method Temporal consistency between frames and spatial denoising to detect and reduce perturbations.
result The proposed method significantly improves robustness against adversarial attacks.
Paper improves video categorization using temporal coherence.
problem Video categorization in multiple modalities.
method Temporal coherence-based regularization for multimodal models.
result Models with temporal coherence outperform state-of-the-art.
Improves video search by balancing text and visual modalities.
problem Modality imbalance in video search models, focusing mainly on text matching.
method Proposes MBVR with MS samples and DM to balance modalities.
result Empirically shows significant improvement in modality balance and search effectiveness.
A new video prediction model treats videos as continuous processes, reducing sampling steps and improving efficiency.
problem Efficiency and temporal coherence in video prediction models.
method Treats videos as a continuous multi-dimensional process, reducing sampling steps.
result Reduction of 75% sampling steps, state-of-the-art performance on benchmark datasets.