CB-GLNs learn video data's complex dependencies via graph representation.
problem Capturing complex dependency structures in sequential data like videos.
method Represent video data as a graph, find compositional dependencies via graph-cut and message passing.
result CB-GLNs efficiently learn video data's semantic compositional structure.
Paper proposes new principles and framework for AVC learning from user-generated videos.
problem Challenges in learning audio-visual correspondence from short-term user-generated videos.
method Introduced new principles and a framework to facilitate AVC learning from videos' themes.
result Proposed approach outperformed baseline by 23.15% on KWAI-AD-AudVis corpus.
SummaryNet automates video summarisation using deep learning.
problem Creating informative video summaries from videos.
method Two-stream convolutional network for spatial and temporal features, encoder-decoder model for salient features, sigmoid regression with LSTM for frame probability.
result SummaryNet achieves comparable or better results than state-of-the-art methods on benchmark datasets.
LumièreNet creates lecture videos from audio narration.
problem Creating high-quality lecture videos from audio narration.
method Modular deep-learning architecture that learns from audio to video.
result Synthesizes high-quality lecture videos from any length of audio.
Proposes TDNs for learning complex video structures.
problem Complex temporal dependencies in sequential data, especially videos.
method Temporal Dependency Networks (TDNs) using graph representations and graph convolutions.
result Efficiently learns complex semantic structures of video data.
This paper highlights the need for explainable AI in video deep learning models.
problem Lack of explainable AI methods for video deep learning models.
method Illustrates the current state of video deep learning and highlights the need for explainability methods.
result Current explainability methods for video deep learning are insufficient and need improvement.
New method learns agent actions from raw video demonstrations.
problem Imitation learning for natural-looking movements.
method Generative Adversarial Networks (GANs) for joint state and reward learning.
result Adversarial imitation learning on raw videos produces similar performance to state-of-the-art methods.
Generative model for high-resolution video generation.
problem Challenges in generating high-resolution videos due to memory and training stability limitations.
method Progressive growing of sliced Wasserstein GANs (SWGAN) for incremental spatiotemporal information learning.
result Generated photorealistic face videos of 256x256x32 resolution with an inception score of 14.57.
UDVD uses deep learning to denoise videos without supervision.
problem Lack of clean video data for training deep learning models.
method UDVD is a CNN trained solely on noisy video data, adapting to local motion.
result UDVD performs as well as supervised methods, even with limited training data.
Paper introduces adversarial lossy compression for video artifacts reduction.
problem Unpleasant reconstruction artifacts in standard video coding schemes at low bit-rates.
method Adversarial lossy video compression model minimizing an adversarial distortion objective.
result Reduction of perceptual artifacts and detail reconstruction under extreme compression.
Paper proposes multi-task learning for multi-modal video Q&A.
problem Expensive to create large-scale datasets for multi-modal video Q&A.
method Composed of three networks: video Q&A, temporal retrieval, and modality alignment.
result State-of-the-art results on TVQA dataset.
Generates, predicts, and completes human action videos with a two-stage deep framework.
problem Severe ill-posedness in video generation, prediction, and completion.
method Two-stage deep framework: 1) Generates human pose sequence from noise, 2) Converts pose sequence to video.
result Produces high-quality video generation/prediction/completion results of longer duration.
Deep networks analyze video snippets to predict outcomes, revealing a border effect that can be adjusted for better accuracy.
problem Improving the accuracy of deep networks trained on small video snippets.
method Applied the deep Taylor / LRP technique to understand and identify a border effect, tuning the step size to improve accuracy.
result The step size used to build video snippets can be adjusted to improve deep network accuracy without retraining.
Develops a deep metric learning approach for detecting bugs in video games.
problem Automated detection of bugs in video games.
method State-State Siamese Networks (S3N) for deep metric learning.
result S3N learns meaningful embeddings to identify various types of bugs.
New method compresses facial videos using GANs and latent space optimization.
problem Efficiently compressing facial videos at low bit rates.
method Leverages StyleGAN for latent space representation and compression, learns optimal compression through entropy model and perceptual loss.
result Significantly reduces perceptual distortion at low bit rates compared to state-of-the-art codecs.
New video compression algorithm outperforms existing codecs.
problem Efficient video compression with low latency.
method End-to-end learned video coding architecture.
result Significantly outperforms existing codecs across bitrate ranges.
New metric for video generation models, addressing quality, coherence, and diversity.
problem Lack of metrics for evaluating generative models of video.
method Proposed Fréchet Video Distance (FVD) and StarCraft 2 Videos (SCV) benchmark.
result FVD correlates well with human judgment of generated videos.
Method learns from video demonstrations with human feedback.
problem Teaching autonomous agents using video demonstrations and human feedback.
method Constructs a mapping between standard and visual representations using a neural network.
result Effective in teaching a hopper agent to perform a backflip with minimal human feedback.
Paper introduces a new video generation model that learns uncertainty.
problem Generating accurate future video frames is challenging.
method The model learns a prior model of uncertainty and generates frames by sampling from this prior.
result Generated frames are varied and sharp, even many frames into the future.
Ground-A-Video edits videos without training, preserving intended changes.
problem Complex multi-attribute video editing with omitted or wrong changes.
method Grounding-guided video-to-video translation with Cross-Frame Gated Attention.
result Zero-shot multi-attribute video editing with improved accuracy and frame consistency.
New method learns robot actions from videos without explicit labels.
problem Training robots to perform tasks from few demonstrations.
method Uses images and text for task-agnostic and general representation, synthesizes hallucinated actions, and applies dense correspondences.
result Trains robot policies solely from RGB videos, achieving diverse tasks across different robots and environments.
Deep learning method extracts medical knowledge from YouTube videos.
problem Improving healthcare information dissemination through machine learning.
method Developed a deep learning method to classify YouTube videos by medical knowledge level.
result Preliminary results show satisfactory performance in extracting medical knowledge from videos.
New method tackles anomaly detection in video surveillance using continual learning.
problem Challenges in continual learning for high-dimensional applications like video surveillance.
method Transfer learning and continual learning for online anomaly detection.
result Significantly reduces training complexity and continual learning from recent data.
DIVE learns video representations even with missing data.
problem Missing data in video sequences.
method Disentangled Imputed Video autoEncoder (DIVE) with missingness latent variable.
result DIVE outperforms state-of-the-art baselines in imputing and predicting missing video frames.
Model predicts video factors independently without cross-factor interference.
problem Predicting video factors without cross-factor interference.
method Unsupervised variational model for disentangling video into independent factors.
result Often learns interpretable factors like objects in a scene.
Paper improves video feature learning for better downstream tasks.
problem Improving video feature learning for better performance on downstream tasks.
method Self-supervised learning approach using contrastive bidirectional transformer, extending BERT for real-valued feature vectors.
result Significantly improved performance on video classification, captioning, and segmentation tasks.
A dataset for evaluating engagement with scientific video lectures.
problem Challenges in managing learning resources due to rapid creation of video lectures.
method Introduction of VLEngagement dataset with content-based and video-specific features, and metrics related to user engagement.
result The largest and most diverse publicly available dataset for understanding context-agnostic engagement in video lectures.
Improved YouTube-8M classification accuracy using ensemble methods.
problem Improving video classification accuracy on YouTube-8M dataset.
method Ensemble methods to improve baseline predictions.
result Global prediction accuracy (GAP) increased from 77% to 80.7%.
System automates discovery and classification of training videos for career progression.
problem Difficulties in planning and navigating career paths due to changing job requirements and emerging sectors.
method Extracted educational videos, built a machine learning classifier, and optimized probability thresholds.
result Significant improvements in model performance by incorporating video attributes.
Generative model learns compact codes for video recovery.
problem Efficiently represent and reconstruct videos from missing data.
method Generative network trained to map compact latent codes to images, with low-rank and similarity constraints.
result Can recover true video sequences even if not in pretrained network's range.
The paper proposes a method to align surgical videos using kinematic data.
problem Difficulty in learning from comparing novice to expert surgical videos due to variability in gesture duration and execution.
method A novel technique using Dynamic Time Warping to synchronize videos of the same gesture at different speeds.
result The proposed approach allows for the alignment of surgical videos, enabling better learning for novice trainees.
DDPAE predicts video frames by decomposing and disentangling high-dimensional video data.
problem Predicting future video frames from input sequences is challenging due to high-dimensionality.
method Combines structured probabilistic models and deep networks to decompose and disentangle video components.
result DDPAE learns latent decomposition and disentanglement without supervision.
Scalable system predicts hot videos for peak VOD service.
problem Improving peak service quality of video on demand.
method Two neural networks: clustering and dispatch policy. Clustering reduces video numbers, dispatch policy ranks videos with probabilities. Networks are trained end-to-end.
result Average prediction accuracy of 17% compared to 3% baseline, for same number of dispatches.
This paper improves video summarization using a new algorithm and dataset.
problem Efficiently summarizing videos for browsing and searching.
method Improves sequential determinantal point process (SeqDPP) with a large-margin algorithm and a new probabilistic distribution.
result Significantly improved video summarization model with better user input integration and diversity.
Automated video conferencing system improves user experience with ASD and VC.
problem Improve remote video conferencing experience through automated speaker detection and virtual cinematography.
method Uses 4K wide-FOV camera, depth camera, and microphone array to extract features and train machine learning models for ASD and VC.
result System performs within 0.3 MOS of an expert cinematographer, as rated by users.
New video compression method outperforms traditional approaches.
problem Efficient video compression in low latency mode.
method Feedback Recurrent Autoencoder network architecture.
result State of the art MS-SSIM/rate performance on UVG dataset.
Improved video and movie description using multitask learning.
problem Lack of training data and poor generalization in video captioning.
method Multitask learning encoder-decoder framework for video sequences.
result Improved performance on multi-caption and single-caption datasets.
Paper describes a video surveillance system for highway traffic events.
problem Detecting specific sequences of situations in highway traffic videos.
method Compares RNN and CNN architectures for analyzing video frames and sequences.
result Best architecture performs well in real conditions.
Jointly trains images and videos using residual vectors.
problem Generating high-quality videos from images and vice versa.
method Simultaneously learns latent variables for images and videos using residual vectors.
result Improves sample quality and diversity in video generation and image generation.
Proposes a new video attack method that multiplies perturbation to improve model robustness.
problem Challenges existing defense methods for video recognition models against additive adversarial attacks.
method Introduces Multiplicative Adversarial Videos (MultAV) to impose perturbation by multiplication.
result Model trained against additive attacks is less robust to MultAV.
CNN-AIRL improves video game performance using expert demonstrations.
problem Designing reward functions for complex video games is difficult.
method Adversarial IRL with CNNs, autoencoder state representation, reward normalization.
result Achieves high-level performance on Catcher game, outperforming baseline.
A deep learning approach for efficient power control in wireless video transmissions.
problem Optimizing power control for real-time wireless video transmissions with quality constraints.
method Proposes a learning-based approach using a deep neural network to solve the non-convex power control problem.
result The deep neural network can quickly provide optimal power levels for given channel conditions.
End-to-end deep generative model for video compression.
problem Efficiently compressing video data with deep learning.
method Variational autoencoder (VAE) for sequential data, combined with neural image compression techniques.
result Our model achieves competitive rate-distortion results on diverse video content.
Paper discusses YouTube-8M challenge and video understanding research.
problem Improving video understanding models for YouTube videos.
method Experimented with various models and ensemble learning techniques.
result Significant improvement in competition score through ensemble learning.
Detect objects from motion without annotations.
problem Weakly supervised object detection.
method Train model on videos of moving objects and negative scenes.
result Detects objects in single images without annotations.
We present a new model DrNET that learns disentangled image representations from video. Our approach leverages the temporal coherence of video and a novel adversarial loss to learn a representation that factorizes each frame into a stationary part and a temporally varying component. The disentangled representation can …
Improves video search by balancing text and visual modalities.
problem Modality imbalance in video search models, focusing mainly on text matching.
method Proposes MBVR with MS samples and DM to balance modalities.
result Empirically shows significant improvement in modality balance and search effectiveness.
Deep learning improves video coding efficiency without client-side changes.
problem Improving video coding efficiency without altering existing standards.
method Cascaded downscaling neural networks coupled with adaptive precoding mode selection.
result 15% to 45% rate reduction in FHD and UHD content encoding.