Generates, predicts, and completes human action videos with a two-stage deep framework.
problem Severe ill-posedness in video generation, prediction, and completion.
method Two-stage deep framework: 1) Generates human pose sequence from noise, 2) Converts pose sequence to video.
result Produces high-quality video generation/prediction/completion results of longer duration.
LumièreNet creates lecture videos from audio narration.
problem Creating high-quality lecture videos from audio narration.
method Modular deep-learning architecture that learns from audio to video.
result Synthesizes high-quality lecture videos from any length of audio.
Paper proposes detecting video manipulation using stream descriptors.
problem Misuse of manipulated video content.
method Binary classifiers on multimedia stream descriptors.
result Scalable approach can detect high-quality manipulations.
Paper improves tensor completion using unitary transforms.
problem Robust tensor completion for various datasets.
method Transformed tensor SVD with unitary matrices.
result Recovered images have better PSNR than traditional methods.
With the growth of user-generated content, we observe the constant rise of the number of companies, such as search engines, content aggregators, etc., that operate with tremendous amounts of web content not being the services hosting it. Thus, aiming to locate the most important content and promote it to the users, the…
Survey of deep learning methods for video multi-object tracking.
problem Tracking multiple objects in video sequences.
method Review of deep learning approaches applied to MOT stages.
result Identification of similarities among top-performing methods.
Deep Fusion improves audio-video emotion recognition accuracy.
problem Challenges in automatic emotion recognition due to abstract concept and multiple expressions of emotion.
method Introduces factorized bilinear pooling (FBP) with embedded attention mechanism to integrate audio and video features.
result Achieves an accuracy of 62.48% on AFEW database, outperforming state-of-the-art results.
Seq2Tens uses tensors to efficiently represent sequences, improving performance on time series and video tasks.
problem Challenges in analyzing sequential data due to complex dependencies and non-commutativity.
method Uses tensor algebra to capture dependencies and low-rank tensor projections to manage computational complexity.
result State-of-the-art performance on multivariate time series classification and video generation benchmarks.
PredNet excels at natural videos but fails on artificial rules, suggesting a trade-off.
problem Generalization of neural network performance across different datasets.
method Evaluation of PredNet on natural and artificial video datasets.
result PredNet's high performance on natural videos is incompatible with its inability to learn artificial rules.
PredNet fails to fully adhere to predictive coding principles.
problem Evaluation of PredNet's adherence to predictive coding theory.
method Critical analysis of PredNet's architecture and performance on a video action classification dataset.
result PredNet does not fully follow predictive coding principles but shows potential with top-down conditioning.
SRTC model for background/foreground separation with missing pixels.
problem Background/foreground separation with missing pixels in videos.
method Smooth robust tensor completion (SRTC) model with tensor proximal alternating minimization (tenPAM).
result Global convergence guarantee for the proposed algorithm.
Optimizes deep reinforcement learning for energy-efficient video streaming.
problem Minimizing energy consumption in video streaming over mobile networks.
method Integrates DDPG algorithm with partially known model to reduce signaling overhead and improve convergence speed.
result Proposed policy converges to optimal policy with improved convergence speed.
This paper presents GRASTA (Grassmannian Robust Adaptive Subspace Tracking Algorithm), an efficient and robust online algorithm for tracking subspaces from highly incomplete information. The algorithm uses a robust l1-norm cost function in order to estimate and track non-stationary subspaces when the streaming data …
Ground-A-Video edits videos without training, preserving intended changes.
problem Complex multi-attribute video editing with omitted or wrong changes.
method Grounding-guided video-to-video translation with Cross-Frame Gated Attention.
result Zero-shot multi-attribute video editing with improved accuracy and frame consistency.
CB-GLNs learn video data's complex dependencies via graph representation.
problem Capturing complex dependency structures in sequential data like videos.
method Represent video data as a graph, find compositional dependencies via graph-cut and message passing.
result CB-GLNs efficiently learn video data's semantic compositional structure.
RaMViD uses diffusion models for video prediction and infilling.
problem Predicting and infilling missing information in videos.
method Extends image diffusion models to videos using 3D convolutions and a new conditioning technique.
result Achieves state-of-the-art results on video prediction benchmarks.
New metric for video generation models, addressing quality, coherence, and diversity.
problem Lack of metrics for evaluating generative models of video.
method Proposed Fréchet Video Distance (FVD) and StarCraft 2 Videos (SCV) benchmark.
result FVD correlates well with human judgment of generated videos.
Paper defends against adversarial videos by detecting and reducing imperceptible perturbations.
problem Adversarial videos can fool well-trained video classification models.
method Temporal consistency between frames and spatial denoising to detect and reduce perturbations.
result The proposed method significantly improves robustness against adversarial attacks.
Paper improves video categorization using temporal coherence.
problem Video categorization in multiple modalities.
method Temporal coherence-based regularization for multimodal models.
result Models with temporal coherence outperform state-of-the-art.
Improves video search by balancing text and visual modalities.
problem Modality imbalance in video search models, focusing mainly on text matching.
method Proposes MBVR with MS samples and DM to balance modalities.
result Empirically shows significant improvement in modality balance and search effectiveness.
A new video prediction model treats videos as continuous processes, reducing sampling steps and improving efficiency.
problem Efficiency and temporal coherence in video prediction models.
method Treats videos as a continuous multi-dimensional process, reducing sampling steps.
result Reduction of 75% sampling steps, state-of-the-art performance on benchmark datasets.
UDVD uses deep learning to denoise videos without supervision.
problem Lack of clean video data for training deep learning models.
method UDVD is a CNN trained solely on noisy video data, adapting to local motion.
result UDVD performs as well as supervised methods, even with limited training data.
Generative model for high-resolution video generation.
problem Challenges in generating high-resolution videos due to memory and training stability limitations.
method Progressive growing of sliced Wasserstein GANs (SWGAN) for incremental spatiotemporal information learning.
result Generated photorealistic face videos of 256x256x32 resolution with an inception score of 14.57.
Proposes TDNs for learning complex video structures.
problem Complex temporal dependencies in sequential data, especially videos.
method Temporal Dependency Networks (TDNs) using graph representations and graph convolutions.
result Efficiently learns complex semantic structures of video data.
SummaryNet automates video summarisation using deep learning.
problem Creating informative video summaries from videos.
method Two-stream convolutional network for spatial and temporal features, encoder-decoder model for salient features, sigmoid regression with LSTM for frame probability.
result SummaryNet achieves comparable or better results than state-of-the-art methods on benchmark datasets.
Paper proposes SMFN for high-res spherical video super-resolution.
problem Super-resolution of 360-degree panoramic videos is expensive and challenging.
method Deformable convolutions, mixed attention mechanism, dual learning strategy, weighted mean square error loss function.
result The proposed SMFN method improves super-resolution of equatorial regions in 360-degree videos.
Generative model produces high-fidelity video samples.
problem Generating high-fidelity videos from complex datasets.
method Large GAN trained on Kinetics-600 dataset, using a computationally efficient discriminator.
result Achieved state-of-the-art metrics in video synthesis and prediction.
Generative model learns compact codes for video recovery.
problem Efficiently represent and reconstruct videos from missing data.
method Generative network trained to map compact latent codes to images, with low-rank and similarity constraints.
result Can recover true video sequences even if not in pretrained network's range.
Paper investigates trade-off between video bitrate and CNN accuracy.
problem Network bandwidth and energy consumption issues in video classification.
method Only retain motion vector and select texture information at reduced bitrates.
result Achieved 11%-94% saving in bitrate with marginal accuracy loss.
Paper proposes new principles and framework for AVC learning from user-generated videos.
problem Challenges in learning audio-visual correspondence from short-term user-generated videos.
method Introduced new principles and a framework to facilitate AVC learning from videos' themes.
result Proposed approach outperformed baseline by 23.15% on KWAI-AD-AudVis corpus.
Lower-dimensional video discriminators improve GAN performance.
problem High curvature in unconstrained video discriminator loss surfaces.
method Proposed Lower-Dimensional Video Discriminators (LDVD GANs).
result LDVD GANs double Temporal-GAN performance and achieve state-of-the-art performance.
Deep networks analyze video snippets to predict outcomes, revealing a border effect that can be adjusted for better accuracy.
problem Improving the accuracy of deep networks trained on small video snippets.
method Applied the deep Taylor / LRP technique to understand and identify a border effect, tuning the step size to improve accuracy.
result The step size used to build video snippets can be adjusted to improve deep network accuracy without retraining.
DDPAE predicts video frames by decomposing and disentangling high-dimensional video data.
problem Predicting future video frames from input sequences is challenging due to high-dimensionality.
method Combines structured probabilistic models and deep networks to decompose and disentangle video components.
result DDPAE learns latent decomposition and disentanglement without supervision.
System automates discovery and classification of training videos for career progression.
problem Difficulties in planning and navigating career paths due to changing job requirements and emerging sectors.
method Extracted educational videos, built a machine learning classifier, and optimized probability thresholds.
result Significant improvements in model performance by incorporating video attributes.
Scalable system predicts hot videos for peak VOD service.
problem Improving peak service quality of video on demand.
method Two neural networks: clustering and dispatch policy. Clustering reduces video numbers, dispatch policy ranks videos with probabilities. Networks are trained end-to-end.
result Average prediction accuracy of 17% compared to 3% baseline, for same number of dispatches.
This paper improves video summarization using a new algorithm and dataset.
problem Efficiently summarizing videos for browsing and searching.
method Improves sequential determinantal point process (SeqDPP) with a large-margin algorithm and a new probabilistic distribution.
result Significantly improved video summarization model with better user input integration and diversity.
Paper proposes graph-based separable transforms for video coding.
problem Improving video coding efficiency by better capturing residual block statistics.
method Derives graph-based separable transforms (GBSTs) from line graphs with weights determined by parameters.
result GBSTs achieve about 0.4% average coding gain over existing transforms in VVC.
Jointly trains images and videos using residual vectors.
problem Generating high-quality videos from images and vice versa.
method Simultaneously learns latent variables for images and videos using residual vectors.
result Improves sample quality and diversity in video generation and image generation.
Model-based RL learns Atari games faster with fewer interactions.
problem Efficiently learning Atari games with minimal interactions.
method Simulated Policy Learning (SimPLe) using video prediction models.
result SimPLe outperforms model-free methods in most Atari games with fewer interactions.
Paper introduces adversarial lossy compression for video artifacts reduction.
problem Unpleasant reconstruction artifacts in standard video coding schemes at low bit-rates.
method Adversarial lossy video compression model minimizing an adversarial distortion objective.
result Reduction of perceptual artifacts and detail reconstruction under extreme compression.
This paper highlights the need for explainable AI in video deep learning models.
problem Lack of explainable AI methods for video deep learning models.
method Illustrates the current state of video deep learning and highlights the need for explainability methods.
result Current explainability methods for video deep learning are insufficient and need improvement.
Paper presents a new video generation model using diffusion probabilistic methods.
problem Generating high-quality video sequences.
method Denoising diffusion probabilistic models, autoregressive, end-to-end optimization.
result Significant improvements in perceptual quality and probabilistic frame forecasting.
Human communication takes many forms, including speech, text and instructional videos. It typically has an underlying structure, with a starting point, ending, and certain objective steps between them. In this paper, we consider instructional videos where there are tens of millions of them on the Internet. We propose a…
A video-based re-identification method using attention mechanisms.
problem Associating videos of the same person from different cameras.
method Siamese framework with attention mechanisms for spatial and temporal information.
result Achieves better performance than state-of-the-art on iLIDS-VID dataset.
C3 compresses images and videos with low complexity and high performance.
problem High complexity and low performance in neural compression models.
method Overfits a small model to each image or video separately, improving RD performance with low complexity.
result Matches the RD performance of state-of-the-art neural and video codecs with significantly lower decoding complexity.
Researchers create a flickering attack to fool video recognition networks.
problem Adversarial manipulation of video classification networks.
method Introducing a flickering temporal perturbation to fool video classifiers.
result Achieved high fooling ratio and temporal-invariant perturbation.
AMS improves video inference on edge devices by adapting a small model with online knowledge distillation.
problem High computation cost of Deep Neural Networks for real-time video inference on edge devices.
method AMS uses a remote server to continually train and adapt a small model on edge devices, using online knowledge distillation from a large model.
result 0.4--17.8 percent mean Intersection-over-Union improvement in video semantic segmentation.
New video compression algorithm outperforms existing codecs.
problem Efficient video compression with low latency.
method End-to-end learned video coding architecture.
result Significantly outperforms existing codecs across bitrate ranges.