Scalable system predicts hot videos for peak VOD service.
problem Improving peak service quality of video on demand.
method Two neural networks: clustering and dispatch policy. Clustering reduces video numbers, dispatch policy ranks videos with probabilities. Networks are trained end-to-end.
result Average prediction accuracy of 17% compared to 3% baseline, for same number of dispatches.
VISION-XL improves HD video quality using latent image diffusion models.
problem Improving high-definition video quality and resolution.
method Latent image diffusion models and pseudo-batch consistent sampling.
result State-of-the-art video reconstruction across various inverse problems.
A dataset for evaluating engagement with scientific video lectures.
problem Challenges in managing learning resources due to rapid creation of video lectures.
method Introduction of VLEngagement dataset with content-based and video-specific features, and metrics related to user engagement.
result The largest and most diverse publicly available dataset for understanding context-agnostic engagement in video lectures.
The growing use of virtual autonomous agents in applications like games and entertainment demands better control policies for natural-looking movements and actions. Unlike the conventional approach of hard-coding motion routines, we propose a deep learning method for obtaining control policies by directly mimicking raw…
Deep learning improves video coding efficiency without client-side changes.
problem Improving video coding efficiency without altering existing standards.
method Cascaded downscaling neural networks coupled with adaptive precoding mode selection.
result 15% to 45% rate reduction in FHD and UHD content encoding.
Generates music with video emotion using deep neural networks.
problem Generating music with video emotion.
method Hybrid deep neural network combining ANFIS and LSTM.
result Low mean absolute errors and similar global features in spectrograms.
Recent advances in video super-resolution have shown that convolutional neural networks combined with motion compensation are able to merge information from multiple low-resolution (LR) frames to generate high-quality images. Current state-of-the-art methods process a batch of LR frames to generate a single high-resolu…
New method forecasts multilinear data using tensor autoregression.
problem Forecasting 2D data in big data.
method L-Transform Tensor autoregressive (L-TAR) method.
result Statistical independence achieved through invertible discrete linear transforms.
Detecting faults and SLA violations in a timely manner is critical for telecom providers, in order to avoid loss in business, revenue and reputation. At the same time predicting SLA violations for user services in telecom environments is difficult, due to time-varying user demands and infrastructure load conditions. In…
Limited annotated data available for the recognition of facial expression and action units embarrasses the training of deep networks, which can learn disentangled invariant features. However, a linear model with just several parameters normally is not demanding in terms of training data. In this paper, we propose an el…
Predicts next item in sequential bundles using Transformers.
problem Predicting next item in sequentially consumed bundles.
method Used custom Transformers, GPT-3, LSTM, reinforcement learning, Markov models.
result Custom Transformer with decoder-only architecture most accurate.
We develop a variational framework for SDEs driven by fractional noise.
problem Capturing long-term dependencies in SDEs driven by fractional noise.
method Markov approximation of fractional Brownian motion, variational inference, neural networks.
result Efficient variational inference of posterior path measures for neural-SDEs.
Ground-A-Video edits videos without training, preserving intended changes.
problem Complex multi-attribute video editing with omitted or wrong changes.
method Grounding-guided video-to-video translation with Cross-Frame Gated Attention.
result Zero-shot multi-attribute video editing with improved accuracy and frame consistency.
CB-GLNs learn video data's complex dependencies via graph representation.
problem Capturing complex dependency structures in sequential data like videos.
method Represent video data as a graph, find compositional dependencies via graph-cut and message passing.
result CB-GLNs efficiently learn video data's semantic compositional structure.
RaMViD uses diffusion models for video prediction and infilling.
problem Predicting and infilling missing information in videos.
method Extends image diffusion models to videos using 3D convolutions and a new conditioning technique.
result Achieves state-of-the-art results on video prediction benchmarks.
Current deep learning results on video generation are limited while there are only a few first results on video prediction and no relevant significant results on video completion. This is due to the severe ill-posedness inherent in these three problems. In this paper, we focus on human action videos, and propose a gene…
New metric for video generation models, addressing quality, coherence, and diversity.
problem Lack of metrics for evaluating generative models of video.
method Proposed Fréchet Video Distance (FVD) and StarCraft 2 Videos (SCV) benchmark.
result FVD correlates well with human judgment of generated videos.
LumièreNet creates lecture videos from audio narration.
problem Creating high-quality lecture videos from audio narration.
method Modular deep-learning architecture that learns from audio to video.
result Synthesizes high-quality lecture videos from any length of audio.
Paper defends against adversarial videos by detecting and reducing imperceptible perturbations.
problem Adversarial videos can fool well-trained video classification models.
method Temporal consistency between frames and spatial denoising to detect and reduce perturbations.
result The proposed method significantly improves robustness against adversarial attacks.
Paper improves video categorization using temporal coherence.
problem Video categorization in multiple modalities.
method Temporal coherence-based regularization for multimodal models.
result Models with temporal coherence outperform state-of-the-art.
Improves video search by balancing text and visual modalities.
problem Modality imbalance in video search models, focusing mainly on text matching.
method Proposes MBVR with MS samples and DM to balance modalities.
result Empirically shows significant improvement in modality balance and search effectiveness.
A new video prediction model treats videos as continuous processes, reducing sampling steps and improving efficiency.
problem Efficiency and temporal coherence in video prediction models.
method Treats videos as a continuous multi-dimensional process, reducing sampling steps.
result Reduction of 75% sampling steps, state-of-the-art performance on benchmark datasets.
UDVD uses deep learning to denoise videos without supervision.
problem Lack of clean video data for training deep learning models.
method UDVD is a CNN trained solely on noisy video data, adapting to local motion.
result UDVD performs as well as supervised methods, even with limited training data.
Proposes TDNs for learning complex video structures.
problem Complex temporal dependencies in sequential data, especially videos.
method Temporal Dependency Networks (TDNs) using graph representations and graph convolutions.
result Efficiently learns complex semantic structures of video data.
SummaryNet automates video summarisation using deep learning.
problem Creating informative video summaries from videos.
method Two-stream convolutional network for spatial and temporal features, encoder-decoder model for salient features, sigmoid regression with LSTM for frame probability.
result SummaryNet achieves comparable or better results than state-of-the-art methods on benchmark datasets.
Paper proposes SMFN for high-res spherical video super-resolution.
problem Super-resolution of 360-degree panoramic videos is expensive and challenging.
method Deformable convolutions, mixed attention mechanism, dual learning strategy, weighted mean square error loss function.
result The proposed SMFN method improves super-resolution of equatorial regions in 360-degree videos.
Generative model produces high-fidelity video samples.
problem Generating high-fidelity videos from complex datasets.
method Large GAN trained on Kinetics-600 dataset, using a computationally efficient discriminator.
result Achieved state-of-the-art metrics in video synthesis and prediction.
Generative model learns compact codes for video recovery.
problem Efficiently represent and reconstruct videos from missing data.
method Generative network trained to map compact latent codes to images, with low-rank and similarity constraints.
result Can recover true video sequences even if not in pretrained network's range.
Paper proposes new principles and framework for AVC learning from user-generated videos.
problem Challenges in learning audio-visual correspondence from short-term user-generated videos.
method Introduced new principles and a framework to facilitate AVC learning from videos' themes.
result Proposed approach outperformed baseline by 23.15% on KWAI-AD-AudVis corpus.
The extension of image generation to video generation turns out to be a very difficult task, since the temporal dimension of videos introduces an extra challenge during the generation process. Besides, due to the limitation of memory and training stability, the generation becomes increasingly challenging with the incre…
Lower-dimensional video discriminators improve GAN performance.
problem High curvature in unconstrained video discriminator loss surfaces.
method Proposed Lower-Dimensional Video Discriminators (LDVD GANs).
result LDVD GANs double Temporal-GAN performance and achieve state-of-the-art performance.
System automates discovery and classification of training videos for career progression.
problem Difficulties in planning and navigating career paths due to changing job requirements and emerging sectors.
method Extracted educational videos, built a machine learning classifier, and optimized probability thresholds.
result Significant improvements in model performance by incorporating video attributes.
With the growth of user-generated content, we observe the constant rise of the number of companies, such as search engines, content aggregators, etc., that operate with tremendous amounts of web content not being the services hosting it. Thus, aiming to locate the most important content and promote it to the users, the…
Paper proposes graph-based separable transforms for video coding.
problem Improving video coding efficiency by better capturing residual block statistics.
method Derives graph-based separable transforms (GBSTs) from line graphs with weights determined by parameters.
result GBSTs achieve about 0.4% average coding gain over existing transforms in VVC.
Jointly trains images and videos using residual vectors.
problem Generating high-quality videos from images and vice versa.
method Simultaneously learns latent variables for images and videos using residual vectors.
result Improves sample quality and diversity in video generation and image generation.
Paper proposes detecting video manipulation using stream descriptors.
problem Misuse of manipulated video content.
method Binary classifiers on multimedia stream descriptors.
result Scalable approach can detect high-quality manipulations.
Paper introduces adversarial lossy compression for video artifacts reduction.
problem Unpleasant reconstruction artifacts in standard video coding schemes at low bit-rates.
method Adversarial lossy video compression model minimizing an adversarial distortion objective.
result Reduction of perceptual artifacts and detail reconstruction under extreme compression.
Advanced video classification systems decode video frames to derive the necessary texture and motion representations for ingestion and analysis by spatio-temporal deep convolutional neural networks (CNNs). However, when considering visual Internet-of-Things applications, surveillance systems and semantic crawlers of la…
A framework uses deep learning for activity recognition in IoT devices.
problem Activity recognition in IoT devices without physical contact.
method Background subtraction followed by 3D-Convolutional Neural Networks.
result Enhanced activity recognition using small IoT devices.
This paper highlights the need for explainable AI in video deep learning models.
problem Lack of explainable AI methods for video deep learning models.
method Illustrates the current state of video deep learning and highlights the need for explainability methods.
result Current explainability methods for video deep learning are insufficient and need improvement.
Paper presents a new video generation model using diffusion probabilistic methods.
problem Generating high-quality video sequences.
method Denoising diffusion probabilistic models, autoregressive, end-to-end optimization.
result Significant improvements in perceptual quality and probabilistic frame forecasting.
Human communication takes many forms, including speech, text and instructional videos. It typically has an underlying structure, with a starting point, ending, and certain objective steps between them. In this paper, we consider instructional videos where there are tens of millions of them on the Internet. We propose a…
A video-based re-identification method using attention mechanisms.
problem Associating videos of the same person from different cameras.
method Siamese framework with attention mechanisms for spatial and temporal information.
result Achieves better performance than state-of-the-art on iLIDS-VID dataset.
C3 compresses images and videos with low complexity and high performance.
problem High complexity and low performance in neural compression models.
method Overfits a small model to each image or video separately, improving RD performance with low complexity.
result Matches the RD performance of state-of-the-art neural and video codecs with significantly lower decoding complexity.
Researchers create a flickering attack to fool video recognition networks.
problem Adversarial manipulation of video classification networks.
method Introducing a flickering temporal perturbation to fool video classifiers.
result Achieved high fooling ratio and temporal-invariant perturbation.
AMS improves video inference on edge devices by adapting a small model with online knowledge distillation.
problem High computation cost of Deep Neural Networks for real-time video inference on edge devices.
method AMS uses a remote server to continually train and adapt a small model on edge devices, using online knowledge distillation from a large model.
result 0.4--17.8 percent mean Intersection-over-Union improvement in video semantic segmentation.
New video compression algorithm outperforms existing codecs.
problem Efficient video compression with low latency.
method End-to-end learned video coding architecture.
result Significantly outperforms existing codecs across bitrate ranges.
The paper solves video object segmentation without supervision using nonconvex optimization.
problem Unsupervised video object segmentation via background subtraction.
method Formulates the problem as a nonnegative variant of robust principal component analysis, ensuring global optimality under certain conditions.
result Conditions guaranteeing the uniqueness and global optimality of object segmentation are derived and demonstrated with real data.