Paper proposes graph-based separable transforms for video coding.
problem Improving video coding efficiency by better capturing residual block statistics.
method Derives graph-based separable transforms (GBSTs) from line graphs with weights determined by parameters.
result GBSTs achieve about 0.4% average coding gain over existing transforms in VVC.
New deep-unfolded network improves video background separation.
problem Video foreground-background separation.
method Deep unfolding of an iterative RPCA algorithm with adaptive learning.
result Proposed network outperforms state-of-the-art in video foreground-background separation.
Novel algorithm for separating moving camera video into static and dynamic components.
problem Foreground-background separation in noisy, moving camera video.
method Augmented robust PCA with total variation regularization, OptShrink low-rank matrix estimator.
result Panoramic low-rank component spanning entire field of view, automatically stitching corrupted data.
New method for separating foreground from background in noisy, moving camera video.
problem Foreground-background separation in noisy, free-moving camera video.
method Registers frames, encodes perspective as missing data, uses OptShrink for low-rank estimation, and weighted total variation for smooth foreground.
result Panoramic background component that stitches together corrupted data from overlapping frames.
This paper separates static and dynamic features in video data.
problem Combining static and dynamic features in video data.
method Hierarchical Variational Auto-encoders with factored prior distributions.
result The model successfully separates static and dynamic features.
Proposes online compressive robust PCA with prior info.
problem Online video foreground-background separation from sparse and low-rank components.
method Combines prior info and compressive sensing for recursive separation.
result Improves separation using multiple prior info and theoretical bounds.
SRTC model for background/foreground separation with missing pixels.
problem Background/foreground separation with missing pixels in videos.
method Smooth robust tensor completion (SRTC) model with tensor proximal alternating minimization (tenPAM).
result Global convergence guarantee for the proposed algorithm.
C3 compresses images and videos with low complexity and high performance.
problem High complexity and low performance in neural compression models.
method Overfits a small model to each image or video separately, improving RD performance with low complexity.
result Matches the RD performance of state-of-the-art neural and video codecs with significantly lower decoding complexity.
This paper presents GRASTA (Grassmannian Robust Adaptive Subspace Tracking Algorithm), an efficient and robust online algorithm for tracking subspaces from highly incomplete information. The algorithm uses a robust l1-norm cost function in order to estimate and track non-stationary subspaces when the streaming data …
New method learns agent actions from raw video demonstrations.
problem Imitation learning for natural-looking movements.
method Generative Adversarial Networks (GANs) for joint state and reward learning.
result Adversarial imitation learning on raw videos produces similar performance to state-of-the-art methods.
Unsupervised model separates appearance and geometry from images and videos.
problem Disentangling appearance and geometry from images and videos without supervision.
method Deformable generator network with two independent latent inputs for appearance and geometry.
result The model successfully disentangles appearance and geometry from images and videos.
SVM with local features improves human action recognition.
problem Improving human action recognition in videos.
method Local appearance and motion features extracted using CNNs, concatenated, and used with SVM for classification.
result SVM with local features outperforms previous methods on benchmark datasets.
New framework detects anomalies in videos without training data.
problem Detect anomalies in videos without training data or temporal context.
method Discriminative learning approach, independent of anomaly order.
result Achieves state-of-the-art results on anomaly detection.
Visual reranking is effective to improve the performance of the text-based video search. However, existing reranking algorithms can only achieve limited improvement because of the well-known semantic gap between low level visual features and high level semantic concepts. In this paper, we adopt interactive video search…
ASTPN improves video-based person re-identification by jointly attending to spatial and temporal features.
problem Video-based person re-identification in surveillance and HCI.
method Joint Spatial and Temporal Attention Pooling Network (ASTPN).
result ASTPN outperforms state-of-the-art methods on multiple datasets.
New algorithm rSVDdpd improves robustness and scalability for video surveillance background modeling.
problem Camera tampering and noisy videos make background separation challenging.
method Introduces rSVDdpd, a robust singular value decomposition technique for scalable video surveillance.
result Demonstrates superior performance on benchmark and real-life datasets.
New architecture separates object state and behavior for better game dynamics.
problem Lack of systematicity in black-box models for dynamic environments.
method Factorizes declarative and procedural knowledge into object files and schemata.
result Improves generalization on environments with multiple object tokens.
Paper introduces graph-based transforms for video compression.
problem Efficiently represent video signals for compression.
method Develops two techniques for designing graph-based transforms (GL-GBTs and EA-GBTs).
result Graph-based transforms outperform traditional KLT in video compression.
End-to-end frame-recurrent video super-resolution reduces computational cost and improves consistency.
problem Efficiently super-resolving video frames while maintaining temporal consistency and reducing computational cost.
method Proposes an end-to-end trainable frame-recurrent video super-resolution framework that uses previous HR estimates to super-resolve subsequent frames.
result Significantly outperforms state-of-the-art methods in computational efficiency and temporal consistency.
New algorithms separate singing voices from accompaniment using complex and quaternionic principal component pursuit.
problem Separating singing voices from instrumental accompaniment using phase information.
method Extended principal component pursuit to complex and quaternionic cases, developed new proximity operators, applied inexact augmented Lagrange multiplier algorithm.
result Phase information improves singing voice separation.
Two-stream model recognizes affect from audio and video.
problem Human affect recognition in real-world settings.
method Two-stream aural-visual analysis model with separate audio and visual processing.
result Model achieves promising results on Aff-Wild2 database.
A new method exposes motion-related relevance in video frames.
problem Deconstructing relevance in spatio-temporal models for video processing.
method Proposes a discriminative method to separate spatial and temporal relevance.
result Demonstrates effectiveness on UCF-101 action recognition dataset.
Ground-A-Video edits videos without training, preserving intended changes.
problem Complex multi-attribute video editing with omitted or wrong changes.
method Grounding-guided video-to-video translation with Cross-Frame Gated Attention.
result Zero-shot multi-attribute video editing with improved accuracy and frame consistency.
Generates, predicts, and completes human action videos with a two-stage deep framework.
problem Severe ill-posedness in video generation, prediction, and completion.
method Two-stage deep framework: 1) Generates human pose sequence from noise, 2) Converts pose sequence to video.
result Produces high-quality video generation/prediction/completion results of longer duration.
CB-GLNs learn video data's complex dependencies via graph representation.
problem Capturing complex dependency structures in sequential data like videos.
method Represent video data as a graph, find compositional dependencies via graph-cut and message passing.
result CB-GLNs efficiently learn video data's semantic compositional structure.
RaMViD uses diffusion models for video prediction and infilling.
problem Predicting and infilling missing information in videos.
method Extends image diffusion models to videos using 3D convolutions and a new conditioning technique.
result Achieves state-of-the-art results on video prediction benchmarks.
New metric for video generation models, addressing quality, coherence, and diversity.
problem Lack of metrics for evaluating generative models of video.
method Proposed Fréchet Video Distance (FVD) and StarCraft 2 Videos (SCV) benchmark.
result FVD correlates well with human judgment of generated videos.
LumièreNet creates lecture videos from audio narration.
problem Creating high-quality lecture videos from audio narration.
method Modular deep-learning architecture that learns from audio to video.
result Synthesizes high-quality lecture videos from any length of audio.
Paper defends against adversarial videos by detecting and reducing imperceptible perturbations.
problem Adversarial videos can fool well-trained video classification models.
method Temporal consistency between frames and spatial denoising to detect and reduce perturbations.
result The proposed method significantly improves robustness against adversarial attacks.
Paper improves video categorization using temporal coherence.
problem Video categorization in multiple modalities.
method Temporal coherence-based regularization for multimodal models.
result Models with temporal coherence outperform state-of-the-art.
Improves video search by balancing text and visual modalities.
problem Modality imbalance in video search models, focusing mainly on text matching.
method Proposes MBVR with MS samples and DM to balance modalities.
result Empirically shows significant improvement in modality balance and search effectiveness.
DSSM separates domain-invariant dynamics from domain-specifics in sequential data.
problem Learning cross-domain sequence representations from diverse data domains.
method Introduce disentangled state space models (DSSM) using unsupervised VAE-based training.
result Improves knowledge transfer and robust prediction across domains.
A new video prediction model treats videos as continuous processes, reducing sampling steps and improving efficiency.
problem Efficiency and temporal coherence in video prediction models.
method Treats videos as a continuous multi-dimensional process, reducing sampling steps.
result Reduction of 75% sampling steps, state-of-the-art performance on benchmark datasets.
UDVD uses deep learning to denoise videos without supervision.
problem Lack of clean video data for training deep learning models.
method UDVD is a CNN trained solely on noisy video data, adapting to local motion.
result UDVD performs as well as supervised methods, even with limited training data.
Paper predicts video popularity from web traces, not hosting services.
problem Predicting video popularity from web traces.
method Combining embed/link data and internal search logs to predict video popularity.
result Embed and link data can significantly improve video popularity prediction.
This work studies two interrelated problems - online robust PCA (RPCA) and online low-rank matrix completion (MC). In recent work by Candès et al., RPCA has been defined as a problem of separating a low-rank matrix (true data), L:=[ℓ1,ℓ2,…ℓt,…,ℓtmax] and a sparse matrix (outliers…
Generative model for high-resolution video generation.
problem Challenges in generating high-resolution videos due to memory and training stability limitations.
method Progressive growing of sliced Wasserstein GANs (SWGAN) for incremental spatiotemporal information learning.
result Generated photorealistic face videos of 256x256x32 resolution with an inception score of 14.57.
Proposes TDNs for learning complex video structures.
problem Complex temporal dependencies in sequential data, especially videos.
method Temporal Dependency Networks (TDNs) using graph representations and graph convolutions.
result Efficiently learns complex semantic structures of video data.
SummaryNet automates video summarisation using deep learning.
problem Creating informative video summaries from videos.
method Two-stream convolutional network for spatial and temporal features, encoder-decoder model for salient features, sigmoid regression with LSTM for frame probability.
result SummaryNet achieves comparable or better results than state-of-the-art methods on benchmark datasets.
Paper proposes SMFN for high-res spherical video super-resolution.
problem Super-resolution of 360-degree panoramic videos is expensive and challenging.
method Deformable convolutions, mixed attention mechanism, dual learning strategy, weighted mean square error loss function.
result The proposed SMFN method improves super-resolution of equatorial regions in 360-degree videos.
Generative model produces high-fidelity video samples.
problem Generating high-fidelity videos from complex datasets.
method Large GAN trained on Kinetics-600 dataset, using a computationally efficient discriminator.
result Achieved state-of-the-art metrics in video synthesis and prediction.
Generative model learns compact codes for video recovery.
problem Efficiently represent and reconstruct videos from missing data.
method Generative network trained to map compact latent codes to images, with low-rank and similarity constraints.
result Can recover true video sequences even if not in pretrained network's range.
Paper investigates trade-off between video bitrate and CNN accuracy.
problem Network bandwidth and energy consumption issues in video classification.
method Only retain motion vector and select texture information at reduced bitrates.
result Achieved 11%-94% saving in bitrate with marginal accuracy loss.
Paper proposes new principles and framework for AVC learning from user-generated videos.
problem Challenges in learning audio-visual correspondence from short-term user-generated videos.
method Introduced new principles and a framework to facilitate AVC learning from videos' themes.
result Proposed approach outperformed baseline by 23.15% on KWAI-AD-AudVis corpus.
5th place in video labeling challenge using deep learning.
problem Efficient large-scale video labeling.
method Ensemble of frame and video level models trained on augmented dataset with cross-validation.
result Ranked 5th in the Google Cloud and YouTube-8M Video Understanding Challenge.
Lower-dimensional video discriminators improve GAN performance.
problem High curvature in unconstrained video discriminator loss surfaces.
method Proposed Lower-Dimensional Video Discriminators (LDVD GANs).
result LDVD GANs double Temporal-GAN performance and achieve state-of-the-art performance.
Deep networks analyze video snippets to predict outcomes, revealing a border effect that can be adjusted for better accuracy.
problem Improving the accuracy of deep networks trained on small video snippets.
method Applied the deep Taylor / LRP technique to understand and identify a border effect, tuning the step size to improve accuracy.
result The step size used to build video snippets can be adjusted to improve deep network accuracy without retraining.
PyKale bridges interdisciplinary ML with Python, enabling accurate predictions.
problem Cross-disciplinary barriers in machine learning.
method Knowledge-aware machine learning on graphs, images, texts, and videos.
result Enables multimodal learning and transfer learning with latest deep learning models.