Study shows image classifiers struggle with temporal changes in videos.
problem Temporal robustness of image classifiers in videos.
method Constructed datasets and evaluated various classifiers and detection models.
result Median classification accuracy drops of 16-10 points in robust datasets.
Mixtures of Gaussians, factor analyzers (probabilistic PCA) and hidden Markov models are staples of static and dynamic data modeling and image and video modeling in particular. We show how topographic transformations in the input, such as translation and shearing in images, can be accounted for in these models by inclu…
Paper improves video categorization using temporal coherence.
problem Video categorization in multiple modalities.
method Temporal coherence-based regularization for multimodal models.
result Models with temporal coherence outperform state-of-the-art.
Jointly trains images and videos using residual vectors.
problem Generating high-quality videos from images and vice versa.
method Simultaneously learns latent variables for images and videos using residual vectors.
result Improves sample quality and diversity in video generation and image generation.
TrackNet tracks tiny balls in sports videos with high precision.
problem Accurately tracking tiny and fast-moving objects like tennis balls in sports videos.
method Developed a heatmap-based deep learning network, TrackNet, trained on public and additional labeled videos.
result Achieved high precision (99.7%) in ball tracking on public domain videos.
A new video prediction model treats videos as continuous processes, reducing sampling steps and improving efficiency.
problem Efficiency and temporal coherence in video prediction models.
method Treats videos as a continuous multi-dimensional process, reducing sampling steps.
result Reduction of 75% sampling steps, state-of-the-art performance on benchmark datasets.
Paper presents a method for recognizing human actions using GLAC features from motion and static images.
problem Action recognition in 3D depth videos.
method 3D Motion Trail Model (3DMTM) for MHIs and SHIs, GLAC features extraction, l2-regularized Collaborative Representation Classifier (l2-CRC) for classification.
result The method outperforms other approaches in recognizing human actions.
The paper tackles video prediction by estimating conditional densities implicitly.
problem Temporal prediction uncertainty and high-dimensional probabilistic inference in natural scenes.
method Score-based conditional density estimation using sequence-to-image networks trained on a resilience-to-noise objective.
result The method handles occlusion boundaries and weights predictive evidence by reliability.
C3 compresses images and videos with low complexity and high performance.
problem High complexity and low performance in neural compression models.
method Overfits a small model to each image or video separately, improving RD performance with low complexity.
result Matches the RD performance of state-of-the-art neural and video codecs with significantly lower decoding complexity.
New method tackles video inverse problems using image diffusion models.
problem Spatio-temporal degradation in video inverse problems.
method Leverages image diffusion models to treat time dimension as batch dimension, introduces batch-consistent diffusion sampling.
result Achieves state-of-the-art reconstructions for various spatio-temporal degradations.
Unified model trained on images and videos using masked autoencoding.
problem Training a single model for multiple visual modalities.
method Masked autoencoding on a Vision Transformer.
result Unified model achieves comparable or better performance than single-modality models.
LVTINO improves high-definition video restoration with consistent temporal details.
problem Restoring high-definition video with fine spatial detail and temporal consistency.
method LVTINO uses Video Consistency Models to bypass frame-by-frame image-based LDMs, achieving perceptual quality and computational efficiency.
result Significant perceptual improvements over current methods, achieving state-of-the-art quality with minimal evaluations.
UDVD uses deep learning to denoise videos without supervision.
problem Lack of clean video data for training deep learning models.
method UDVD is a CNN trained solely on noisy video data, adapting to local motion.
result UDVD performs as well as supervised methods, even with limited training data.
This paper highlights the need for explainable AI in video deep learning models.
problem Lack of explainable AI methods for video deep learning models.
method Illustrates the current state of video deep learning and highlights the need for explainability methods.
result Current explainability methods for video deep learning are insufficient and need improvement.
VISION-XL improves HD video quality using latent image diffusion models.
problem Improving high-definition video quality and resolution.
method Latent image diffusion models and pseudo-batch consistent sampling.
result State-of-the-art video reconstruction across various inverse problems.
New video compression method outperforms traditional approaches.
problem Efficient video compression in low latency mode.
method Feedback Recurrent Autoencoder network architecture.
result State of the art MS-SSIM/rate performance on UVG dataset.
Deep learning animates objects from input images and videos.
problem Animating arbitrary objects from input images and videos.
method A deep learning framework with three modules: Keypoint Detector, Dense Motion prediction network, and Motion Transfer Network.
result Our method outperforms state-of-the-art image animation and video generation methods.
TokenCut detects and segments objects in images and videos without supervision.
problem Detecting and segmenting salient objects in images and videos without labeled data.
method Graph-based approach using self-supervised transformer features and Normalized Cut algorithm.
result Achieves state-of-the-art results on various detection and segmentation tasks.
Paper develops real-time through-wall human pose imaging using RF signals.
problem Developing real-time vision through opaque objects.
method Many-to-Many Encoder/Decoder Paradigm, student/teacher learning, Residual CNN, RPN, LSTM.
result Accurately predicts human pose through visual occlusion using RF signals.
UniFinEval benchmarks financial models across text, images, and videos.
problem Challenges in evaluating financial multimodal models across text, images, and videos.
method Proposes UniFinEval, a unified multimodal benchmark for financial scenarios.
result Gemini-3-pro-preview achieves best performance but still lags behind experts.
Paper introduces DACAL for high-resolution photo and video enhancement.
problem Photo and video enhancement with weak supervision.
method Divide-and-conquer adversarial learning approach with hierarchical decomposition.
result State-of-the-art performance in high-resolution photo and video enhancement.
Proposes a method to add adversarial framing to images without changing them.
problem Neural networks are vulnerable to adversarial attacks.
method Adds adversarial framing to the border of images without altering their content.
result Successfully attacks state-of-the-art methods on image and video classification.
V-BAD is a first black-box video attack framework that successfully fools deep video recognition models.
problem Vulnerability of video recognition models to black-box adversarial attacks.
method Tentative perturbations transferred from image models and partition-based rectifications for patches of tentative perturbations.
result V-BAD can craft both untargeted and targeted attacks with high success rates using a manageable number of queries.
Unsupervised fire and smoke segmentation from IR videos.
problem Early detection of fire from infrared videos.
method Spatial, temporal, and motion information fusion using optical flow, divergence, and intensity values.
result Markov Random Field outperforms other clustering algorithms in fire and smoke segmentation.
Recent advances in deep generative models have lead to remarkable progress in synthesizing high quality images. Following their successful application in image processing and representation learning, an important next step is to consider videos. Learning generative models of video is a much harder task, requiring a mod…
Automatic video modification to hide faces while maintaining pose, illumination, and expression.
problem Face de-identification in video to protect identities.
method A novel feed-forward encoder-decoder network architecture conditioned on facial image high-level representation.
result Fully automatic video modification at high frame rates with minimal distortion.
Sparse coding is a common approach to learning local features for object recognition. Recently, there has been an increasing interest in learning features from spatio-temporal, binocular, or other multi-observation data, where the goal is to encode the relationship between images rather than the content of a single ima…
New method compresses facial videos using GANs and latent space optimization.
problem Efficiently compressing facial videos at low bit rates.
method Leverages StyleGAN for latent space representation and compression, learns optimal compression through entropy model and perceptual loss.
result Significantly reduces perceptual distortion at low bit rates compared to state-of-the-art codecs.
Making predictions of future frames is a critical challenge in autonomous driving research. Most of the existing methods for video prediction attempt to generate future frames in simple and fixed scenes. In this paper, we propose a novel and effective optical flow conditioned method for the task of video prediction wit…
Researchers create a flickering attack to fool video recognition networks.
problem Adversarial manipulation of video classification networks.
method Introducing a flickering temporal perturbation to fool video classifiers.
result Achieved high fooling ratio and temporal-invariant perturbation.
Two-stream model recognizes affect from audio and video.
problem Human affect recognition in real-world settings.
method Two-stream aural-visual analysis model with separate audio and visual processing.
result Model achieves promising results on Aff-Wild2 database.
Detect objects from motion without annotations.
problem Weakly supervised object detection.
method Train model on videos of moving objects and negative scenes.
result Detects objects in single images without annotations.
A new method for effective VAE training using calibrated decoders.
problem Training VAEs requires hyperparameter tuning, leading to inefficiency.
method Calibrated decoders that learn uncertainty and automatically determine information retention.
result Calibrated decoders can simplify VAE training without heuristic modifications.
We present a deformable generator model to disentangle the appearance and geometric information for both image and video data in a purely unsupervised manner. The appearance generator network models the information related to appearance, including color, illumination, identity or category, while the geometric generator…
The usage of deep generative models for image compression has led to impressive performance gains over classical codecs while neural video compression is still in its infancy. Here, we propose an end-to-end, deep generative modeling approach to compress temporal sequences with a focus on video. Our approach builds upon…
Deep neural networks have been developed drawing inspiration from the brain visual pathway, implementing an end-to-end approach: from image data to video object classes. However building an fMRI decoder with the typical structure of Convolutional Neural Network (CNN), i.e. learning multiple level of representations, se…
Generative model learns to navigate image manifolds for unsupervised video generation.
problem Generating natural-looking scenes of moving objects without labeled data.
method Two-step training scheme with GANs, using a recurrent model to generate coherent sequences.
result Recurrent model learns to navigate the image manifold induced by the frames generator, producing more natural scenes.
Recognizing facial expressions from static images or video sequences is a widely studied but still challenging problem. The recent progresses obtained by deep neural architectures, or by ensembles of heterogeneous models, have shown that integrating multiple input representations leads to state-of-the-art results. In p…
Proposes a defense method against multiple adversarial video types.
problem Lack of multi-perturbation robustness in existing defense approaches.
method Adversarial training with multiple independent BN layers and a BN selection module.
result Demonstrates stronger multi-perturbation robustness against different adversarial video types.
New video super-resolution method robust to multiple degradation models.
problem Current methods are not robust to mismatch between training and testing degradation models.
method Convolutional Neural Network with pseudo-inverse image formation model and perceptual losses.
result Outperforms current state-of-the-art methods and is robust to multiple degradations.
SCNN improves video object detection speed by 178%.
problem Limited throughput in existing CNNs for video object detection.
method Proposes SCNN, a statistical CNN that processes correlated distributions.
result Achieves 178% speedup over existing CNNs for video object detection.
This work predicts and interpolates long-range videos using unsupervised landmarks.
problem Predicting and interpolating long-range video data with occlusions and appearance changes.
method Unsupervised latent structure inference followed by temporal prediction in a latent space.
result High-quality long-range video interpolation and extrapolation achieved through landmark representation.
Recent progress in using recurrent neural networks (RNNs) for image description has motivated the exploration of their application for video description. However, while images are static, working with videos requires modeling their dynamic temporal structure and then properly integrating that information into a natural…
Stochastic video prediction models take in a sequence of image frames, and generate a sequence of consecutive future image frames. These models typically generate future frames in an autoregressive fashion, which is slow and requires the input and output frames to be consecutive. We introduce a model that overcomes the…
Paper proposes detecting video manipulation using stream descriptors.
problem Misuse of manipulated video content.
method Binary classifiers on multimedia stream descriptors.
result Scalable approach can detect high-quality manipulations.
Learning to predict future images from a video sequence involves the construction of an internal representation that models the image evolution accurately, and therefore, to some degree, its content and dynamics. This is why pixel-space video prediction may be viewed as a promising avenue for unsupervised feature learn…
Simple method detects deepfakes with few labeled samples.
problem Automated detection of AI-generated fake images.
method Frequency domain analysis followed by basic classifier.
result Achieved 100% accuracy in high-resolution face images with minimal training data.
Generative model produces high-fidelity video samples.
problem Generating high-fidelity videos from complex datasets.
method Large GAN trained on Kinetics-600 dataset, using a computationally efficient discriminator.
result Achieved state-of-the-art metrics in video synthesis and prediction.