A CNN method removes atmospheric distortion from video sequences.
problem Mitigating atmospheric distortion in video sequences.
method End-to-end supervised convolutional neural network (CNN) with residual learning.
result The method outperforms existing techniques in terms of image quality and speed.
VISION-XL improves HD video quality using latent image diffusion models.
problem Improving high-definition video quality and resolution.
method Latent image diffusion models and pseudo-batch consistent sampling.
result State-of-the-art video reconstruction across various inverse problems.
New method tackles video inverse problems using image diffusion models.
problem Spatio-temporal degradation in video inverse problems.
method Leverages image diffusion models to treat time dimension as batch dimension, introduces batch-consistent diffusion sampling.
result Achieves state-of-the-art reconstructions for various spatio-temporal degradations.
Physen-Noise2Noise tackles defocus deblurring in low-light conditions with physics-guided self-supervised learning.
problem Defocus deblurring in low-light conditions with complex biased noise.
method Physics-guided self-supervised deblurring framework that leverages noisy multi-frame observations and a learnable noise bias parameter.
result Physen-Noise2Noise consistently outperforms state-of-the-art methods in defocus deblurring with complex biased noise.
Blind image deblurring remains a topic of enduring interest. Learning based approaches, especially those that employ neural networks have emerged to complement traditional model based methods and in many cases achieve vastly enhanced performance. That said, neural network approaches are generally empirically designed a…
Novel algorithm accelerates PnP methods for image deblurring and super-resolution.
problem Efficiently solving inverse problems and imaging with provable convergence guarantees.
method Incorporates quasi-Newton steps into provable PnP framework based on proximal denoisers.
result 2--8x faster convergence compared to other provable PnP methods with similar quality.
While neural networks have achieved vastly enhanced performance over traditional iterative methods in many cases, they are generally empirically designed and the underlying structures are difficult to interpret. The algorithm unrolling approach has helped connect iterative algorithms to neural network architectures. Ho…
New method tackles tensor regression with robust Kaczmarz approach.
problem Reconstructing tensor signals from corrupted measurements.
method Quantile-based randomized Kaczmarz method for tensor linear systems.
result Improved convergence and robustness to adversarial corruptions.
New method improves image generation for inverse problems using text prompts.
problem Suboptimal performance of existing latent diffusion models for inverse problems.
method Prompt tuning and latent variable projection to optimize text embeddings.
result P2L method outperforms existing methods on various inverse problem tasks.
Generative mixture models of VAEs learn manifolds for inverse problems.
problem Representing high-dimensional data manifolds efficiently and accurately.
method Mixture model of variational autoencoders (VAEs) with Riemannian gradient descent.
result Learned manifold enables solving inverse problems with data fidelity.
The paper studies sparsity in EBF with hyperpriors and proposes a PALM algorithm.
problem Promoting sparsity in sparse learning problems.
method Empirical Bayes framework, hyperpriors, proximal alternating linearized minimization (PALM) algorithm.
result Appropriate hyperpriors can significantly enhance sparsity and restoration accuracy.
A fast method approximates likelihood scores for noisy linear inverse problems.
problem Solving noisy linear inverse problems efficiently.
method Proposes a simple closed-form approximation to the likelihood score for diffusion and flow-based models.
result Significantly faster than baseline methods while maintaining competitive or better reconstruction performances.
New method solves blind inverse problems by optimizing both operator and image parameters.
problem Solving blind inverse problems with known forward operator.
method Parallel reverse diffusion guided by gradients from intermediate stages.
result State-of-the-art performance on blind deblurring and imaging through turbulence.
Ground-A-Video edits videos without training, preserving intended changes.
problem Complex multi-attribute video editing with omitted or wrong changes.
method Grounding-guided video-to-video translation with Cross-Frame Gated Attention.
result Zero-shot multi-attribute video editing with improved accuracy and frame consistency.
BP-DIP combines DIP and backprojection for image restoration.
problem Performance drop of deep learning methods due to mismatched observation models.
method Combines Deep Image Prior (DIP) and backprojection.
result BP-DIP outperforms DIP in deblurring tasks with higher PSNR and faster inference.
A neural network learns a convex regularizer for better image reconstruction.
problem Improving image reconstruction in inverse problems.
method Adversarial training of a data-adaptive ICNN as a convex regularizer.
result The convex regularizer leads to better convergence and error reduction in image reconstruction.
Solving inverse problems continues to be a challenge in a wide array of applications ranging from deblurring, image inpainting, source separation etc. Most existing techniques solve such inverse problems by either explicitly or implicitly finding the inverse of the model. The former class of techniques require explicit…
CB-GLNs learn video data's complex dependencies via graph representation.
problem Capturing complex dependency structures in sequential data like videos.
method Represent video data as a graph, find compositional dependencies via graph-cut and message passing.
result CB-GLNs efficiently learn video data's semantic compositional structure.
Improved image restoration using frequency-guided sampling.
problem Restoring high-quality images from degraded observations with known degradation processes.
method Proposed a frequency-guided sampling approach for diffusion-based image restoration, incorporating a time-varying low-pass filter.
result Significantly improved performance on challenging image restoration tasks, including motion deblurring and image dehazing.
RaMViD uses diffusion models for video prediction and infilling.
problem Predicting and infilling missing information in videos.
method Extends image diffusion models to videos using 3D convolutions and a new conditioning technique.
result Achieves state-of-the-art results on video prediction benchmarks.
Current deep learning results on video generation are limited while there are only a few first results on video prediction and no relevant significant results on video completion. This is due to the severe ill-posedness inherent in these three problems. In this paper, we focus on human action videos, and propose a gene…
Paper improves video categorization using temporal coherence.
problem Video categorization in multiple modalities.
method Temporal coherence-based regularization for multimodal models.
result Models with temporal coherence outperform state-of-the-art.
Improves video search by balancing text and visual modalities.
problem Modality imbalance in video search models, focusing mainly on text matching.
method Proposes MBVR with MS samples and DM to balance modalities.
result Empirically shows significant improvement in modality balance and search effectiveness.
A new video prediction model treats videos as continuous processes, reducing sampling steps and improving efficiency.
problem Efficiency and temporal coherence in video prediction models.
method Treats videos as a continuous multi-dimensional process, reducing sampling steps.
result Reduction of 75% sampling steps, state-of-the-art performance on benchmark datasets.
UDVD uses deep learning to denoise videos without supervision.
problem Lack of clean video data for training deep learning models.
method UDVD is a CNN trained solely on noisy video data, adapting to local motion.
result UDVD performs as well as supervised methods, even with limited training data.
SummaryNet automates video summarisation using deep learning.
problem Creating informative video summaries from videos.
method Two-stream convolutional network for spatial and temporal features, encoder-decoder model for salient features, sigmoid regression with LSTM for frame probability.
result SummaryNet achieves comparable or better results than state-of-the-art methods on benchmark datasets.
DPMC improves inverse problem solving with MCMC, reducing error in noisy conditions.
problem Inaccurate posterior approximation in inverse problems with high noise levels.
method DPMC uses Annealed MCMC to sample through a series of intermediate distributions, reducing accumulated error.
result DPMC outperforms DPS in various inverse problems, reducing error and evaluations.
Paper proposes SMFN for high-res spherical video super-resolution.
problem Super-resolution of 360-degree panoramic videos is expensive and challenging.
method Deformable convolutions, mixed attention mechanism, dual learning strategy, weighted mean square error loss function.
result The proposed SMFN method improves super-resolution of equatorial regions in 360-degree videos.
Paper proposes new principles and framework for AVC learning from user-generated videos.
problem Challenges in learning audio-visual correspondence from short-term user-generated videos.
method Introduced new principles and a framework to facilitate AVC learning from videos' themes.
result Proposed approach outperformed baseline by 23.15% on KWAI-AD-AudVis corpus.
The extension of image generation to video generation turns out to be a very difficult task, since the temporal dimension of videos introduces an extra challenge during the generation process. Besides, due to the limitation of memory and training stability, the generation becomes increasingly challenging with the incre…
Recent advances in deep generative models have lead to remarkable progress in synthesizing high quality images. Following their successful application in image processing and representation learning, an important next step is to consider videos. Learning generative models of video is a much harder task, requiring a mod…
Video classification is a challenging task in computer vision. Although Deep Neural Networks (DNNs) have achieved excellent performance in video classification, recent research shows adding imperceptible perturbations to clean videos can make the well-trained models output wrong labels with high confidence. In this pap…
Lower-dimensional video discriminators improve GAN performance.
problem High curvature in unconstrained video discriminator loss surfaces.
method Proposed Lower-Dimensional Video Discriminators (LDVD GANs).
result LDVD GANs double Temporal-GAN performance and achieve state-of-the-art performance.
Conventional sequential learning methods such as Recurrent Neural Networks (RNNs) focus on interactions between consecutive inputs, i.e. first-order Markovian dependency. However, most of sequential data, as seen with videos, have complex temporal dependencies that imply variable-length semantic flows and their composi…
With the growth of user-generated content, we observe the constant rise of the number of companies, such as search engines, content aggregators, etc., that operate with tremendous amounts of web content not being the services hosting it. Thus, aiming to locate the most important content and promote it to the users, the…
Paper proposes graph-based separable transforms for video coding.
problem Improving video coding efficiency by better capturing residual block statistics.
method Derives graph-based separable transforms (GBSTs) from line graphs with weights determined by parameters.
result GBSTs achieve about 0.4% average coding gain over existing transforms in VVC.
Jointly trains images and videos using residual vectors.
problem Generating high-quality videos from images and vice versa.
method Simultaneously learns latent variables for images and videos using residual vectors.
result Improves sample quality and diversity in video generation and image generation.
Finding compact representation of videos is an essential component in almost every problem related to video processing or understanding. In this paper, we propose a generative model to learn compact latent codes that can efficiently represent and reconstruct a video sequence from its missing or under-sampled measuremen…
Paper introduces adversarial lossy compression for video artifacts reduction.
problem Unpleasant reconstruction artifacts in standard video coding schemes at low bit-rates.
method Adversarial lossy video compression model minimizing an adversarial distortion objective.
result Reduction of perceptual artifacts and detail reconstruction under extreme compression.
Advanced video classification systems decode video frames to derive the necessary texture and motion representations for ingestion and analysis by spatio-temporal deep convolutional neural networks (CNNs). However, when considering visual Internet-of-Things applications, surveillance systems and semantic crawlers of la…
Paper presents a new video generation model using diffusion probabilistic methods.
problem Generating high-quality video sequences.
method Denoising diffusion probabilistic models, autoregressive, end-to-end optimization.
result Significant improvements in perceptual quality and probabilistic frame forecasting.
Generative models of natural images have progressed towards high fidelity samples by the strong leveraging of scale. We attempt to carry this success to the field of video modeling by showing that large Generative Adversarial Networks trained on the complex Kinetics-600 dataset are able to produce video samples of subs…
Human communication takes many forms, including speech, text and instructional videos. It typically has an underlying structure, with a starting point, ending, and certain objective steps between them. In this paper, we consider instructional videos where there are tens of millions of them on the Internet. We propose a…
C3 compresses images and videos with low complexity and high performance.
problem High complexity and low performance in neural compression models.
method Overfits a small model to each image or video separately, improving RD performance with low complexity.
result Matches the RD performance of state-of-the-art neural and video codecs with significantly lower decoding complexity.
Researchers create a flickering attack to fool video recognition networks.
problem Adversarial manipulation of video classification networks.
method Introducing a flickering temporal perturbation to fool video classifiers.
result Achieved high fooling ratio and temporal-invariant perturbation.
AMS improves video inference on edge devices by adapting a small model with online knowledge distillation.
problem High computation cost of Deep Neural Networks for real-time video inference on edge devices.
method AMS uses a remote server to continually train and adapt a small model on edge devices, using online knowledge distillation from a large model.
result 0.4--17.8 percent mean Intersection-over-Union improvement in video semantic segmentation.
LVTINO improves high-definition video restoration with consistent temporal details.
problem Restoring high-definition video with fine spatial detail and temporal consistency.
method LVTINO uses Video Consistency Models to bypass frame-by-frame image-based LDMs, achieving perceptual quality and computational efficiency.
result Significant perceptual improvements over current methods, achieving state-of-the-art quality with minimal evaluations.
New method compresses facial videos using GANs and latent space optimization.
problem Efficiently compressing facial videos at low bit rates.
method Leverages StyleGAN for latent space representation and compression, learns optimal compression through entropy model and perceptual loss.
result Significantly reduces perceptual distortion at low bit rates compared to state-of-the-art codecs.