Semantic inpainting is the task of inferring missing pixels in an image given surrounding pixels and high level image semantics. Most semantic inpainting algorithms are deterministic: given an image with missing regions, a single inpainted image is generated. However, there are often several plausible inpaintings for a…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,694 papers · 148 categories
Trend · papers per month
48 results for “Inpainting”
A Theoretical Justification for Image Inpainting using Denoising Diffusion Probabilistic Modelsstat.ML
Theoretical justification for image inpainting using diffusion models.
problem Improving sample recovery in image inpainting without retraining.
method Analysis of RePaint algorithm and proposing RePaint to correct misalignment.
result RePaint algorithm provably recovers the true sample with linear convergence.
New method separates objects from images using deep neural networks trained to inpaint.
problem Fully self-supervised instance separation of occluded objects in images.
method Maximizes independence of two image regions given a fully self-supervised inpainting network.
result Method achieves similar segmentation performance to fully supervised methods on microscopy image datasets.
Multiple sclerosis (MS) is an inflammatory demyelinating disease of the central nervous system (CNS) that results in focal injury to the grey and white matter. The presence of white matter lesions biases morphometric analyses such as registration, individual longitudinal measurements and tissue segmentation for brain v…
A fundamental problem in geostatistical modeling is to infer the heterogeneous geological field based on limited measurements and some prior spatial statistics. Semantic inpainting, a technique for image processing using deep generative models, has been recently applied for this purpose, demonstrating its effectiveness…
GACELA fills long gaps in musical audio with a GAN and context conditioning.
problem Restoring long gaps in musical audio with varying complexity and duration.
method Generative adversarial network (GAN) with five parallel discriminators and context conditioning.
result Reduced artifacts in inpaintings from unacceptable to mildly disturbing.
EnSF uses image inpainting to handle partial observations in data assimilation.
problem Data assimilation challenges with partial observations.
method EnSF integrates image inpainting with diffusion models to predict unobserved states.
result EnSF successfully tracks SQG dynamics with partial observations.
We address the problem of surface inpainting, which aims to fill in holes or missing regions on a Riemann surface based on its surface geometry. In practical situation, surfaces obtained from range scanners often have holes where the 3D models are incomplete. In order to analyze the 3D shapes effectively, restoring the…
Generative adversarial network improves audio inpainting for long gaps.
problem Generating missing audio content in long-range gaps using WGAN.
method Proposed WGAN architecture with short-range and long-range neighboring borders.
result The proposed model outperforms classical WGAN in reconstructing high-frequency content.
Extended RDS filtering for positions and orientations, improving crossing structure enhancement and inpainting.
problem Enhancing and inpainting images with crossing structures.
method Extended RDS filtering to space, using gauge frames to mitigate issues.
result RDS filtering outperforms existing techniques in denoising and inpainting images with crossing structures.
Generative adversarial networks have been successfully applied to inpainting in natural images. However, the current state-of-the-art models have not yet been widely adopted in the medical imaging domain. In this paper, we investigate the performance of three recently published deep learning based inpainting models: co…
We develop a method for user-controllable semantic image inpainting: Given an arbitrary set of observed pixels, the unobserved pixels can be imputed in a user-controllable range of possibilities, each of which is semantically coherent and locally consistent with the observed pixels. We achieve this using a deep generat…
Music Inpainting is the task of filling in missing or lost information in a piece of music. We investigate this task from an interactive music creation perspective. To this end, a novel deep learning-based approach for musical score inpainting is proposed. The designed model takes both past and future musical context i…
New method denoises and fills in missing image data without clean training data.
problem Denoising and inpainting images with noisy, incomplete data.
method Robust Hadamard Autoencoders trained on noisy, incomplete data.
result Autoencoders can perform denoising and inpainting simultaneously.
We consider inpainting in an unsupervised setting where there is neither access to paired nor unpaired training data. The only available information is provided by the uncomplete observations and the inpainting process statistics. In this context, an observation should give rise to several plausible reconstructions whi…
Natural image statistics exhibit hierarchical dependencies across multiple scales. Representing such prior knowledge in non-factorial latent tree models can boost performance of image denoising, inpainting, deconvolution or reconstruction substantially, beyond standard factorial "sparse" methodology. We derive a large …
In this paper we present an end-to-end deep learning framework to turn images that show dynamic content, such as vehicles or pedestrians, into realistic static frames. This objective encounters two main challenges: detecting all the dynamic objects, and inpainting the static occluded background with plausible imagery. …
New method solves linear inverse problems using diffusion models.
problem Linear inverse problems in various domains.
method Posterior sampling with latent diffusion models.
result Provable sample recovery in linear models, outperforming previous methods.
Perturbation-based explanation methods often measure the contribution of an input feature to an image classifier's outputs by heuristically removing it via e.g. blurring, adding noise, or graying out, which often produce unrealistic, out-of-samples. Instead, we propose to integrate a generative inpainter into three rep…
Cosmic microwave background radiation (CMB) is critical to the understanding of the early universe and precise estimation of cosmological constants. Due to the contamination of thermal dust noise in the galaxy, the CMB map that is an image on the two-dimensional sphere has missing observations, mainly concentrated on t…
Physics-informed neural networks in the recreation of hydrodynamic simulations from dark matterastro-ph.CO
Physics-informed neural networks improve baryonic predictions from dark matter simulations.
problem Recreating hydrodynamic simulations from dark matter requires expensive and time-consuming computations.
method Combining neural network architectures with physical constraints and using Kullback-Leibler divergence for prediction comparison.
result Improved accuracy of baryonic predictions based on dark matter halo properties, successful recovery of the metallicity relation, and preserved scatter.
Matrix completion is a widely used technique for image inpainting and personalized recommender system, etc. In this work, we focus on accelerating the matrix completion using faster randomized singular value decomposition (rSVD). Firstly, two fast randomized algorithms (rSVD-PI and rSVD- BKI) are proposed for handling …
In this paper, we study the missing sample recovery problem using methods based on sparse approximation. In this regard, we investigate the algorithms used for solving the inverse problem associated with the restoration of missed samples of image signal. This problem is also known as inpainting in the context of image …
A multi-way factor analysis model is introduced for tensor-variate data of any order. Each data item is represented as a (sparse) sum of Kruskal decompositions, a Kruskal-factor analysis (KFA). KFA is nonparametric and can infer both the tensor-rank of each dictionary atom and the number of dictionary atoms. The model …
Temporal observations such as videos contain essential information about the dynamics of the underlying scene, but they are often interleaved with inessential, predictable details. One way of dealing with this problem is by focusing on the most informative moments in a sequence. We propose a model that learns to discov…
PAC-Bayesian matrix completion with a spectral scaled Student prior offers efficient inference.
problem Matrix completion with underlying low-rank structure.
method Spectral scaled Student prior and PAC-Bayesian bounds.
result Minimax-optimal oracle inequality for model misspecification and general sampling distribution.
Recent advances in conditional image generation tasks, such as image-to-image translation and image inpainting, are largely accounted to the success of conditional GAN models, which are often optimized by the joint use of the GAN loss with the reconstruction loss. However, we reveal that this training recipe shared by …
Extends RDS filtering to position-orientation space for better image processing.
problem Enhancing and inpainting images with crossing structures.
method Created a version of RDS filtering using gauge frames, studying generalised diffusion.
result RDS filtering on position-orientation space improves denoising and inpainting of crossing structures.
Sparsity and low-rank models have been popular for reconstructing images and videos from limited or corrupted measurements. Dictionary or transform learning methods are useful in applications such as denoising, inpainting, and medical image reconstruction. This paper proposes a framework for online (or time-sequential)…
RGI improves robustness of GAN-inversion for image restoration and anomaly detection.
problem Robustness of GAN-inversion to unknown gross corruptions.
method Proposes RGI and R-RGI methods with provable robustness guarantees.
result Restored images and corrupted region masks converge to ground truth under mild assumptions.
Hinge-FM2I fills missing data in time series with high accuracy.
problem Handling missing data in univariate time series.
method Inspired by door hinges, Hinge-FM2I imputes missing data using FM2I and selects the best imputed gap.
result Hinge-FM2I significantly outperforms other methods in sMAPE scores.
The problem of recovering a low -rank tensor is an extension of sparse recovery problem from the low dimensional space (matrix space) to the high dimensional space (tensor space) and has many applications in computer vision and graphics such as image inpainting and video inpainting. In this paper, we consider a new …
Combines variational and evolutionary optimization for generative models.
problem Optimizing generative models with discrete latent variables.
method Truncated posteriors as variational distributions, evolutionary algorithms applied to variational parameters.
result Evolutionary algorithms effectively optimize variational bounds for generative models.
In-vivo examination of the physical connectivity of axonal projections through the white matter of the human brain is made possible by diffusion weighted magnetic resonance imaging (dMRI) Analysis of dMRI commonly considers derived scalar metrics such as fractional anisotrophy as proxies for "white matter integrity," a…
Inpainting the Neural Picture: Inferring Unrecorded Brain Area Dynamics from Multi-Animal Datasetsq-bio.NC
NeuroPaint infers missing brain area dynamics from multi-animal datasets.
problem Leveraging multi-animal datasets to understand interactions between brain areas.
method Masked autoencoding approach trained across animals with partial observations.
result Models can successfully reconstruct dynamics of unrecorded brain areas.
Unified framework for data-driven priors in Bayesian inverse problems
problem Bayesian inverse problems
method Unified framework using score functions
result Evaluation of four data-driven priors
Improved diffusion models solve inverse problems more accurately by correcting sample paths off the data manifold.
problem Current diffusion models for inverse problems often produce suboptimal results due to sample paths deviating from the data manifold.
method Proposed an additional correction term inspired by manifold constraints to make iterations closer to the data manifold.
result The proposed method boosts performance by a large margin, producing promising results in various applications.
We propose a single neural probabilistic model based on variational autoencoder that can be conditioned on an arbitrary subset of observed features and then sample the remaining features in "one shot". The features may be both real-valued and categorical. Training of the model is performed by stochastic variational Bay…
CCDF reduces diffusion sampling steps for inverse problems.
problem Slow sampling from diffusion models in inverse problems.
method Starting from a single forward diffusion step with better initialization, followed by stochastic contraction.
result Significantly reduced sampling steps for state-of-the-art reconstruction.
New function-space autoencoders improve data handling across resolutions.
problem Handling data as functions rather than discrete points.
method Introducing function-space autoencoders (FAE and FVAE) and neural operator architectures.
result Function-space autoencoders are more broadly applicable than variational autoencoders.
Generative diffusion models improve financial LOB simulation and forecasting.
problem High noise and complexity in financial LOB data makes deep generative models ineffective.
method Convert LOB data to images, apply diffusion models with inpainting for long-term sequence generation.
result Our method achieves state-of-the-art performance on LOB-Bench, improving coherence over local details.
This paper learns variational models and solvers for inverse problems from incomplete data.
problem Solving inverse problems with partially observed data.
method Joint learning of variational cost and gradient-based solver as neural networks.
result Joint learning leads to improved reconstruction performance.
Paper addresses unsupervised learning from incomplete measurements in inverse problems.
problem Learning from incomplete measurements is challenging in inverse problems.
method Use multiple measurement operators to overcome nullspace issues; propose a novel unsupervised learning loss.
result Presented necessary and sufficient conditions for successful unsupervised learning.
SPLICE generates accurate time-series imputations with reliable prediction intervals.
problem Lack of reliability guarantees in time-series imputation models.
method Modular framework combining latent generative imputation with distribution-free prediction intervals.
result SPLICE achieves lowest mean Load-only MSE and best CRPS on various datasets.
New VAE models reveal hierarchical visual cortex computations.
problem Capturing hierarchical visual cortex computations in generative models.
method Sparse coding hierarchical VAEs trained on natural images with varied generative and recognition components.
result Representations similar to those in visual cortex emerge under inductive biases.
Generative Adversarial Networks (GAN) have shown great promise in tasks like synthetic image generation, image inpainting, style transfer, and anomaly detection. However, generating discrete data is a challenge. This work presents an adversarial training based correlated discrete data (CDD) generation model. It also de…
This paper extends the recently proposed and theoretically justified iterative thresholding and residual means algorithm ITKrM to learning dicionaries from incomplete/masked training data (ITKrMM). It further adapts the algorithm to the presence of a low rank component in the data and provides a strategy for recove…
This paper considers the problem of matrix completion when the observed entries are noisy and contain outliers. It begins with introducing a new optimization criterion for which the recovered matrix is defined as its solution. This criterion uses the celebrated Huber function from the robust statistics literature to do…