Semantic inpainting is the task of inferring missing pixels in an image given surrounding pixels and high level image semantics. Most semantic inpainting algorithms are deterministic: given an image with missing regions, a single inpainted image is generated. However, there are often several plausible inpaintings for a…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “semantic inpainting”
Physics-informed semantic inpainting improves geostatistical modeling by incorporating indirect measurements.
problem Inferring heterogeneous geological fields from limited measurements and prior spatial statistics.
method Proposes a physics-informed semantic inpainting framework using WGAN-GP to incorporate both direct and indirect measurements.
result The method satisfies physical conservation laws and enhances inpainting performance compared to using only direct measurements.
We develop a method for user-controllable semantic image inpainting: Given an arbitrary set of observed pixels, the unobserved pixels can be imputed in a user-controllable range of possibilities, each of which is semantically coherent and locally consistent with the observed pixels. We achieve this using a deep generat…
Generative adversarial networks have been successfully applied to inpainting in natural images. However, the current state-of-the-art models have not yet been widely adopted in the medical imaging domain. In this paper, we investigate the performance of three recently published deep learning based inpainting models: co…
In this paper we present an end-to-end deep learning framework to turn images that show dynamic content, such as vehicles or pedestrians, into realistic static frames. This objective encounters two main challenges: detecting all the dynamic objects, and inpainting the static occluded background with plausible imagery. …
Interpreting the Latent Space of GANs via Correlation Analysis for Controllable Concept Manipulationcs.CV
Method interprets GAN latent space via latent variable correlation analysis.
problem Understanding the inner workings of GANs.
method Analyzing correlation between latent variables and semantic contents in generated images.
result A method for controllable semantic content generation in GANs.
A Theoretical Justification for Image Inpainting using Denoising Diffusion Probabilistic Modelsstat.ML
Theoretical justification for image inpainting using diffusion models.
problem Improving sample recovery in image inpainting without retraining.
method Analysis of RePaint algorithm and proposing RePaint to correct misalignment.
result RePaint algorithm provably recovers the true sample with linear convergence.
RGI improves robustness of GAN-inversion for image restoration and anomaly detection.
problem Robustness of GAN-inversion to unknown gross corruptions.
method Proposes RGI and R-RGI methods with provable robustness guarantees.
result Restored images and corrupted region masks converge to ground truth under mild assumptions.
New method separates objects from images using deep neural networks trained to inpaint.
problem Fully self-supervised instance separation of occluded objects in images.
method Maximizes independence of two image regions given a fully self-supervised inpainting network.
result Method achieves similar segmentation performance to fully supervised methods on microscopy image datasets.
Multiple sclerosis (MS) is an inflammatory demyelinating disease of the central nervous system (CNS) that results in focal injury to the grey and white matter. The presence of white matter lesions biases morphometric analyses such as registration, individual longitudinal measurements and tissue segmentation for brain v…
GACELA fills long gaps in musical audio with a GAN and context conditioning.
problem Restoring long gaps in musical audio with varying complexity and duration.
method Generative adversarial network (GAN) with five parallel discriminators and context conditioning.
result Reduced artifacts in inpaintings from unacceptable to mildly disturbing.
EnSF uses image inpainting to handle partial observations in data assimilation.
problem Data assimilation challenges with partial observations.
method EnSF integrates image inpainting with diffusion models to predict unobserved states.
result EnSF successfully tracks SQG dynamics with partial observations.
Unsupervised image inpainting models generate plausible reconstructions from incomplete data.
problem Inpainting without paired or unpaired training data.
method Conditional GAN with latent component dependency for generating image distributions.
result Model generates a distribution of plausible images from incomplete observations.
We address the problem of surface inpainting, which aims to fill in holes or missing regions on a Riemann surface based on its surface geometry. In practical situation, surfaces obtained from range scanners often have holes where the 3D models are incomplete. In order to analyze the 3D shapes effectively, restoring the…
An important problem in geostatistics is to build models of the subsurface of the Earth given physical measurements at sparse spatial locations. Typically, this is done using spatial interpolation methods or by reproducing patterns from a reference image. However, these algorithms fail to produce realistic patterns and…
Generative adversarial network improves audio inpainting for long gaps.
problem Generating missing audio content in long-range gaps using WGAN.
method Proposed WGAN architecture with short-range and long-range neighboring borders.
result The proposed model outperforms classical WGAN in reconstructing high-frequency content.
Extended RDS filtering for positions and orientations, improving crossing structure enhancement and inpainting.
problem Enhancing and inpainting images with crossing structures.
method Extended RDS filtering to space, using gauge frames to mitigate issues.
result RDS filtering outperforms existing techniques in denoising and inpainting images with crossing structures.
In recent years, advances in machine learning algorithms, cheap computational resources, and the availability of big data have spurred the deep learning revolution in various application domains. In particular, supervised learning techniques in image analysis have led to superhuman performance in various tasks, such as…
Music Inpainting is the task of filling in missing or lost information in a piece of music. We investigate this task from an interactive music creation perspective. To this end, a novel deep learning-based approach for musical score inpainting is proposed. The designed model takes both past and future musical context i…
New method denoises and fills in missing image data without clean training data.
problem Denoising and inpainting images with noisy, incomplete data.
method Robust Hadamard Autoencoders trained on noisy, incomplete data.
result Autoencoders can perform denoising and inpainting simultaneously.
Natural image statistics exhibit hierarchical dependencies across multiple scales. Representing such prior knowledge in non-factorial latent tree models can boost performance of image denoising, inpainting, deconvolution or reconstruction substantially, beyond standard factorial "sparse" methodology. We derive a large …
CosmoVAE uses deep learning to fill in missing parts of the cosmic microwave background map.
problem Missing observations in the CMB map due to thermal dust noise.
method Variational autoencoder (VAE) trained with a new loss function.
result CosmoVAE achieves state-of-the-art performance in CMB map inpainting.
RG-Flow: A hierarchical and explainable flow model based on renormalization group and sparse priorcs.LG
RG-Flow combines RG and sparse priors for hierarchical image disentanglement.
problem Disentangling and manipulating image representations at different scales.
method Hierarchical flow model using RG and sparse prior distributions.
result RG-Flow enables semantic manipulation and style mixing at different image scales.
New method solves linear inverse problems using diffusion models.
problem Linear inverse problems in various domains.
method Posterior sampling with latent diffusion models.
result Provable sample recovery in linear models, outperforming previous methods.
Estimates localized complexity of white-matter wiring using GANs.
problem Complexity of white-matter wiring, especially in ambiguous regions, confounds analysis.
method Bayesian estimate of heteroscedastic aleatoric uncertainty through image inpainting.
result Localized wiring complexity quantified, directly reflecting difficulty of lesion inpainting.
InvGAN combines generative and inference models for photo-realistic image manipulation.
problem GANs lack an inference model for image editing and downstream tasks.
method Train inference and generative models together to adapt and converge.
result InvGAN embeds real images into a high-quality generative model's latent space.
Physics-informed neural networks in the recreation of hydrodynamic simulations from dark matterastro-ph.CO
Physics-informed neural networks improve baryonic predictions from dark matter simulations.
problem Recreating hydrodynamic simulations from dark matter requires expensive and time-consuming computations.
method Combining neural network architectures with physical constraints and using Kullback-Leibler divergence for prediction comparison.
result Improved accuracy of baryonic predictions based on dark matter halo properties, successful recovery of the metallicity relation, and preserved scatter.
Matrix completion is a widely used technique for image inpainting and personalized recommender system, etc. In this work, we focus on accelerating the matrix completion using faster randomized singular value decomposition (rSVD). Firstly, two fast randomized algorithms (rSVD-PI and rSVD- BKI) are proposed for handling …
Generative models improve image classifier explanations by realistically removing features.
problem Perturbation-based explanations often produce unrealistic counterfactual samples.
method Integrate generative inpainters into attribution methods to remove input features.
result Improved explanation methods in object localization, deletion, and saliency metrics.
In this paper, we study the missing sample recovery problem using methods based on sparse approximation. In this regard, we investigate the algorithms used for solving the inverse problem associated with the restoration of missed samples of image signal. This problem is also known as inpainting in the context of image …
A multi-way factor analysis model is introduced for tensor-variate data of any order. Each data item is represented as a (sparse) sum of Kruskal decompositions, a Kruskal-factor analysis (KFA). KFA is nonparametric and can infer both the tensor-rank of each dictionary atom and the number of dictionary atoms. The model …
Temporal observations such as videos contain essential information about the dynamics of the underlying scene, but they are often interleaved with inessential, predictable details. One way of dealing with this problem is by focusing on the most informative moments in a sequence. We propose a model that learns to discov…
PAC-Bayesian matrix completion with a spectral scaled Student prior offers efficient inference.
problem Matrix completion with underlying low-rank structure.
method Spectral scaled Student prior and PAC-Bayesian bounds.
result Minimax-optimal oracle inequality for model misspecification and general sampling distribution.
Ada-LISTA adapts neural solvers for varying models.
problem Adapting neural solvers for varying models.
method Ada-LISTA receives pairs of signals and dictionaries, learns a universal architecture, and solves sparse coding in linear rate.
result Ada-LISTA solves sparse coding in linear rate for varying models.
Recent advances in conditional image generation tasks, such as image-to-image translation and image inpainting, are largely accounted to the success of conditional GAN models, which are often optimized by the joint use of the GAN loss with the reconstruction loss. However, we reveal that this training recipe shared by …
Extends RDS filtering to position-orientation space for better image processing.
problem Enhancing and inpainting images with crossing structures.
method Created a version of RDS filtering using gauge frames, studying generalised diffusion.
result RDS filtering on position-orientation space improves denoising and inpainting of crossing structures.
Sparsity and low-rank models have been popular for reconstructing images and videos from limited or corrupted measurements. Dictionary or transform learning methods are useful in applications such as denoising, inpainting, and medical image reconstruction. This paper proposes a framework for online (or time-sequential)…
Hinge-FM2I fills missing data in time series with high accuracy.
problem Handling missing data in univariate time series.
method Inspired by door hinges, Hinge-FM2I imputes missing data using FM2I and selects the best imputed gap.
result Hinge-FM2I significantly outperforms other methods in sMAPE scores.
The problem of recovering a low -rank tensor is an extension of sparse recovery problem from the low dimensional space (matrix space) to the high dimensional space (tensor space) and has many applications in computer vision and graphics such as image inpainting and video inpainting. In this paper, we consider a new …
Combines variational and evolutionary optimization for generative models.
problem Optimizing generative models with discrete latent variables.
method Truncated posteriors as variational distributions, evolutionary algorithms applied to variational parameters.
result Evolutionary algorithms effectively optimize variational bounds for generative models.
Semantic TrueLearn uses semantic graphs to improve educational recommendation systems.
problem Challenges in handling semantic and hierarchical structure in knowledge areas.
method Introduces a novel learner model that exploits semantic relatedness between knowledge components using a Wikipedia link graph.
result Achieves statistically significant improvements in predictive performance for educational engagement.
Inpainting the Neural Picture: Inferring Unrecorded Brain Area Dynamics from Multi-Animal Datasetsq-bio.NC
NeuroPaint infers missing brain area dynamics from multi-animal datasets.
problem Leveraging multi-animal datasets to understand interactions between brain areas.
method Masked autoencoding approach trained across animals with partial observations.
result Models can successfully reconstruct dynamics of unrecorded brain areas.
IdBench benchmarks semantic representations of identifiers, revealing strengths and weaknesses.
problem Evaluating semantic representations of identifiers in source code.
method Created a benchmark using developer ratings, evaluated natural language and source code embeddings, and compared lexical string distance functions.
result No single technique provides a satisfactory representation of semantic similarities, but ensemble models can improve performance.
Unified framework for data-driven priors in Bayesian inverse problems
problem Bayesian inverse problems
method Unified framework using score functions
result Evaluation of four data-driven priors
Transfer learning aims at building robust prediction models by transferring knowledge gained from one problem to another. In the semantic Web, learning tasks are enhanced with semantic representations. We exploit their semantics to augment transfer learning by dealing with when to transfer with semantic measurements an…
Improved diffusion models solve inverse problems more accurately by correcting sample paths off the data manifold.
problem Current diffusion models for inverse problems often produce suboptimal results due to sample paths deviating from the data manifold.
method Proposed an additional correction term inspired by manifold constraints to make iterations closer to the data manifold.
result The proposed method boosts performance by a large margin, producing promising results in various applications.
This paper presents a Semantic Attribute Modulation (SAM) for language modeling and style variation. The semantic attribute modulation includes various document attributes, such as titles, authors, and document categories. We consider two types of attributes, (title attributes and category attributes), and a flexible a…
The paper shows how integrating categorical semantics can enhance unsupervised domain translation.
problem Improving unsupervised domain translation between perceptually different domains.
method Learning invariant categorical semantic features in an unsupervised manner and conditioning them on the style encoder.
result Conditioning the style encoder on learned categorical semantics improves translation and stylization.