Learning the distribution of natural images is one of the hardest and most important problems in machine learning. The problem remains open, because the enormous complexity of the structures in natural images spans all length scales. We break down the complexity of the problem and show that the hierarchy of structures …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Natural image processing”
Latent feature models are attractive for image modeling, since images generally contain multiple objects. However, many latent feature models ignore that objects can appear at different locations or require pre-segmentation of images. While the transformed Indian buffet process (tIBP) provides a method for modeling tra…
Recurrent iterated function systems (RIFSs) are improvements of iterated function systems (IFSs) using elements of the theory of Marcovian stochastic processes which can produce more natural looking images. We construct new RIFSs consisting substantially of a vertical contraction factor function and nonlinear transform…
The ability to characterize the color content of natural imagery is an important application of image processing. The pixel by pixel coloring of images may be viewed naturally as points in color space, and the inherent structure and distribution of these points affords a quantization, through clustering, of the color i…
A method for diffusion on probability simplex for generative models.
problem Tension between continuous and discrete data in diffusion models.
method Proposes using softmax function applied to Ornstein-Uhlenbeck Process on probability simplex.
result Method extends to bounded image generation.
Study reveals differences in medical image models' hidden representation refinement.
problem Understanding how intrinsic dimensionality changes in neural network hidden representations across different domains.
method Analysis of 11 natural and medical image datasets using 6 network architectures.
result Medical image models refine hidden representations earlier, suggesting differences in feature abstraction.
New model generates images by reversing heat equation, revealing disentanglement.
problem Image generation without considering image structure.
method Stochastically reverses the heat equation to generate images, using variational approximation.
result Emergent disentanglement of overall colour and shape in images.
Planetary exploration missions with Mars rovers are complicated, which generally require elaborated task planning by human experts, from the path to take to the images to capture. NASA has been using this process to acquire over 22 million images from the planet Mars. In order to improve the degree of automation and th…
Visual question answering is a recently proposed artificial intelligence task that requires a deep understanding of both images and texts. In deep learning, images are typically modeled through convolutional neural networks, and texts are typically modeled through recurrent neural networks. While the requirement for mo…
Improved image restoration using frequency-guided sampling.
problem Restoring high-quality images from degraded observations with known degradation processes.
method Proposed a frequency-guided sampling approach for diffusion-based image restoration, incorporating a time-varying low-pass filter.
result Significantly improved performance on challenging image restoration tasks, including motion deblurring and image dehazing.
Recent progress in deep generative models has led to tremendous breakthroughs in image generation. However, while existing models can synthesize photorealistic images, they lack an understanding of our underlying 3D world. We present a new generative model, Visual Object Networks (VON), synthesizing natural images of o…
Diffusion models reveal a phase transition in reconstructing high-level features.
problem Understanding the hierarchical structure of natural data.
method Study of hierarchical generative models of data using diffusion models.
result The backward diffusion process shows a phase transition at a threshold time, where high-level features suddenly drop in reconstructibility.
Low-dimensional structure in images helps deep learning models generalize better.
problem Understanding the intrinsic dimensionality of images for better model performance.
method Applied dimension estimation tools to popular image datasets and used GANs to manipulate intrinsic dimensionality.
result Natural image datasets have very low intrinsic dimensionality, which aids neural networks in learning and generalizing.
Boomerang generates nonidentical images similar to input on image manifolds.
problem Generating nonidentical images similar to input on image manifolds.
method Adding noise to input image, moving closer to latent space, and mapping back through partial reverse diffusion.
result Boomerang generates nonidentical images similar to input on image manifolds.
High-contrast "gaudy" images improve the training of deep neural network models of visual cortexcs.CV
Gaudy images help train deep neural networks with less data.
problem Training deep neural networks with limited real data from visual cortex neurons.
method Used high-contrast binarized natural images (gaudy images) to train DNNs.
result Reduced training data needed for accurate DNN predictions of visual cortex neuron responses.
This study presents a new lossy image compression method that utilizes the multi-scale features of natural images. Our model consists of two networks: multi-scale lossy autoencoder and parallel multi-scale lossless coder. The multi-scale lossy autoencoder extracts the multi-scale image features to quantized variables a…
An ability to model a generative process and learn a latent representation for speech in an unsupervised fashion will be crucial to process vast quantities of unlabelled speech data. Recently, deep probabilistic generative models such as Variational Autoencoders (VAEs) have achieved tremendous success in modeling natur…
Matrix product states (MPS), a tensor network designed for one-dimensional quantum systems, has been recently proposed for generative modeling of natural data (such as images) in terms of `Born machine'. However, the exponential decay of correlation in MPS restricts its representation power heavily for modeling complex…
A machine learning method predicts rock permeability from 3D images.
problem Efficiently predict permeability of heterogeneous rocks for planetary and robotic applications.
method Machine learning guided 3D properties recognition of rock morphology from 3D micro CT and MRI images.
result The morphology decoder method accurately predicts permeability from 3D images.
The lottery ticket hypothesis proposes that over-parameterization of deep neural networks (DNNs) aids training by increasing the probability of a "lucky" sub-network initialization being present rather than by helping the optimization process (Frankle & Carbin, 2019). Intriguingly, this phenomenon suggests that initial…
This paper reviews GANs algorithms, theory, and applications.
problem Lack of comprehensive study on GANs connections and evolution.
method Detailed introduction of GANs algorithms, theoretical investigation, and application illustrations.
result Comprehensive review of GANs from algorithms, theory, and applications perspectives.
New VAE models reveal hierarchical visual cortex computations.
problem Capturing hierarchical visual cortex computations in generative models.
method Sparse coding hierarchical VAEs trained on natural images with varied generative and recognition components.
result Representations similar to those in visual cortex emerge under inductive biases.
A novel method for visual question answering using scene graphs and reinforcement learning.
problem Answering free-form questions about images with deep linguistic and visual understanding.
method Context-driven, sequential reasoning based on scene graphs and reinforcement learning.
result Our method almost reaches human performance on the GQA dataset.
A novel method compresses point cloud attributes by folding them onto a 2D grid.
problem Efficiently compressing point cloud attributes for storage and transmission.
method Interpreting point clouds as 2D manifolds, folding onto a grid, and mapping attributes to the grid using optimized methods.
result The proposed folding-based approach achieves performance comparable to state-of-the-art codecs.
This work abstracts deep neural networks into concept graphs for better interpretability in medical tasks.
problem Lack of interpretability in deep learning models, especially in medical domains.
method Developed a graphical representation of medical image processing models to understand concept-based reasoning.
result Extracted a concept-level graph that reveals the decision-making process of deep learning models.
What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established…
Paper introduces new methods for modeling categorical data.
problem Training generative models on categorical data like text and segmentation.
method Argmax Flows and Multinomial Diffusion models.
result Models outperform existing methods in log-likelihood.
Variational approach improves diffusion models for solving inverse problems.
problem Challenges in solving inverse problems with diffusion models due to the nonlinear and iterative nature of the diffusion process.
method Proposes a variational approach to approximate the posterior distribution, leading to regularization by denoising diffusion process (RED-Diff).
result Demonstrates improved performance in image restoration tasks compared to state-of-the-art sampling-based diffusion models.
Following great success in the image processing field, the idea of adversarial training has been applied to tasks in the natural language processing (NLP) field. One promising approach directly applies adversarial training developed in the image processing field to the input word embedding space instead of the discrete…
Deep learning on an edge device requires energy efficient operation due to ever diminishing power budget. Intentional low quality data during the data acquisition for longer battery life, and natural noise from the low cost sensor degrade the quality of target output which hinders adoption of deep learning on an edge d…
Improved image translation using asymmetric gradient guidance.
problem Trade-off between style transformation and content preservation in diffusion models.
method Asymmetric gradient guidance to guide reverse diffusion sampling.
result Our method outperforms state-of-the-art models in image translation tasks.
This study investigates how much knowledge from natural images can be transferred to pathology images.
problem Quantifying how much knowledge from natural images can be transferred to pathology images.
method Proposes a framework to quantify knowledge gain by a particular layer, conducts empirical investigation in pathology image centered transfer learning.
result Early layers of deep models can transfer knowledge to pathology image classification tasks.
Go With the Flow, on Jupiter and Snow. Coherence From Model-Free Video Data without Trajectoriesphysics.data-an
Viewing a data set such as the clouds of Jupiter, coherence is readily apparent to human observers, especially the Great Red Spot, but also other great storms and persistent structures. There are now many different definitions and perspectives mathematically describing coherent structures, but we will take an image pro…
New algorithm speeds up diffusion model sampling 4-14 times.
problem Time-consuming sampling from diffusion models.
method Parallelizing autoregressive process through fixed-point iteration.
result ParaTAA reduces inference steps by 4-14 times.
CSI method learns conditional distributions by estimating flow equations.
problem Learning conditional distributions in generative models.
method Estimates probability flow equations to transport reference to target distribution.
result Derives explicit expressions for conditional drift and score functions.
Deep learning constitutes a recent, modern technique for image processing and data analysis, with promising results and large potential. As deep learning has been successfully applied in various domains, it has recently entered also the domain of agriculture. In this paper, we perform a survey of 40 research efforts th…
Deep neural networks (DNNs) have recently achieved state-of-the-art performance and provide significant progress in many machine learning tasks, such as image classification, speech processing, natural language processing, etc. However, recent studies have shown that DNNs are vulnerable to adversarial attacks. For inst…
Current Flash X-ray single-particle diffraction Imaging (FXI) experiments, which operate on modern X-ray Free Electron Lasers (XFELs), can record millions of interpretable diffraction patterns from individual biomolecules per day. Due to the stochastic nature of the XFELs, those patterns will to a varying degree includ…
This study shows how EEG can be used to generate fMRI data.
problem Mapping fMRI from EEG signals.
method Deep learning approaches (Autoencoders, GANs, Pairwise Learning).
result Feasibility of EEG to fMRI brain image mappings.
Effective learning of asymmetric and local features in images and other data observed on multi-dimensional grids is a challenging objective critical for a wide range of image processing applications involving biomedical and natural images. It requires methods that are sensitive to local details while fast enough to han…
The deep image prior was recently introduced as a prior for natural images. It represents images as the output of a convolutional network with random inputs. For "inference", gradient descent is performed to adjust network parameters to make the output match observations. This approach yields good performance on a rang…
Sparse Vision MoE matches dense networks in image recognition while using less compute.
problem Scaling vision models efficiently in computer vision.
method Vision MoE (V-MoE) - a sparse version of Vision Transformer.
result V-MoE matches state-of-the-art dense networks in image recognition with half the compute.
Survey of deep learning methods for fMRI natural image reconstruction.
problem Reconstructing natural images from fMRI brain activity.
method Survey of deep learning approaches, including architectural design, datasets, and evaluation metrics.
result Performance evaluation across standardized metrics.
Statistical analysis of Diffusion Tensor Imaging (DTI) data requires a computational framework that is both numerically tractable (to account for the high dimensional nature of the data) and geometric (to account for the nonlinear nature of diffusion tensors). Building upon earlier studies that have shown that a Rieman…
This work debiases deep chest X-ray classifiers using intra- and post-processing methods.
problem Bias in deep neural networks for chest X-ray classification.
method Intra-processing techniques (fine-tuning and pruning) and post-processing methods.
result Successfully mitigates biases in fully connected and convolutional neural networks, offering stable performance.
The Transformer architecture has become increasingly popular over the past two years, owing to its impressive performance on a number of natural language processing (NLP) tasks. However, all Transformer computations occur at the level of word representations and therefore, it may be argued that Transformer models do no…
Paper proposes a new framework for hypothesis testing in imaging.
problem Challenges in hypothesis testing for imaging data.
method Combines self-supervised imaging, vision-language models, and non-parametric hypothesis testing.
result Demonstrates improved power and robust error control in image-based phenotyping.
Regularizes attention scores in vision transformers using bootstrapping.
problem Noisy and diffused attention maps in ViT limit interpretability.
method Statistical learning techniques, bootstrapping of attention scores.
result Improves shrinkage and sparsity of attention scores.