Analyzes how diffusion models learn, revealing a spectral bias in structure mastery.
problem Understanding the learning dynamics and bias in diffusion models.
method Developed an analytical framework using a Gaussian-equivalence principle to solve gradient-flow dynamics and integrate probability-flow ODEs.
result Exposes a universal inverse-variance spectral law: high-variance structure is mastered faster than low-variance detail.
Theory explains creativity in diffusion models generating novel images.
problem Diffusion models generate highly original images far from training data.
method Identified locality and equivariance as inductive biases to prevent optimal score-matching.
result Analytic models predict diffusion model outputs with high accuracy.
Graph convolution is the core of most Graph Neural Networks (GNNs) and usually approximated by message passing between direct (one-hop) neighbors. In this work, we remove the restriction of using only the direct neighbors by introducing a powerful, yet spatially localized graph convolution: Graph diffusion convolution …
RaMViD uses diffusion models for video prediction and infilling.
problem Predicting and infilling missing information in videos.
method Extends image diffusion models to videos using 3D convolutions and a new conditioning technique.
result Achieves state-of-the-art results on video prediction benchmarks.
Spatiotemporal forecasting has various applications in neuroscience, climate and transportation domain. Traffic forecasting is one canonical example of such learning task. The task is challenging due to (1) complex spatial dependency on road networks, (2) non-linear temporal dynamics with changing road conditions and (…
DELIMIT is a framework extension for deep learning in diffusion imaging, which extends the basic framework PyTorch towards spherical signals. Based on several novel layers, deep learning can be applied to spherical diffusion imaging data in a very convenient way. First, two spherical harmonic interpolation layers are a…
Bi-Lipschitz flows approximate a wide range of distributions.
problem Characterizing the expressivity of bi-Lipschitz normalizing flows.
method Linking score regularity to transport map bi-Lipschitzness via probability flow ODE.
result Gaussian pullbacks induced by bi-Lipschitz variance-preserving transport maps are L1-dense among all probability densities. Graph diffusion processes approximate manifold heat semigroups using graph transition matrices.
problem Approximating manifold heat semigroups from graph data under low regularity conditions.
method Iterating graph transition matrix P to approximate Qt=etΔ, bounding error in ∞-norm. result Convergence rates O(N−2/(d+6)) for manifold heat semigroup approximation, valid for in-sample and out-of-sample. Proposes an alternative approach to propagate labels in GCNs using network diffusion and clustering.
problem Challenges of training GCNs with limited labeled data and bias in network diffusion methods.
method Clustering nodes into communities, using diffusion to quantify proximity, and comparing topological profiles.
result Identifies nodes most similar to labeled nodes, improving label propagation in GCNs.
Two architectures that generalize convolutional neural networks (CNNs) for the processing of signals supported on graphs are introduced. We start with the selection graph neural network (GNN), which replaces linear time invariant filters with linear shift invariant graph filters to generate convolutional features and r…
New proof shows diffusion models implicitly estimate intrinsic dimensionality.
problem Estimating intrinsic dimensionality of data from diffusion models.
method Formal proof of FLIPD under realistic assumptions.
result FLIPD's correctness proven under realistic conditions.
DALMC provides non-asymptotic error bounds for generative models.
problem Efficiently generating samples from complex data distributions.
method Analysis of diffusion paths and Langevin Monte Carlo.
result Theoretical guarantees for a class of generative models.
A new neural network approach for diffusion on networks.
problem Inference and estimation of diffusion on network structures.
method Neural mean-field dynamics derived from Mori-Zwanzig formalism, approximated by learnable time convolution operators.
result Significantly outperforms existing approaches in accuracy and efficiency.
Textual network embedding leverages rich text information associated with the network to learn low-dimensional vectorial representations of vertices. Rather than using typical natural language processing (NLP) approaches, recent research exploits the relationship of texts on the same edge to graphically embed text. How…
In this work we detail the application of a fast convolution algorithm computing high dimensional integrals to the context of multiplicative noise stochastic processes. The algorithm provides a numerical solution to the problem of characterizing conditional probability density functions at arbitrary time, and we applie…
Improves CNN stability by translating classical signal denoising methods.
problem Stability of CNNs is poorly understood.
method Interprets classical signal denoising methods as ResNet architectures.
result Translates diffusivities, shrinkage functions, and regularizers into CNN activation functions.
The financial market is nonpredictable, as according to the Bachelier, the mathematical expectation of the speculator is zero. Nevertheless, we observe in the price fluctuations the two distinct scales, short and long time. Behaviour of a market in long terms, such as year intervals, is different from that in short ter…
Stability is a key aspect of data analysis. In many applications, the natural notion of stability is geometric, as illustrated for example in computer vision. Scattering transforms construct deep convolutional representations which are certified stable to input deformations. This stability to deformations can be interp…
Due to the rapid growth of machine learning tools and specifically deep networks in various computer vision and image processing areas, application of Convolutional Neural Networks for watermarking have recently emerged. In this paper, we propose a deep end-to-end diffusion watermarking framework (ReDMark) which can be…
Tract-specific diffusion measures, as derived from brain diffusion MRI, have been linked to white matter tract structural integrity and neurodegeneration. As a consequence, there is a large interest in the automatic segmentation of white matter tract in diffusion tensor MRI data. Methods based on the tractography are p…
Diffusion models adapt to low-dimensional structures for nonparametric density estimation.
problem High-dimensional statistical inference challenges.
method Viewing diffusion models as implicit density estimators and exploiting their low-dimensional structure.
result Achieves minimax optimal rate for total variation distance with factorizable density.
We introduce two constructions in geometric deep learning for 1) transporting orientation-dependent convolutional filters over a manifold in a continuous way and thereby defining a convolution operator that naturally incorporates the rotational effect of holonomy; and 2) allowing efficient evaluation of manifold convol…
Nonparametric method measures influence of training images on diffusion model outputs.
problem Quantifying influence of individual training examples on diffusion model outputs.
method Patch-level similarity between generated and training images, using optimal score function.
result Strong attribution performance, matching gradient-based approaches and outperforming baselines.
DiGS improves sampling from multi-modal distributions.
problem Inadequate mixing in MCMC methods for multi-modal distributions.
method Integrates diffusion models and Gibbs sampling to create an auxiliary noisy distribution.
result DiGS exhibits better mixing for multi-modal distributions than state-of-the-art methods.
Proposes graph neural network layers for manifold-valued graphs.
problem Graphs with features in a Riemannian manifold.
method Diffusion layer and tangent multilayer perceptron.
result Outperforms state-of-the-art networks on Alzheimer's classification.
The paper develops a new probabilistic framework for denoising diffusion models using free entropy and stochastic analysis.
problem Developing a mathematical framework for denoising diffusion models in noncommutative settings.
method Formulating diffusion and reverse processes governed by operator-valued stochastic dynamics, using tools from free stochastic analysis.
result Establishing an information-geometric link between entropy production, transport, and deconvolution.
U-Nets use belief propagation for efficient image denoising and classification.
problem Efficiently denoise and classify images using generative hierarchical models.
method Interpreted U-Nets as implementing belief propagation in tree-structured models.
result U-Nets can efficiently approximate denoising functions with sample complexity bounds.
Paper proposes a weak supervision technique for CNN semantic segmentation of lung diseases using partially annotated data.
problem Creating annotated datasets for semantic segmentation of lung diseases is laborious and time-consuming.
method Proposes a weak supervision technique that utilizes partially annotated datasets to improve CNN semantic segmentation accuracy.
result Significantly improved segmentation accuracy using partially annotated datasets.
The paper proves sampling methods using discrete-time processes and information theory.
problem Proving convergence guarantees for diffusion-based sampling methods.
method Directly works with discrete-time stochastic processes and uses information theory.
result Discrepancy between sampling and comparison processes is bounded using information theory.
We simplify diffusion models by defining a design space and improving FID scores.
problem Complexity in diffusion-based generative models.
method Defined a design space, improved sampling and training processes, and pre-conditioned score networks.
result Improved FID scores of 1.79 for CIFAR-10 and 1.97 for unconditional settings.
Unified kernel framework extends to stochastic systems, improving numerical stability.
problem Extending kernel methods to stochastic dynamical systems with diffusion.
method Unified kernel framework, Feynman-Kac path-integral representations, collocation-based computational framework.
result Kernel equivalence under uniform ellipticity assumptions and improved numerical stability with moderate diffusion.
The paper introduces a new loss function to prevent overfitting in semi-supervised graph networks.
problem Overfitting in semi-supervised graph networks trained with cross-entropy loss.
method Proposes an unsupervised manifold smoothness loss to regularize the graph convolutional networks.
result Adding the proposed loss consistently improves performance of graph networks.
Developed a monotone numerical method for MV portfolio optimization under jump-diffusion models.
problem Efficiently optimizing portfolios with jump-diffusion dynamics and investment constraints.
method Strictly monotone numerical integration method using Fourier transforms and composite quadrature rules.
result Proven to be ℓ∞-stable and pointwise consistent, converging to the MV optimization solution. A CNN-based method improves DTI of the human heart, compensating for motion.
problem Signal loss due to heart motion in DTI.
method Invertible Wavelet Scattering using CNN.
result Effective motion compensation and improved fiber structures.
A hybrid model combines diffusion and neural operator methods for stress prediction in hyperelastic materials.
problem Challenges in predicting stress fields in hyperelastic materials with complex microstructures.
method A hybrid surrogate framework combining a conditional denoising diffusion probabilistic model (cDDPM) and a modified DeepONet.
result The hybrid model consistently outperforms traditional methods by one to two orders of magnitude.
Transfer learning improves highway traffic forecasting using graph neural networks.
problem Lack of historical data for traffic forecasting on large highway networks.
method Developed a transfer learning approach for DCRNN, a graph neural network for highway forecasting.
result TL-DCRNN can forecast traffic on unseen regions of the highway network with high accuracy.
Generative model uses random convolutional features to create financial time series.
problem Generating realistic financial time series with limited data and avoiding overfitting.
method Train generators by matching random convolutional features of real and generated time series, using SOCK (SOft Competing Kernels) feature map.
result Generators trained with random SOCK features outperform baselines across various financial datasets.
Improved graph neural network bounds using graph diffusion matrix.
problem Empirical performance of graph neural networks on real-world graphs.
method Unified model of graph neural networks, focusing on feature diffusion matrix stability.
result Generalization bounds scale with largest singular value of feature diffusion matrix, smaller than prior bounds.
Algorithm learns mixtures of Gaussians efficiently using diffusion models.
problem Learning mixtures of Gaussians with identity covariance.
method Analytic approach using diffusion models to learn score functions.
result Quasi-polynomial time and sample complexity for learning mixtures.
Study evaluates data augmentation methods for prostate cancer detection in MRI.
problem Limited data for prostate cancer detection in MRI.
method Static application of five augmentation techniques (rotation, flip, crop, translation) to MRI dataset.
result Rotation method improved 2D slice-based AUC to 0.85.
We propose the Lanczos network (LanczosNet), which uses the Lanczos algorithm to construct low rank approximations of the graph Laplacian for graph convolution. Relying on the tridiagonal decomposition of the Lanczos algorithm, we not only efficiently exploit multi-scale information via fast approximated computation of…
Proposes a new method for sampling from unknown distributions.
problem Challenges in sampling from distributions without direct sampling.
method Uses a dilation path to estimate score vectors in closed-form, guiding Langevin dynamics.
result Demonstrates improved sampling performance compared to classical methods.
New method transforms complex stochastic equations into simpler ones for efficient simulation.
problem Efficient simulation of complex path-dependent stochastic processes.
method Transforms Volterra-type SDEs into standard diffusion processes using convolution kernels.
result Proposes a numerical simulation scheme with a strong convergence rate of 1/2.
The purpose of this work is to develop and study a distributed strategy for Pareto optimization of an aggregate cost consisting of regularized risks. Each risk is modeled as the expectation of some loss function with unknown probability distribution while the regularizers are assumed deterministic, but are not required…
A novel capsule network model improves surrogate modeling and uncertainty quantification from sparse data.
problem Surrogate modeling and uncertainty quantification of systems from sparse data.
method Adapted Capsule Network (CapsNet) architecture into image-to-image regression encoder-decoder network.
result The proposed approach accurately, efficiently, and robustly predicts responses for arbitrary diffusion fields.
Diffuse optical tomography (DOT) has been investigated as an alternative imaging modality for breast cancer detection thanks to its excellent contrast to hemoglobin oxidization level. However, due to the complicated non-linear photon scattering physics and ill-posedness, the conventional reconstruction algorithms are s…
Traffic forecasting approaches are critical to developing adaptive strategies for mobility. Traffic patterns have complex spatial and temporal dependencies that make accurate forecasting on large highway networks a challenging task. Recently, diffusion convolutional recurrent neural networks (DCRNNs) have achieved stat…
New methods implement manifold scattering transform for high-dimensional point cloud data.
problem Classifying data on complex, non-linear manifolds.
method Adapting diffusion maps theory for numerical implementation.
result Effective for signal and manifold classification tasks.