The paper investigates the convergence of Vendi scores under finite samples and introduces a truncated version for better performance.
problem The Vendi score's convergence is hindered by computational limitations when using large sample sizes.
method The authors introduce the t-truncated Vendi score to address this issue by truncating the eigenspectrum of the kernel matrix.
result The t-truncated Vendi score converges to its asymptotic limit with a smaller number of samples, improving upon the standard Vendi score.
This work sets lower bounds on the number of score queries needed for diffusion sampling.
problem Establishing information-theoretic limits on the number of score evaluations required for diffusion sampling.
method Proving lower bounds on the number of adaptive score queries needed for sampling.
result Any sampling algorithm requires at least \(\widetilde{\Omega}(\sqrt{d})\) adaptive score queries for \(d\)-dimensional distributions.
Score matching method improves image generation quality.
problem Score matching method underperforms GANs in metrics like Fréchet Inception Distance.
method DSM-ALS with denoising and improved sampling techniques.
result Score matching methods can match GANs in image generation quality.
New method generates novel samples from closed-form diffusion models.
problem Closed-form SGMs memorize training data and cannot generate novel samples.
method Explicitly smooth closed-form score, use nearest-neighbor estimator.
result Efficient method generates novel samples without training.
A new method samples from a target density without initial samples using Monte Carlo estimation of the score.
problem Sampling from a target density without initial samples.
method Monte Carlo estimation of the score using oracle access to the log likelihood.
result Samples can be produced from the target density without needing initial samples.
A new method for generating samples without training, using smoothed score matching.
problem Generating samples efficiently and without training.
method Moment-matched score-smoothed overdamped Langevin dynamics (MM-SOLD).
result The method enables fast, robust, training-free sampling with competitive sample fidelity and diversity.
Improved sampling efficiency for inverse problems using variance-reduced diffusion methods.
problem Efficiently estimating noisy scores in inverse problems.
method Developed a nonparametric self-normalized importance sampling estimator and a state-dependent blending rule.
result Improved sample quality for fixed simulation budgets in synthetic targets and PDE-governed inverse problems.
Score function estimators improve k-subset sampling efficiency.
problem Efficiently sampling k-subsets in machine learning tasks. method Revisit score function estimators, using discrete Fourier transform and control variates.
result Efficient and unbiased gradient estimates for k-subset sampling. New bounds for score matching in polynomial exponential families.
problem Understanding the sample complexity of score matching for polynomial exponential families.
method Non-asymptotic sample complexity analysis for score matching.
result First finite sample bounds for score matching in polynomial exponential families.
Generalizes leverage score sampling for neural networks, accelerating kernel methods and deep learning.
problem Accelerating kernel methods and deep learning training.
method Generalizes leverage score sampling to neural networks and proves equivalence to neural tangent kernel ridge regression.
result Equivalence between regularized neural network and neural tangent kernel ridge regression under leverage score sampling initialization.
We explain theoretically a curious empirical phenomenon: "Approximating a matrix by deterministically selecting a subset of its columns with the corresponding largest leverage scores results in a good low-rank matrix surrogate". To obtain provable guarantees, previous work requires randomized sampling of the columns wi…
New characterization limits sampling with inexact scores.
problem Limiting sampling with inexact scores for unbiased results.
method Characterized types of inexact score oracle access.
result Weaker error assumptions rule out tractability of unbiased sampling.
The paper establishes bounds for score-matching in causal discovery and generative modeling.
problem Estimating causal relationships from data.
method Training a deep neural network to estimate the score function and applying it to causal discovery.
result Bounds on the error rate of causal discovery methods using score-matching.
Leverage score sampling provides an appealing way to perform approximate computations for large matrices. Indeed, it allows to derive faithful approximations with a complexity adapted to the problem at hand. Yet, performing leverage scores sampling is a challenge in its own right requiring further approximations. In th…
This work extends diffusion models to handle heavy-tailed targets, improving score estimation and sampling guarantees.
problem Score estimation and sampling guarantees for heavy-tailed targets in diffusion models.
method Kernel density estimation and minimax rates analysis for score estimation and sampling guarantees.
result Sharp minimax rates for score estimation and sampling guarantees for heavy-tailed targets, revealing qualitative differences between exponential and polynomial tails.
A new method simplifies sampling from complex distributions without using diffusions.
problem Sampling from complex, high-dimensional distributions efficiently.
method Reduces sampling to solving a sequence of 'nice' sampling problems using SLC distributions.
result Shows how to traverse backwards paths using high-accuracy routines for SLC distributions.
In this paper, we consider a statistical problem of learning a linear model from noisy samples. Existing work has focused on approximating the least squares solution by using leverage-based scores as an importance sampling distribution. However, no finite sample statistical guarantees and no computationally efficient o…
SGMs fail to generate samples from complex distributions even when the score function is learned well.
problem Score-based Generative Models fail to produce high-quality samples from complex distributions.
method Score-based Generative Models (SGMs) are evaluated under conditions where the score function is learned well.
result SGMs can only generate Gaussian blurring of training data points, not complex distributions.
Extends importance sampling to nonlinear models using adjoint operators.
problem Lack of tools for identifying important data points in nonlinear models.
method Introduces adjoint operator for nonlinear maps, generalizes norm and leverage scores.
result Generalized scores provide approximation guarantees for nonlinear mappings.
Use of an autoencoder (AE) as a normal model is a state-of-the-art technique for unsupervised-anomaly detection in sounds (ADS). The AE is trained to minimize the sample mean of the anomaly score of normal sounds in a mini-batch. One problem with this approach is that the anomaly score of rare-normal sounds becomes hig…
Accelerates data generation in score-based models.
problem Slow generation of realistic data by score-based models.
method Developed an adaptive step size SDE solver.
result Generates data 2-10 times faster with high quality.
Improves robustness of propensity score estimators in challenging settings.
problem Limited overlap, small sample sizes, or unbalanced data.
method Extends calibration techniques for propensity score models, focusing on sample-splitting schemes.
result Calibration reduces variance and bias in inverse probability weighting and double/debiased machine learning frameworks.
Human decision-makers often receive assistance from data-driven algorithmic systems that provide a score for evaluating objects, including individuals. The scores are generated by a function (mechanism) that takes a set of features as input and generates a score.The scoring functions are either machine-learned or human…
New bounds close the score matching gap for diffusion models.
problem The difference between sample quality and score matching loss in diffusion models.
method Theoretical analysis of score matching gap, developing tighter bounds for KL divergence, reverse KL divergence, and Wasserstein distance.
result The quality of score approximation impacts closing the score matching gap for low noise scales.
Proposes a new method for sampling from unknown distributions.
problem Challenges in sampling from distributions without direct sampling.
method Uses a dilation path to estimate score vectors in closed-form, guiding Langevin dynamics.
result Demonstrates improved sampling performance compared to classical methods.
UCoS avoids forward model evaluations in sampling for large-scale linear inverse problems.
problem Efficient sampling from posterior distributions in large-scale linear inverse problems.
method UCoS approach that learns a task-dependent score function offline and uses affine transformations to derive the conditional score.
result UCoS eliminates the need for forward model evaluations during sampling, making it more efficient.
Reconstruction error is a prevalent score used to identify anomalous samples when data are modeled by generative models, such as (variational) auto-encoders or generative adversarial networks. This score relies on the assumption that normal samples are located on a manifold and all anomalous samples are located outside…
Combining diffusion models with Langevin dynamics improves posterior sampling efficiency.
problem Sampling from noisy posterior distributions efficiently.
method Annealed Langevin dynamics combined with diffusion models.
result Achieves posterior sampling in polynomial time with a weaker score error bound.
A new algorithm improves credit scoring accuracy for imbalanced data.
problem Poor classification of minority class in credit scoring data sets.
method Weighted-Hybrid-Sampling-Boost (WHSBoost) algorithm with balanced data sampling.
result WHSBoost outperforms other methods in credit scoring accuracy.
Improves sampling quality in model composition using MH-like acceptance rule for score-based diffusion models.
problem Inability to apply MH corrections in score-based diffusion models for model composition.
method Introduces a novel MH-like acceptance rule based on line integration of the score function.
result Relative improvements similar to energy-based models without explicit energy parameterization.
Improved score-based models generate high-quality images up to 256x256.
problem Training score-based models for high-resolution images is unstable and limited.
method Theoretical analysis, exponential moving average of model weights.
result Score-based models can generate high-fidelity images up to 256x256.
Improved diffusion models for generative tasks without dimensionality constraints.
problem Sample complexity bounds for learning score functions in diffusion models.
method Dimension-free sample complexity bounds, martingale-based error decomposition, variance reduction technique (Bootstrapped Score Matching).
result Achieved a double exponential improvement in sample complexity over prior results.
Paper proposes a method to reduce hallucinations in diffusion models using Laplacian score sharpening.
problem Hallucinations in diffusion models create incoherent or unrealistic samples.
method Post-hoc adjustment to the score function during inference using Laplacian approximation.
result Significantly reduces the rate of hallucinated samples across various data types.
Most multi-class classifiers make their prediction for a test sample by scoring the classes and selecting the one with the highest score. Analyzing these prediction scores is useful to understand the classifier behavior and to assess its reliability. We present an interactive visualization that facilitates per-class an…
MIRA scores assess conditional distribution accuracy using joint samples.
problem Assessing the accuracy of candidate conditional distributions.
method Analytic expression for Mira score based on equal probability mass regions.
result Mira enables Bayesian model comparison by quantifying alignment with true process.
CTM improves diffusion model sampling quality with efficient ODE traversal.
problem Lack of natural trade-off between sample quality and speed in consistency models.
method CTM trains a neural network to output scores and traverse ODE trajectories efficiently.
result CTM achieves state-of-the-art FIDs and improves sample quality with increased computational budget.
We study convergence of a generative modeling method that first estimates the score function of the distribution using Denoising Auto-Encoders (DAE) or Denoising Score Matching (DSM) and then employs Langevin diffusion for sampling. We show that both DAE and DSM provide estimates of the score of the Gaussian smoothed p…
LSGM trains SGMs in latent space for faster sampling.
problem Efficiently generating high-quality samples from complex distributions.
method LSGM trains SGMs in latent space using a variational autoencoder framework, introducing new score-matching objectives and parameterizations.
result LSGM achieves state-of-the-art FID score of 2.10 on CIFAR-10 and outperforms previous SGMs in sampling time.
Improved sampling from high-dimensional Gaussians using smoothed scores.
problem Sampling from high-dimensional Gaussian distributions with gradient information.
method Using smoothed scores, which are gradients of the logarithms of Gaussian-convolved densities, to overcome approximation barriers.
result Improved sampling efficiency with a complexity of \(O\left(\left(\logκ+\log(e\sqrt d/δ_{
m TV})
ight)\log(e\sqrt d/δ_{
m TV})
ight)\) smoothed-score queries.
Novel estimator reduces diffusion model variance.
problem High variance in score function estimation for diffusion models.
method Uses nearest neighbour samples to estimate the score function.
result Significant decrease in variance, leading to improved model performance.
Improved sample complexity for diffusion models without needing empirical risk minimizers.
problem Theoretical limitations in sample complexity for diffusion models.
method Structured decomposition of score estimation error, eliminating dependence on neural network parameters.
result Achieved sample complexity bound of O(ε^(-4)) without empirical risk minimizer access.
We introduce a new discrepancy score between two distributions that gives an indication on their similarity. While much research has been done to determine if two samples come from exactly the same distribution, much less research considered the problem of determining if two finite samples come from similar distributio…
New optimal rates for score estimation improve diffusion model performance.
problem Improving statistical rates for score estimation in diffusion models.
method Sharp minimax rates for score estimation of diffused distributions.
result Achieves sharp minimax rate without extraneous logarithmic terms.
A new sampling method estimates scores without training or nested MCMC.
problem Efficient sampling from complex, unnormalised distributions.
method Multiscale averaging in SDEs for score estimation.
result Empirical results show competitive accuracy and efficiency.
Proposes GM Score to evaluate GANs considering diversity, disentanglement, and discriminability.
problem Evaluation of GANs for sample quality and diversity.
method Integrates various factors including intra-class and inter-class diversity, disentanglement, and discriminability metrics.
result Demonstrates improved evaluation of GANs on MNIST dataset.
We study strictly proper scoring rules in the Reproducing Kernel Hilbert Space. We propose a general Kernel Scoring rule and associated Kernel Divergence. We consider conditions under which the Kernel Score is strictly proper. We then demonstrate that the Kernel Score includes the Maximum Mean Discrepancy as a special …
This work extends score-based methods to binary data on the Boolean hypercube.
problem Learning and sampling binary data on the Boolean hypercube.
method Adopting Bernoulli noise as a smoothing device, deriving a TMF-like expression for the optimal denoiser, and using a Langevin-like sampler.
result The method successfully samples noisy binary data and reduces effective noise through multiple measurements.
A new method improves data generation quality by correcting score mismatches.
problem Score mismatch issue in conditional score-based data generation methods.
method Denoising Likelihood Score Matching (DLSM) loss for classifier training.
result The proposed method outperforms previous methods on Cifar-10 and Cifar-100 benchmarks.