BNEM improves Boltzmann sampler efficiency.
problem Generating IID samples from Boltzmann distributions efficiently.
method Bootstrapped Noised Energy Matching (NEM) combined with diffusion-based learning and bootstrapping.
result BNEM achieves state-of-the-art performance with improved robustness.
Energy Matching unifies flow matching and energy-based models for generative modeling.
problem Inability of flow-based models to integrate partial observations and priors.
method Energy Matching framework that integrates flow matching and energy-based models.
result Substantially outperforms existing EBMs on CIFAR-10 and ImageNet generation.
Graph Energy Matching improves generation quality for molecular graphs.
problem Discrete energy-based models struggle with efficient and high-quality sampling for graph generation.
method Inspired by transport-map optimization, Graph Energy Matching learns a permutation-invariant potential energy to guide sampling.
result GEM matches or surpasses discrete diffusion baselines on molecular graph benchmarks.
Energy-Based Models (EBMs) assign unnormalized log-probability to data samples. This functionality has a variety of applications, such as sample synthesis, data denoising, sample restoration, outlier detection, Bayesian reasoning, and many more. But training of EBMs using standard maximum likelihood is extremely slow b…
EBMs are flexible but hard to train; this paper explains methods.
problem Training Energy-Based Models is difficult due to the unknown normalizing constant.
method Explains MCMC, SM, and NCE for training EBMs, highlighting connections.
result Provides a friendly introduction to modern EBM training methods.
ScoreGrad predicts multivariate time series with energy-based models, achieving state-of-the-art results.
problem Predicting multivariate time series with generative models while considering noise and distribution.
method ScoreGrad uses continuous energy-based generative models with a feature extraction and score matching module.
result ScoreGrad achieves state-of-the-art results on six real-world datasets.
DP-SGD can update fewer coordinates while maintaining privacy.
problem How to update fewer coordinates in DP-SGD without losing optimization signal.
method TP-TopK (Two-Phase TopK DP-SGD), a two-phase method for coordinate-sparse private training.
result Private training can update fewer coordinates without losing optimization signal, scaling noise with active dimension \(k\) instead of full dimension \(d\).
Training energy-based probabilistic models is confronted with apparently intractable sums, whose Monte Carlo estimation requires sampling from the estimated probability distribution in the inner loop of training. This can be approximately achieved by Markov chain Monte Carlo methods, but may still face a formidable obs…
iDEM generates samples from Boltzmann densities without data.
problem Generating statistically independent samples from unnormalized distributions.
method Iterative algorithm using energy and gradient for diffusion-based sampler training.
result iDEM achieves state-of-the-art performance and trains faster than existing methods.
Method matches noisy remote sensing images robustly.
problem Matching noisy remote sensing images.
method Combining attention mechanism with feature enhancement.
result More efficient and accurate matches achieved.
New method improves sampling from noisy energy models.
problem Training and sampling challenges in Energy-Based Models.
method Pseudo-Gibbs sampling with moment matching.
result Effective sampling from clean model using noisy DSM-trained model.
Noise-resilient method improves Hurst exponent estimation accuracy in noisy data.
problem Noise degrades accuracy of Hurst exponent estimation methods.
method Noise-Controlled ALPHEE (NC-ALPHEE) using wavelet multi-scale analysis and neural network combination.
result NC-ALPHEE consistently outperforms existing techniques in noisy conditions.
Vector approximate message passing (VAMP) is a computationally simple approach to the recovery of a signal x from noisy linear measurements y=Ax+w. Like the AMP proposed by Donoho, Maleki, and Montanari in 2009, VAMP is characterized by a rigorous state evolution (SE) that holds …
Low precision operations can provide scalability, memory savings, portability, and energy efficiency. This paper proposes SWALP, an approach to low precision training that averages low-precision SGD iterates with a modified learning rate schedule. SWALP is easy to implement and can match the performance of full-precisi…
A new method improves training of energy-based models.
problem Training energy-based models is challenging.
method Diffusive Classification (DiffCLF) objective.
result DiffCLF enables EBMs with higher fidelity and broader applicability.
A new loss function ED simplifies training energy-based models without scores.
problem Training energy-based models is computationally expensive.
method Energy Discrepancy (ED) loss function that does not rely on scores or MCMC.
result ED effectively interpolates between score matching and negative log-likelihood.
Graph matching in noisy environments with Markovian errors.
problem Graph matching under time-dependent Markovian noise.
method Introduced edgelighter error model and analyzed graph matching thresholds.
result Graph matching thresholds and mixing times are of order Θ(n2logn) for Erdős-Rényi graphs, and O(nαlogn) for Stochastic Block Model graphs. Improves score estimation for noised targets using known clean scores.
problem Poor score estimation at low noise levels in Denoising Score Matching.
method Introduces Target Score Identity and Target Score Matching loss.
result Score estimates are more accurate at low noise levels.
Improved VAEs by training a contrastive prior to match posterior.
problem Prior hole problem in VAEs, leading to poor image generation.
method Introduced a contrastive energy-based prior and trained it using noise contrastive estimation.
result Significant improvement in VAE generative performance on various datasets.
Improves generative models by using heavy-tailed noise in score matching.
problem High-dimensional limitations of Gaussian noise in generative models.
method Extended DSM to generalised normal distribution, relaxed key assumptions, developed iterative noise scaling algorithm.
result Heavy-tailed DSM leads to improved generative performance.
Optimizes quadratic bandits with tight Hessian-dependent sample complexity bounds.
problem Understanding optimal sample complexity for quadratic functions.
method Introduces energy allocation and optimal energy spectrum to prove tight lower bounds. Solves for Hessian-independent optimal algorithm.
result Proves optimal Hessian-dependent sample complexities and existence of a universally optimal algorithm.
EWFM trains continuous flows with only energy evaluations, improving sample quality with fewer computations.
problem Efficiently sampling from complex, high-dimensional Boltzmann distributions using only energy evaluations.
method Energy-Weighted Flow Matching (EWFM) using importance sampling and iterative/annealed training.
result Improved sample quality with up to 3 orders of magnitude fewer energy evaluations compared to existing methods.
iEFM trains CNF models from unnormalized densities efficiently.
problem Training generators from energy functions or unnormalized densities.
method Iterated energy-based flow matching (iEFM) with simulation-free objective.
result iEFM outperforms existing methods in probabilistic modeling.
New method trains energy-based models faster and more stably.
problem Training efficiency and stability of energy-based models.
method EBFlow with score-matching objectives.
result EBFlow achieves significant speedup and better performance.
SGD with machine learning noise converges to global minimum exponentially fast.
problem Optimizing machine learning models with stochastic gradient descent.
method Analysis of SGD with machine learning noise, focusing on energy landscapes and gradient noise.
result SGD converges to the global minimum exponentially fast under certain conditions.
Paper proposes a new method for training diffusion models using Markov operators.
problem Training efficiency and accuracy in diffusion models.
method Operator-informed score matching using spectral decomposition of Markov operators.
result Improved score matching for both low and high-dimensional distributions.
Improved PINNs for solving PDEs with unknown measurement noise.
problem Handling non-Gaussian noise in physics-informed neural networks.
method Jointly train an EBM to learn the correct noise distribution.
result Improved performance in solving PDEs with non-Gaussian noise.
A new method trains discrete EBMs without sampling.
problem Training EBMs on discrete spaces is hard.
method Energy Discrepancy (ED), a contrastive loss.
result ED offers theoretical guarantees for various perturbation types.
Proposes linking energy and force uncertainty in deep learning potentials.
problem Uncertainty in predicted energies and forces in machine learning models.
method Introduces a spatially correlated noise process to link energy and force uncertainty.
result Demonstrates the approach on molecular datasets, linking energy and force uncertainties.
Improved graph matching algorithm robust to noise.
problem Finding a bijection between vertex sets of two graphs.
method Uses multistage signature vectors to match vertices.
result Recover matching exactly with high probability for α≤1/(loglogn)C. Stochastic attention learns to retrieve and generate from memory without training.
problem Learning to retrieve and generate from memory efficiently.
method Langevin dynamics on modern Hopfield energy for stochastic attention.
result Stochastic attention can retrieve and generate from memory without training, matching gold standards.
EBIL simplifies IL by estimating expert energy as reward, achieving effective performance.
problem Recovering optimal policy from expert demonstrations without reward signals.
method EBIL uses a two-stage solution: first estimating expert energy as reward, then learning policy.
result EBIL achieves effective performance and interpretable reward signals.
Extends denoising and score estimation to energy models via Tweedie's formula.
problem Linking denoising and score estimation for a wider range of distributions.
method Derives a fundamental identity connecting energy score derivatives and scores.
result Establishes a new identity for energy scores analogous to Tweedie's formula.
Efficiently matches subgraphs in noisy data without node labels.
problem Subgraph isomorphism in noisy, real-valued graphs.
method Two-step approach: extract topology, then expand matches.
result Realistically sub-linear computational efficiency, robustness to noise.
LFM learns a sequence of smaller models to generate data from noise.
problem Learning continuous, invertible flows between distributions.
method Stepwise Local Flow Matching (LFM) model, matching diffusion processes up to time-step size.
result LFM achieves competitive generative performance compared to Flow Matching.
Paper uses VAEs to detect radar targets in complex noise.
problem Detecting radar targets in compound clutter and thermal noise.
method Proposes a VAE architecture to distinguish radar targets from various noise types.
result The VAE outperforms classical detectors in challenging noise conditions.
CDC-FM improves generative model quality-generalization tradeoff by regularizing with geometry-aware noise.
problem Tradeoff between high sample quality and memorization in deep generative models.
method Introduces Carré du champ flow matching (CDC-FM) that replaces homogeneous noise with anisotropic Gaussian noise capturing latent data manifold geometry.
result CDC-FM consistently offers better quality-generalization tradeoff across diverse datasets and architectures.
Automatically detects and down-weights noisy samples in machine learning training.
problem Numerical noise in reference data hampers the accuracy of machine learning models.
method On-the-fly outlier detection using exponential moving average to identify and down-weight noisy samples.
result The method prevents overfitting and matches the performance of iterative refinement with reduced overhead.
ICIL learns policies invariant to multiple environments, improving generalization.
problem Learning policies from multiple environments leads to spurious correlations.
method ICIL learns invariant feature representations and a matching imitation policy.
result ICIL policies generalize better to unseen environments.
Generative model for condensed matter using Riemannian flow matching.
problem Sampling equilibrium distributions in condensed-phase systems.
method Riemannian flow matching to incorporate periodicity, using Hutchinson's trace estimator and cumulant expansion for bias correction.
result Highly accurate free energy estimates on monatomic ice without multistage estimators.
New method recovers causal graphs from data scores in non-linear models.
problem Recovering causal graphs from data scores in non-linear models.
method Score matching algorithms and efficient Jacobian approximation.
result New method, SCORE, is competitive and faster than state-of-the-art methods.
Extends Penrose's method to null shells with pressure and energy flux.
problem Constructing null thin shells with arbitrary gravitational/matter content.
method Derive locally Lipschitz metric and coordinate transformation.
result Example of null shell with non-trivial energy density, flux, and pressure in Minkowski space.
New insights into noise distribution for self-supervised learning.
problem Challenges the assumption that optimal noise should match data distribution.
method Turns to Noise-Contrastive Estimation (NCE) to define optimality of noise distribution.
result Optimal noise distribution is different from data distribution, challenging GANs assumption.
Adjoint Matching improves flow and diffusion models with reward fine-tuning.
problem Improving generative models with reward fine-tuning.
method Casting reward fine-tuning as stochastic optimal control (SOC) and enforcing a specific noise schedule.
result Adjoint Matching outperforms existing SOC algorithms.
WS diffusion models handle anisotropic Gaussian noise better than conventional methods.
problem Handling anisotropic Gaussian noise in imaging inverse problems.
method Whitened Score (WS) diffusion models based on stochastic differential equations.
result WS DMs outperform conventional DMs on anisotropic Gaussian noise.
In this paper, we attack the anomaly detection problem by directly modeling the data distribution with deep architectures. We propose deep structured energy based models (DSEBMs), where the energy function is the output of a deterministic deep neural network with structure. We develop novel model architectures to integ…
Proves weight polytope matches with energy vectors in toric varieties.
problem Understanding the relationship between weight polytopes and energy functionals in toric varieties.
method Combines two slope formulas of K-energy in the toric setting.
result Weight polytope of Hurwitz form matches with convex hull of characteristic vectors.
Density estimation is a fundamental problem in statistical learning. This problem is especially challenging for complex high-dimensional data due to the curse of dimensionality. A promising solution to this problem is given here in an inference-free hierarchical framework that is built on score matching. We revisit the…