EBPs model exchangeable data with flexible distributions.
problem Current energy-based models restrict set cardinality and limited distribution forms.
method Introduced Energy-Based Processes (EBPs) that extend energy models to exchangeable data with neural network parameterizations.
result EBPs can express more flexible distributions over sets without cardinality restrictions.
Versatile model for High Energy Physics events.
problem Modeling complex interactions in high-energy physics data.
method Energy-based probabilistic model with multi-purpose architecture.
result Achieves success in diverse applications like simulation, anomaly detection, and particle identification.
EBM model for functional data using path measure tilting.
problem Modeling distributions of functions from irregularly sampled data.
method Spectral decomposition of an energy-based model with a Gaussian Process path measure to reweight the distribution.
result Effective approach to up-scaling functional data.
BiDVL improves EBLVMs for visual tasks by optimizing two variational distributions.
problem Training EBLVMs is challenging due to intractable distributions.
method Bi-level doubly variational learning with two tractable distributions.
result BiDVL achieves impressive image generation and reconstruction performance.
RNE provides a flexible framework for diffusion models, enabling inference-time control and energy-based training.
problem Insufficient knowledge of marginal densities in diffusion models.
method Introduces Radon-Nikodym Estimator (RNE) to reveal the connection between marginal densities and transition kernels.
result RNE delivers strong results in inference-time control and energy-based diffusion training.
A new method trains discrete EBMs without sampling.
problem Training EBMs on discrete spaces is hard.
method Energy Discrepancy (ED), a contrastive loss.
result ED offers theoretical guarantees for various perturbation types.
Improved language models by incorporating statistical discriminators.
problem Distinguishing model-generated text from real text reliably.
method Energy-Based Model framework to incorporate discriminators.
result Improves language model performance in perplexity and human evaluation.
This paper proposes the divergence triangle as a framework for joint training of generator model, energy-based model and inference model. The divergence triangle is a compact and symmetric (anti-symmetric) objective function that seamlessly integrates variational learning, adversarial learning, wake-sleep algorithm, an…
Proposes a model to generate 3D-aware images from 2D images.
problem Generating 3D-aware images from 2D images.
method Likelihood-based top-down model using Neural Radiance Fields and energy-based latent variables.
result Model can infer 3D object structures from 2D images and generate novel views.
This paper proposes a more efficient training method for energy-based models.
problem The computational burden and validity trade-off in Contrastive Divergence training.
method Introducing Diffusion Contrastive Divergence (DCD) to replace Langevin dynamics with diffusion processes.
result The proposed DCDs are more computationally efficient and handle gradient terms better than Contrastive Divergence.
STANLEY improves sampling for complex data models.
problem Training Energy-Based models with intractable normalizing constants.
method Anisotropic Langevin Dynamics with gradient-informed covariance.
result Geometrically uniformly ergodic Markov Chain for sampling.
EBMs trained on discrete data using heat equations on graph structures.
problem Training EBMs on discrete or mixed data.
method Heat equations on graph structures for data perturbation.
result Efficacy demonstrated in various applications.
EB-RANSAC uses energy-based model for robust estimation without complex sampling.
problem Robust estimation of parameters in noisy data.
method EB-RANSAC combines RANSAC's sampling scheme with an energy-based model, simplifying the process and reducing hyperparameter requirements.
result EB-RANSAC effectively solves linear regression and maximum likelihood estimation problems.
VAV method optimizes learning rate for faster, stable SGD convergence.
problem Optimizing learning rate for efficient and stable machine learning models.
method Energy-based self-adaptive learning rate with auxiliary variable r. result VAV method achieves faster convergence and superior stability with larger learning rates.
Paper proposes CoopFlow, a two-flow generator for energy-based models.
problem Training energy-based models with Langevin flow and normalizing flow.
method CoopFlow trains an energy-based model using a normalizing flow initialization and a short-run Langevin flow revision.
result CoopFlow converges to a moment matching estimator and synthesizes realistic images.
A new method trains and samples from energy-based models using diffusion recovery likelihood.
problem Training and sampling high-dimensional datasets with energy-based models is challenging.
method Trains EBMs with a diffusion recovery likelihood method, maximizing conditional probabilities of data at different noise levels.
result Generates high-fidelity images with low FID and inception scores, and accurately estimates normalized data density.
A method to automatically learn proposal distributions for energy-based regression models.
problem Manual design and initial estimate of proposal distributions for energy-based regression models.
method Introduces a method to learn an effective proposal distribution automatically, parameterized by a separate network head, and derives a unified training objective to minimize KL divergence and negative log-likelihood.
result Consistently outperforms conventional MDN training on four real-world regression tasks within computer vision.
Proposes Learnergy, a Python framework for energy-based machine learning.
problem Lack of research and implementations around energy-based systems.
method Built upon PyTorch, Learnergy provides a more friendly environment and faster prototyping.
result Learnergy speeds up computational time using CUDA.
StrEBM learns distinct latent components for better source separation.
problem Blind source separation with identifiable and decoupled latent components.
method Structured latent energy-based model with learnable structural biases.
result The model effectively recovers source components from mixed signals.
THRML uses energy-based models for index tracking, reducing portfolio tracking error and improving returns.
problem NP-hard combinatorial optimization in portfolio optimization under cardinality constraints.
method THRML reformulates index tracking as probabilistic inference on an Ising Hamiltonian, using GPU-accelerated block Gibbs sampling.
result THRML achieves 4.31 percent annualized tracking error compared to 5.66-6.30 percent for baselines, with 128.63 percent total return.
New method learns latent energy models using particle algorithms.
problem Learning latent variable models with energy priors.
method Continuous-time SDEs for MMLE, particle-based discretization.
result Practical algorithm converges to solve MMLE problem.
A new loss function ED simplifies training energy-based models without scores.
problem Training energy-based models is computationally expensive.
method Energy Discrepancy (ED) loss function that does not rely on scores or MCMC.
result ED effectively interpolates between score matching and negative log-likelihood.
CEBMs learn flexible latent mappings from data.
problem Learning flexible latent mappings from data.
method CEBMs decompose joint density into tractable posterior over latent variables.
result CEBMs achieve competitive results in image modeling and latent space predictive power.
Improves sample quality of generative models using energy-based methods.
problem Low sample quality in generative models.
method Constructs an energy function on latent space, trains an energy-based model, and generates improved samples.
result Significant improvement in sample quality with minimal computational overhead.
EBR improves NMT by re-ranking samples drawn from MLE-trained models.
problem Discrepancy between MLE and BLEU score in neural machine translation.
method Train an energy-based model to mimic BLEU score, then use it for re-ranking.
result EBR consistently improves NMT performance by +4 BLEU points on IWSLT'14 German-English.
Bidirectional bounds stabilize training of energy-based models.
problem Training energy-based models is difficult and prone to instability.
method Propose bidirectional bounds linking to gradient penalty and Jacobi-determinant estimator.
result Significant stabilization and high-quality density estimation achieved.
New method trains energy-based models faster and more stably.
problem Training efficiency and stability of energy-based models.
method EBFlow with score-matching objectives.
result EBFlow achieves significant speedup and better performance.
ScoreGrad predicts multivariate time series with energy-based models, achieving state-of-the-art results.
problem Predicting multivariate time series with generative models while considering noise and distribution.
method ScoreGrad uses continuous energy-based generative models with a feature extraction and score matching module.
result ScoreGrad achieves state-of-the-art results on six real-world datasets.
An energy based approach for stabilizing a mechanical system has offered a simple yet powerful control scheme. However, since it does not impose such strong constraints on parameter space of the controller, finding appropriate parameter values for an optimal controller is known to be hard. This paper intends to generat…
We propose to reinterpret a standard discriminative classifier of p(y|x) as an energy based model for the joint distribution p(x,y). In this setting, the standard class probabilities can be easily computed as well as unnormalized values of p(x) and p(x|y). Within this framework, standard discriminative architectures ma…
EB-GFN models discrete data with amortized MCMC sampling.
problem Probabilistic modeling of high-dimensional discrete data.
method EB-GFN combines GFlowNets with energy-based models for efficient sampling.
result EB-GFN effectively models various discrete data tasks.
Efficiently samples and learns densities with symmetries using equivariant methods.
problem Efficiently sampling and learning densities with symmetries.
method Equivariant Stein Variational Gradient Descent (SVGD) and equivariant energy based models.
result Improves and scales up training of energy based models.
Energy-based diffusion models improve molecular sampling and simulation.
problem Inconsistency between diffusion model scores and equilibrium distributions.
method Fokker-Planck regularization to enforce consistency.
result Improved consistency and efficient sampling of biomolecular systems.
Energy-based models can generate complex images by combining simpler concepts.
problem Generating natural images that satisfy complex logical combinations of concepts.
method Energy-based models combine probability distributions of simpler concepts to generate compositions.
result Energy-based models can generate images that satisfy conjunctions, disjunctions, and negations of concepts.
This paper studies a training method to jointly estimate an energy-based model and a flow-based model, in which the two models are iteratively updated based on a shared adversarial value function. This joint training method has the following traits. (1) The update of the energy-based model is based on noise contrastive…
This paper proposes a multi-grid method for learning energy-based generative ConvNet models of images. For each grid, we learn an energy-based probabilistic model where the energy function is defined by a bottom-up convolutional neural network (ConvNet or CNN). Learning such a model requires generating synthesized exam…
Proposes an energy-based sliced Wasserstein distance for improved probability measure comparison.
problem Inefficiencies and limitations in existing sliced Wasserstein distance approaches.
method Introduces an energy-based slicing distribution for better performance and stability.
result Demonstrates superior performance of the EBSW distance in various applications.
This paper introduces a new neural ODE model for continuous-time sequence generation.
problem Representing and predicting continuous-time sequences with high accuracy.
method A neural emission model and neural ODE define the latent state evolution, with an Energy-based model for prior distribution.
result The model outperforms existing methods in various tasks, including long-horizon predictions.
3D object detection improved using energy-based models.
problem Accurate 3D object detection in cluttered environments from sparse LiDAR data.
method Designing a differentiable pooling operator for 3D bounding boxes integrated into a state-of-the-art 3D object detector.
result Our approach consistently outperforms the SA-SSD baseline across all 3DOD metrics on the KITTI dataset.
Training energy-based probabilistic models is confronted with apparently intractable sums, whose Monte Carlo estimation requires sampling from the estimated probability distribution in the inner loop of training. This can be approximately achieved by Markov chain Monte Carlo methods, but may still face a formidable obs…
Paper proposes a method to improve MCMC sampling for energy-based models.
problem MCMC sampling of energy-based models is often not mixing in high-dimensional data.
method Proposes using a flow-based model as a backbone to correct the energy-based model, enabling mixing in latent space.
result MCMC sampling of the corrected EBM in the latent space mixes well and traverses modes in the data space.
Graph Energy Matching improves generation quality for molecular graphs.
problem Discrete energy-based models struggle with efficient and high-quality sampling for graph generation.
method Inspired by transport-map optimization, Graph Energy Matching learns a permutation-invariant potential energy to guide sampling.
result GEM matches or surpasses discrete diffusion baselines on molecular graph benchmarks.
iEFM trains CNF models from unnormalized densities efficiently.
problem Training generators from energy functions or unnormalized densities.
method Iterated energy-based flow matching (iEFM) with simulation-free objective.
result iEFM outperforms existing methods in probabilistic modeling.
This paper tackles sampling issues in latent space EBMs by introducing diffusion-based amortization.
problem Degenerate MCMC sampling quality hinders latent space EBM learning and generation quality.
method Introduces diffusion-based amortization for long-run MCMC sampling.
result The learned amortization of MCMC is a valid long-run MCMC sampler.
EMIX minimizes surprise in multi-agent reinforcement learning.
problem Surprise and approximation bias in multi-agent reinforcement learning.
method Energy-based MIXer (EMIX) for minimizing surprise across multiple agents.
result EMIX demonstrates consistent stable performance in challenging StarCraft II scenarios.
Unified view of contrastive and supervised learning improves model performance.
problem Improving model performance in classification, robustness, and detection.
method Hybrid discriminative-generative training of energy-based models.
result Improved performance on classification, robustness, out-of-distribution detection, and calibration.
Unified model explains AT's generative ability.
problem Understanding the generative ability of AT.
method Contrastive Energy-based Models (CEM).
result Improved sample quality in supervised and unsupervised learning.
Future autonomous systems need reliable world models and complex action sequences.
problem Current automated systems lack reliable world models and complex action sequences.
method Introduce energy-based and latent variable models combined in a hierarchical joint embedding predictive architecture (H-JEPA).
result Combining energy-based and latent variable models in H-JEPA can lead to reliable world models and complex action sequences.