ERM uses energy-based selection to improve recursive reasoning.
problem Lack of principled inference mechanism in recursive models.
method Energy-guided Recursive Model (ERM) introduces Hopfield energies for trajectory selection.
result ERM achieves optimal solutions on various puzzles.
A new method for decomposing non-negative tensors using energy-based modeling.
problem Challenges in traditional tensor decomposition methods, especially global optimization and rank selection.
method Energy-based modeling of tensors, considering interactions between modes for global optimization.
result Demonstrates effectiveness in tensor completion and approximation, revealing a relationship between many-body and low-rank approximations.
EORM boosts LLM accuracy with a lightweight, energy-based verifier.
problem Efficiently verifying mathematical reasoning in large language models.
method Energy-based framework to rank Chain-of-Thought solutions using simple outcome labels.
result EORM boosts LLM accuracy to 90.7% on GSM8k and 63.7% on MATH with only 55M parameters.
EB-RANSAC uses energy-based model for robust estimation without complex sampling.
problem Robust estimation of parameters in noisy data.
method EB-RANSAC combines RANSAC's sampling scheme with an energy-based model, simplifying the process and reducing hyperparameter requirements.
result EB-RANSAC effectively solves linear regression and maximum likelihood estimation problems.
Paper proposes CoopFlow, a two-flow generator for energy-based models.
problem Training energy-based models with Langevin flow and normalizing flow.
method CoopFlow trains an energy-based model using a normalizing flow initialization and a short-run Langevin flow revision.
result CoopFlow converges to a moment matching estimator and synthesizes realistic images.
Versatile model for High Energy Physics events.
problem Modeling complex interactions in high-energy physics data.
method Energy-based probabilistic model with multi-purpose architecture.
result Achieves success in diverse applications like simulation, anomaly detection, and particle identification.
New method optimizes sensor placement for stochastic systems efficiently.
problem Optimizing sensor placements for black-box stochastic systems with computational constraints.
method Trains a joint energy-based model on simulation data to learn parameter and solution distributions, allowing efficient sensor placement.
result Demonstrates lower computational cost and more informative sensor locations compared to conventional approaches.
A method to automatically learn proposal distributions for energy-based regression models.
problem Manual design and initial estimate of proposal distributions for energy-based regression models.
method Introduces a method to learn an effective proposal distribution automatically, parameterized by a separate network head, and derives a unified training objective to minimize KL divergence and negative log-likelihood.
result Consistently outperforms conventional MDN training on four real-world regression tasks within computer vision.
Proposes Learnergy, a Python framework for energy-based machine learning.
problem Lack of research and implementations around energy-based systems.
method Built upon PyTorch, Learnergy provides a more friendly environment and faster prototyping.
result Learnergy speeds up computational time using CUDA.
New method learns latent energy models using particle algorithms.
problem Learning latent variable models with energy priors.
method Continuous-time SDEs for MMLE, particle-based discretization.
result Practical algorithm converges to solve MMLE problem.
BiDVL improves EBLVMs for visual tasks by optimizing two variational distributions.
problem Training EBLVMs is challenging due to intractable distributions.
method Bi-level doubly variational learning with two tractable distributions.
result BiDVL achieves impressive image generation and reconstruction performance.
New method decomposes KL error using refined information and mode interactions.
problem Learning probability distributions over discrete variables with higher-order interactions.
method Using information geometry, refined mode interactions, and a novel Monte-Carlo sampling technique.
result Complete decomposition of KL error and efficient data use.
A new loss function ED simplifies training energy-based models without scores.
problem Training energy-based models is computationally expensive.
method Energy Discrepancy (ED) loss function that does not rely on scores or MCMC.
result ED effectively interpolates between score matching and negative log-likelihood.
CEBMs learn flexible latent mappings from data.
problem Learning flexible latent mappings from data.
method CEBMs decompose joint density into tractable posterior over latent variables.
result CEBMs achieve competitive results in image modeling and latent space predictive power.
EBPs model exchangeable data with flexible distributions.
problem Current energy-based models restrict set cardinality and limited distribution forms.
method Introduced Energy-Based Processes (EBPs) that extend energy models to exchangeable data with neural network parameterizations.
result EBPs can express more flexible distributions over sets without cardinality restrictions.
Improves sample quality of generative models using energy-based methods.
problem Low sample quality in generative models.
method Constructs an energy function on latent space, trains an energy-based model, and generates improved samples.
result Significant improvement in sample quality with minimal computational overhead.
EBR improves NMT by re-ranking samples drawn from MLE-trained models.
problem Discrepancy between MLE and BLEU score in neural machine translation.
method Train an energy-based model to mimic BLEU score, then use it for re-ranking.
result EBR consistently improves NMT performance by +4 BLEU points on IWSLT'14 German-English.
Bidirectional bounds stabilize training of energy-based models.
problem Training energy-based models is difficult and prone to instability.
method Propose bidirectional bounds linking to gradient penalty and Jacobi-determinant estimator.
result Significant stabilization and high-quality density estimation achieved.
New method trains energy-based models faster and more stably.
problem Training efficiency and stability of energy-based models.
method EBFlow with score-matching objectives.
result EBFlow achieves significant speedup and better performance.
ScoreGrad predicts multivariate time series with energy-based models, achieving state-of-the-art results.
problem Predicting multivariate time series with generative models while considering noise and distribution.
method ScoreGrad uses continuous energy-based generative models with a feature extraction and score matching module.
result ScoreGrad achieves state-of-the-art results on six real-world datasets.
An energy based approach for stabilizing a mechanical system has offered a simple yet powerful control scheme. However, since it does not impose such strong constraints on parameter space of the controller, finding appropriate parameter values for an optimal controller is known to be hard. This paper intends to generat…
We propose to reinterpret a standard discriminative classifier of p(y|x) as an energy based model for the joint distribution p(x,y). In this setting, the standard class probabilities can be easily computed as well as unnormalized values of p(x) and p(x|y). Within this framework, standard discriminative architectures ma…
RNE provides a flexible framework for diffusion models, enabling inference-time control and energy-based training.
problem Insufficient knowledge of marginal densities in diffusion models.
method Introduces Radon-Nikodym Estimator (RNE) to reveal the connection between marginal densities and transition kernels.
result RNE delivers strong results in inference-time control and energy-based diffusion training.
Efficiently samples and learns densities with symmetries using equivariant methods.
problem Efficiently sampling and learning densities with symmetries.
method Equivariant Stein Variational Gradient Descent (SVGD) and equivariant energy based models.
result Improves and scales up training of energy based models.
Energy-based diffusion models improve molecular sampling and simulation.
problem Inconsistency between diffusion model scores and equilibrium distributions.
method Fokker-Planck regularization to enforce consistency.
result Improved consistency and efficient sampling of biomolecular systems.
Energy-based models can generate complex images by combining simpler concepts.
problem Generating natural images that satisfy complex logical combinations of concepts.
method Energy-based models combine probability distributions of simpler concepts to generate compositions.
result Energy-based models can generate images that satisfy conjunctions, disjunctions, and negations of concepts.
This paper studies a training method to jointly estimate an energy-based model and a flow-based model, in which the two models are iteratively updated based on a shared adversarial value function. This joint training method has the following traits. (1) The update of the energy-based model is based on noise contrastive…
This paper proposes a multi-grid method for learning energy-based generative ConvNet models of images. For each grid, we learn an energy-based probabilistic model where the energy function is defined by a bottom-up convolutional neural network (ConvNet or CNN). Learning such a model requires generating synthesized exam…
Proposes an energy-based sliced Wasserstein distance for improved probability measure comparison.
problem Inefficiencies and limitations in existing sliced Wasserstein distance approaches.
method Introduces an energy-based slicing distribution for better performance and stability.
result Demonstrates superior performance of the EBSW distance in various applications.
This paper introduces a new neural ODE model for continuous-time sequence generation.
problem Representing and predicting continuous-time sequences with high accuracy.
method A neural emission model and neural ODE define the latent state evolution, with an Energy-based model for prior distribution.
result The model outperforms existing methods in various tasks, including long-horizon predictions.
3D object detection improved using energy-based models.
problem Accurate 3D object detection in cluttered environments from sparse LiDAR data.
method Designing a differentiable pooling operator for 3D bounding boxes integrated into a state-of-the-art 3D object detector.
result Our approach consistently outperforms the SA-SSD baseline across all 3DOD metrics on the KITTI dataset.
This paper proposes a more efficient training method for energy-based models.
problem The computational burden and validity trade-off in Contrastive Divergence training.
method Introducing Diffusion Contrastive Divergence (DCD) to replace Langevin dynamics with diffusion processes.
result The proposed DCDs are more computationally efficient and handle gradient terms better than Contrastive Divergence.
Paper proposes a method to improve MCMC sampling for energy-based models.
problem MCMC sampling of energy-based models is often not mixing in high-dimensional data.
method Proposes using a flow-based model as a backbone to correct the energy-based model, enabling mixing in latent space.
result MCMC sampling of the corrected EBM in the latent space mixes well and traverses modes in the data space.
STANLEY improves sampling for complex data models.
problem Training Energy-Based models with intractable normalizing constants.
method Anisotropic Langevin Dynamics with gradient-informed covariance.
result Geometrically uniformly ergodic Markov Chain for sampling.
Graph Energy Matching improves generation quality for molecular graphs.
problem Discrete energy-based models struggle with efficient and high-quality sampling for graph generation.
method Inspired by transport-map optimization, Graph Energy Matching learns a permutation-invariant potential energy to guide sampling.
result GEM matches or surpasses discrete diffusion baselines on molecular graph benchmarks.
iEFM trains CNF models from unnormalized densities efficiently.
problem Training generators from energy functions or unnormalized densities.
method Iterated energy-based flow matching (iEFM) with simulation-free objective.
result iEFM outperforms existing methods in probabilistic modeling.
This paper tackles sampling issues in latent space EBMs by introducing diffusion-based amortization.
problem Degenerate MCMC sampling quality hinders latent space EBM learning and generation quality.
method Introduces diffusion-based amortization for long-run MCMC sampling.
result The learned amortization of MCMC is a valid long-run MCMC sampler.
EMIX minimizes surprise in multi-agent reinforcement learning.
problem Surprise and approximation bias in multi-agent reinforcement learning.
method Energy-based MIXer (EMIX) for minimizing surprise across multiple agents.
result EMIX demonstrates consistent stable performance in challenging StarCraft II scenarios.
Unified view of contrastive and supervised learning improves model performance.
problem Improving model performance in classification, robustness, and detection.
method Hybrid discriminative-generative training of energy-based models.
result Improved performance on classification, robustness, out-of-distribution detection, and calibration.
A new method trains discrete EBMs without sampling.
problem Training EBMs on discrete spaces is hard.
method Energy Discrepancy (ED), a contrastive loss.
result ED offers theoretical guarantees for various perturbation types.
Unified model explains AT's generative ability.
problem Understanding the generative ability of AT.
method Contrastive Energy-based Models (CEM).
result Improved sample quality in supervised and unsupervised learning.
Future autonomous systems need reliable world models and complex action sequences.
problem Current automated systems lack reliable world models and complex action sequences.
method Introduce energy-based and latent variable models combined in a hierarchical joint embedding predictive architecture (H-JEPA).
result Combining energy-based and latent variable models in H-JEPA can lead to reliable world models and complex action sequences.
Improved language models by incorporating statistical discriminators.
problem Distinguishing model-generated text from real text reliably.
method Energy-Based Model framework to incorporate discriminators.
result Improves language model performance in perplexity and human evaluation.
EBM model for functional data using path measure tilting.
problem Modeling distributions of functions from irregularly sampled data.
method Spectral decomposition of an energy-based model with a Gaussian Process path measure to reweight the distribution.
result Effective approach to up-scaling functional data.
This paper proposes the divergence triangle as a framework for joint training of generator model, energy-based model and inference model. The divergence triangle is a compact and symmetric (anti-symmetric) objective function that seamlessly integrates variational learning, adversarial learning, wake-sleep algorithm, an…
Persistently trained EBMs generate images and estimate complex densities.
problem Challenges in ML learning for energy-based models, especially non-convergence of MCMC.
method Introduce diffusion data, learn a joint EBM through persistent training with enhanced sampling.
result First simultaneous achievement of stability, post-training image generation, and superior out-of-distribution detection for image data.
A new method improves training of energy-based models.
problem Training energy-based models is challenging.
method Diffusive Classification (DiffCLF) objective.
result DiffCLF enables EBMs with higher fidelity and broader applicability.
EBMs improve continual learning without external memory or regularization.
problem Improving continual learning without external memory or regularization.
method Energy-Based Models with contrastive divergence training objective.
result EBMs outperform baseline methods on various benchmarks.