New method learns latent energy models using particle algorithms.
problem Learning latent variable models with energy priors.
method Continuous-time SDEs for MMLE, particle-based discretization.
result Practical algorithm converges to solve MMLE problem.
EB-RANSAC uses energy-based model for robust estimation without complex sampling.
problem Robust estimation of parameters in noisy data.
method EB-RANSAC combines RANSAC's sampling scheme with an energy-based model, simplifying the process and reducing hyperparameter requirements.
result EB-RANSAC effectively solves linear regression and maximum likelihood estimation problems.
Improves sample quality of generative models using energy-based methods.
problem Low sample quality in generative models.
method Constructs an energy function on latent space, trains an energy-based model, and generates improved samples.
result Significant improvement in sample quality with minimal computational overhead.
A method to automatically learn proposal distributions for energy-based regression models.
problem Manual design and initial estimate of proposal distributions for energy-based regression models.
method Introduces a method to learn an effective proposal distribution automatically, parameterized by a separate network head, and derives a unified training objective to minimize KL divergence and negative log-likelihood.
result Consistently outperforms conventional MDN training on four real-world regression tasks within computer vision.
New method trains energy-based models faster and more stably.
problem Training efficiency and stability of energy-based models.
method EBFlow with score-matching objectives.
result EBFlow achieves significant speedup and better performance.
This paper studies a training method to jointly estimate an energy-based model and a flow-based model, in which the two models are iteratively updated based on a shared adversarial value function. This joint training method has the following traits. (1) The update of the energy-based model is based on noise contrastive…
A new loss function ED simplifies training energy-based models without scores.
problem Training energy-based models is computationally expensive.
method Energy Discrepancy (ED) loss function that does not rely on scores or MCMC.
result ED effectively interpolates between score matching and negative log-likelihood.
Efficiently samples and learns densities with symmetries using equivariant methods.
problem Efficiently sampling and learning densities with symmetries.
method Equivariant Stein Variational Gradient Descent (SVGD) and equivariant energy based models.
result Improves and scales up training of energy based models.
STANLEY improves sampling for complex data models.
problem Training Energy-Based models with intractable normalizing constants.
method Anisotropic Langevin Dynamics with gradient-informed covariance.
result Geometrically uniformly ergodic Markov Chain for sampling.
This paper proposes a multi-grid method for learning energy-based generative ConvNet models of images. For each grid, we learn an energy-based probabilistic model where the energy function is defined by a bottom-up convolutional neural network (ConvNet or CNN). Learning such a model requires generating synthesized exam…
Unified model explains AT's generative ability.
problem Understanding the generative ability of AT.
method Contrastive Energy-based Models (CEM).
result Improved sample quality in supervised and unsupervised learning.
A deep learning method solves nonlinear filtering problems efficiently.
problem Nonlinear filtering problem
method Deep splitting method combined with energy-based neural network approximation
result Computational efficiency and performance comparable to Kalman and bootstrap filters
Paper proposes CoopFlow, a two-flow generator for energy-based models.
problem Training energy-based models with Langevin flow and normalizing flow.
method CoopFlow trains an energy-based model using a normalizing flow initialization and a short-run Langevin flow revision.
result CoopFlow converges to a moment matching estimator and synthesizes realistic images.
This paper tackles sampling issues in latent space EBMs by introducing diffusion-based amortization.
problem Degenerate MCMC sampling quality hinders latent space EBM learning and generation quality.
method Introduces diffusion-based amortization for long-run MCMC sampling.
result The learned amortization of MCMC is a valid long-run MCMC sampler.
EBMs improve continual learning without external memory or regularization.
problem Improving continual learning without external memory or regularization.
method Energy-Based Models with contrastive divergence training objective.
result EBMs outperform baseline methods on various benchmarks.
This paper proposes a more efficient training method for energy-based models.
problem The computational burden and validity trade-off in Contrastive Divergence training.
method Introducing Diffusion Contrastive Divergence (DCD) to replace Langevin dynamics with diffusion processes.
result The proposed DCDs are more computationally efficient and handle gradient terms better than Contrastive Divergence.
This paper proposes the divergence triangle as a framework for joint training of generator model, energy-based model and inference model. The divergence triangle is a compact and symmetric (anti-symmetric) objective function that seamlessly integrates variational learning, adversarial learning, wake-sleep algorithm, an…
Versatile model for High Energy Physics events.
problem Modeling complex interactions in high-energy physics data.
method Energy-based probabilistic model with multi-purpose architecture.
result Achieves success in diverse applications like simulation, anomaly detection, and particle identification.
A new method improves training of energy-based models.
problem Training energy-based models is challenging.
method Diffusive Classification (DiffCLF) objective.
result DiffCLF enables EBMs with higher fidelity and broader applicability.
3D object detection improved using energy-based models.
problem Accurate 3D object detection in cluttered environments from sparse LiDAR data.
method Designing a differentiable pooling operator for 3D bounding boxes integrated into a state-of-the-art 3D object detector.
result Our approach consistently outperforms the SA-SSD baseline across all 3DOD metrics on the KITTI dataset.
Proposes Learnergy, a Python framework for energy-based machine learning.
problem Lack of research and implementations around energy-based systems.
method Built upon PyTorch, Learnergy provides a more friendly environment and faster prototyping.
result Learnergy speeds up computational time using CUDA.
Proposes an energy-based sliced Wasserstein distance for improved probability measure comparison.
problem Inefficiencies and limitations in existing sliced Wasserstein distance approaches.
method Introduces an energy-based slicing distribution for better performance and stability.
result Demonstrates superior performance of the EBSW distance in various applications.
A new method for decomposing non-negative tensors using energy-based modeling.
problem Challenges in traditional tensor decomposition methods, especially global optimization and rank selection.
method Energy-based modeling of tensors, considering interactions between modes for global optimization.
result Demonstrates effectiveness in tensor completion and approximation, revealing a relationship between many-body and low-rank approximations.
Paper proposes a method to improve MCMC sampling for energy-based models.
problem MCMC sampling of energy-based models is often not mixing in high-dimensional data.
method Proposes using a flow-based model as a backbone to correct the energy-based model, enabling mixing in latent space.
result MCMC sampling of the corrected EBM in the latent space mixes well and traverses modes in the data space.
A new method trains discrete EBMs without sampling.
problem Training EBMs on discrete spaces is hard.
method Energy Discrepancy (ED), a contrastive loss.
result ED offers theoretical guarantees for various perturbation types.
iEFM trains CNF models from unnormalized densities efficiently.
problem Training generators from energy functions or unnormalized densities.
method Iterated energy-based flow matching (iEFM) with simulation-free objective.
result iEFM outperforms existing methods in probabilistic modeling.
Two synthetic likelihood methods learn EBM of likelihood from simulator data for SBI.
problem Conduct inference from experimental observations using high-fidelity simulators.
method Learn conditional EBM of likelihood using synthetic data conditioned on parameters.
result Learned likelihood combined with prior yields posterior estimate for sampling.
BiDVL improves EBLVMs for visual tasks by optimizing two variational distributions.
problem Training EBLVMs is challenging due to intractable distributions.
method Bi-level doubly variational learning with two tractable distributions.
result BiDVL achieves impressive image generation and reconstruction performance.
VAV method optimizes learning rate for faster, stable SGD convergence.
problem Optimizing learning rate for efficient and stable machine learning models.
method Energy-based self-adaptive learning rate with auxiliary variable r. result VAV method achieves faster convergence and superior stability with larger learning rates.
A new method trains and samples from energy-based models using diffusion recovery likelihood.
problem Training and sampling high-dimensional datasets with energy-based models is challenging.
method Trains EBMs with a diffusion recovery likelihood method, maximizing conditional probabilities of data at different noise levels.
result Generates high-fidelity images with low FID and inception scores, and accurately estimates normalized data density.
EMIX minimizes surprise in multi-agent reinforcement learning.
problem Surprise and approximation bias in multi-agent reinforcement learning.
method Energy-based MIXer (EMIX) for minimizing surprise across multiple agents.
result EMIX demonstrates consistent stable performance in challenging StarCraft II scenarios.
New method for LVEBMs using saddle-point optimization and Langevin updates.
problem Expressive generative modeling of latent variables with hidden structure.
method Reformulate LVEBM training as a saddle problem, using Langevin updates and gradient flows.
result Proves existence and convergence of the algorithm under standard assumptions, with improved ELBO bounds.
Paper proposes an alternative to MCMC for sampling in energy-based models.
problem Difficulty in generating samples from the current energy function in contrastive approaches.
method Viewing the evolution of the modeling distribution as the evolution of the energy function and samples from this distribution along a time-dependent vector field.
result The proposed method efficiently matches the current distribution in a finite time, unlike MCMC.
CEBMs learn flexible latent mappings from data.
problem Learning flexible latent mappings from data.
method CEBMs decompose joint density into tractable posterior over latent variables.
result CEBMs achieve competitive results in image modeling and latent space predictive power.
EBPs model exchangeable data with flexible distributions.
problem Current energy-based models restrict set cardinality and limited distribution forms.
method Introduced Energy-Based Processes (EBPs) that extend energy models to exchangeable data with neural network parameterizations.
result EBPs can express more flexible distributions over sets without cardinality restrictions.
Persistently trained EBMs generate images and estimate complex densities.
problem Challenges in ML learning for energy-based models, especially non-convergence of MCMC.
method Introduce diffusion data, learn a joint EBM through persistent training with enhanced sampling.
result First simultaneous achievement of stability, post-training image generation, and superior out-of-distribution detection for image data.
EBR improves NMT by re-ranking samples drawn from MLE-trained models.
problem Discrepancy between MLE and BLEU score in neural machine translation.
method Train an energy-based model to mimic BLEU score, then use it for re-ranking.
result EBR consistently improves NMT performance by +4 BLEU points on IWSLT'14 German-English.
The problem of continuous inverse optimal control (over finite time horizon) is to learn the unknown cost function over the sequence of continuous control variables from expert demonstrations. In this article, we study this fundamental problem in the framework of energy-based model, where the observed expert trajectori…
Bidirectional bounds stabilize training of energy-based models.
problem Training energy-based models is difficult and prone to instability.
method Propose bidirectional bounds linking to gradient penalty and Jacobi-determinant estimator.
result Significant stabilization and high-quality density estimation achieved.
ScoreGrad predicts multivariate time series with energy-based models, achieving state-of-the-art results.
problem Predicting multivariate time series with generative models while considering noise and distribution.
method ScoreGrad uses continuous energy-based generative models with a feature extraction and score matching module.
result ScoreGrad achieves state-of-the-art results on six real-world datasets.
A new method normalizes EBM training by introducing a learnable parameter.
problem Training energy-based models with maximum likelihood is challenging due to intractable normalisation constants.
method Proposes a self-normalised log-likelihood (SNL) objective that introduces a learnable parameter representing the normalisation constant.
result The SNL objective is a lower bound of the log-likelihood and can be directly optimised using stochastic gradient techniques.
Proposes a new learning method for RBMs that combines strengths of forward and reverse KLD.
problem Underfitting and mode-collapse issues in RBM learning.
method Ratio divergence learning using target energy.
result Significantly outperforms other learning methods in energy function fitting, mode-covering, and stability.
Paper proposes a method to design molecules with specific properties.
problem Designing molecules with desired chemical and biological properties.
method Energy-based model in latent space, SGDS algorithm for gradual distribution shifting.
result Method achieves strong performances on various molecule design tasks.
An energy based approach for stabilizing a mechanical system has offered a simple yet powerful control scheme. However, since it does not impose such strong constraints on parameter space of the controller, finding appropriate parameter values for an optimal controller is known to be hard. This paper intends to generat…
This paper proposes a method to train energy-based models using variational auto-encoders for efficient sampling.
problem Training energy-based models by maximum likelihood is challenging due to intractable partition functions and difficult sampling from the model distribution.
method The authors propose using a variational auto-encoder to initialize finite-step MCMC sampling, specifically Langevin dynamics, to train the energy-based model.
result The proposed method enables training energy-based models using maximum likelihood, generating samples comparable to GANs and EBMs.
We propose to reinterpret a standard discriminative classifier of p(y|x) as an energy based model for the joint distribution p(x,y). In this setting, the standard class probabilities can be easily computed as well as unnormalized values of p(x) and p(x|y). Within this framework, standard discriminative architectures ma…
RNE provides a flexible framework for diffusion models, enabling inference-time control and energy-based training.
problem Insufficient knowledge of marginal densities in diffusion models.
method Introduces Radon-Nikodym Estimator (RNE) to reveal the connection between marginal densities and transition kernels.
result RNE delivers strong results in inference-time control and energy-based diffusion training.
MPT improves CNN and energy-based models' OOD detection and generalization.
problem Challenging out-of-distribution detection in computer vision.
method Applying Maximum Probability Theorem as a regularization scheme in CNN and energy-based models.
result MPT-based regularization strategy stabilizes and improves generalization and robustness of base models.