Versatile model for High Energy Physics events.
problem Modeling complex interactions in high-energy physics data.
method Energy-based probabilistic model with multi-purpose architecture.
result Achieves success in diverse applications like simulation, anomaly detection, and particle identification.
Improves sample quality of generative models using energy-based methods.
problem Low sample quality in generative models.
method Constructs an energy function on latent space, trains an energy-based model, and generates improved samples.
result Significant improvement in sample quality with minimal computational overhead.
A neural network model minimizes region-based free energy for faster inference in MRFs.
problem Efficient inference in complex Markov random fields (MRFs).
method Region-based Energy Neural Network (RENN) that directly minimizes region-based free energy.
result RENN outperforms other methods in marginal distribution estimation, partition function estimation, and MRF learning.
GEBM combines energy function and base distribution for better generative modeling.
problem Improving generative models with better quality samples and performance.
method Alternating training between energy function and base distribution, using MCMC for sampling.
result GEBMs produce higher quality samples and better performance than GANs.
A new method trains discrete EBMs without sampling.
problem Training EBMs on discrete spaces is hard.
method Energy Discrepancy (ED), a contrastive loss.
result ED offers theoretical guarantees for various perturbation types.
EBR improves NMT by re-ranking samples drawn from MLE-trained models.
problem Discrepancy between MLE and BLEU score in neural machine translation.
method Train an energy-based model to mimic BLEU score, then use it for re-ranking.
result EBR consistently improves NMT performance by +4 BLEU points on IWSLT'14 German-English.
A new loss function ED simplifies training energy-based models without scores.
problem Training energy-based models is computationally expensive.
method Energy Discrepancy (ED) loss function that does not rely on scores or MCMC.
result ED effectively interpolates between score matching and negative log-likelihood.
Energy Matching unifies flow matching and energy-based models for generative modeling.
problem Inability of flow-based models to integrate partial observations and priors.
method Energy Matching framework that integrates flow matching and energy-based models.
result Substantially outperforms existing EBMs on CIFAR-10 and ImageNet generation.
Energy Transformer integrates attention, energy models, and associative memory.
problem Lack of clear theoretical foundations in attention mechanisms and straightforward design of energy functions in energy-based models.
method Proposes Energy Transformer, a sequence of attention layers with a specifically engineered energy function.
result Obtained strong results on graph anomaly detection and classification tasks.
New method learns latent energy models using particle algorithms.
problem Learning latent variable models with energy priors.
method Continuous-time SDEs for MMLE, particle-based discretization.
result Practical algorithm converges to solve MMLE problem.
EB-RANSAC uses energy-based model for robust estimation without complex sampling.
problem Robust estimation of parameters in noisy data.
method EB-RANSAC combines RANSAC's sampling scheme with an energy-based model, simplifying the process and reducing hyperparameter requirements.
result EB-RANSAC effectively solves linear regression and maximum likelihood estimation problems.
Paper proposes CoopFlow, a two-flow generator for energy-based models.
problem Training energy-based models with Langevin flow and normalizing flow.
method CoopFlow trains an energy-based model using a normalizing flow initialization and a short-run Langevin flow revision.
result CoopFlow converges to a moment matching estimator and synthesizes realistic images.
Graph Energy Matching improves generation quality for molecular graphs.
problem Discrete energy-based models struggle with efficient and high-quality sampling for graph generation.
method Inspired by transport-map optimization, Graph Energy Matching learns a permutation-invariant potential energy to guide sampling.
result GEM matches or surpasses discrete diffusion baselines on molecular graph benchmarks.
mAIS improves free energy evaluation efficiency.
problem Computational infeasibility of exact free energy evaluation.
method mAIS, a marginalized version of AIS.
result mAIS is more efficient under certain conditions.
ERM uses energy-based selection to improve recursive reasoning.
problem Lack of principled inference mechanism in recursive models.
method Energy-guided Recursive Model (ERM) introduces Hopfield energies for trajectory selection.
result ERM achieves optimal solutions on various puzzles.
EBM predicts protein conformations at atomic scale using crystallized data.
problem Predicting the conformation of a side chain from its context within a protein structure.
method Energy-based model trained on crystallized protein data, evaluating performance on rotamer recovery task.
result EBM achieves performance close to state-of-the-art methods, including Rosetta energy function.
Paper proposes an alternative to MCMC for sampling in energy-based models.
problem Difficulty in generating samples from the current energy function in contrastive approaches.
method Viewing the evolution of the modeling distribution as the evolution of the energy function and samples from this distribution along a time-dependent vector field.
result The proposed method efficiently matches the current distribution in a finite time, unlike MCMC.
Quantum computing offers energy savings over classical computing.
problem Energy efficiency in computing services.
method Cournot competition model constrained by energy usage.
result Quantum computing firms can outperform classical counterparts in energy efficiency.
Introduces Causal Energy Minimization to understand Transformer layers.
problem Empirical parameterization of Transformer blocks remains largely unexplored.
method Causal Energy Minimization framework that recasts Transformer layers as optimization steps on conditional energy functions.
result Identifies design space for Transformer layers including weight sharing and energy-based interpretations.
Paper proposes energy objective for training normalizing flows without determinants.
problem Challenges in training normalizing flows due to Jacobian determinants.
method Introduces energy objective based on proper scoring rules, determinant-free.
result Energy objective supports novel model families and competitive performance.
iEFM trains CNF models from unnormalized densities efficiently.
problem Training generators from energy functions or unnormalized densities.
method Iterated energy-based flow matching (iEFM) with simulation-free objective.
result iEFM outperforms existing methods in probabilistic modeling.
TPBS models improve robustness to overfitting with localized Dirichlet energy regularization.
problem Global Dirichlet energy-based regularization fails for TPBS models due to perfect interpolation.
method Propose local Dirichlet energy regularization and two inference estimators.
result TPBS models outperform neural networks in overfitting regimes and maintain competitive performance otherwise.
Adversarial attacks can fool ML energy theft detection models.
problem Vulnerability of ML-based energy theft detection models to adversarial attacks.
method Design of an adversarial measurement generation algorithm.
result ML models can be significantly fooled by adversarial attacks, reducing their detection accuracy.
Improves sampling quality in model composition using MH-like acceptance rule for score-based diffusion models.
problem Inability to apply MH corrections in score-based diffusion models for model composition.
method Introduces a novel MH-like acceptance rule based on line integration of the score function.
result Relative improvements similar to energy-based models without explicit energy parameterization.
EBPs model exchangeable data with flexible distributions.
problem Current energy-based models restrict set cardinality and limited distribution forms.
method Introduced Energy-Based Processes (EBPs) that extend energy models to exchangeable data with neural network parameterizations.
result EBPs can express more flexible distributions over sets without cardinality restrictions.
A new method improves training of energy-based models.
problem Training energy-based models is challenging.
method Diffusive Classification (DiffCLF) objective.
result DiffCLF enables EBMs with higher fidelity and broader applicability.
Efficiently samples and learns densities with symmetries using equivariant methods.
problem Efficiently sampling and learning densities with symmetries.
method Equivariant Stein Variational Gradient Descent (SVGD) and equivariant energy based models.
result Improves and scales up training of energy based models.
Paper proposes a method to improve MCMC sampling for energy-based models.
problem MCMC sampling of energy-based models is often not mixing in high-dimensional data.
method Proposes using a flow-based model as a backbone to correct the energy-based model, enabling mixing in latent space.
result MCMC sampling of the corrected EBM in the latent space mixes well and traverses modes in the data space.
This paper proposes the divergence triangle as a framework for joint training of generator model, energy-based model and inference model. The divergence triangle is a compact and symmetric (anti-symmetric) objective function that seamlessly integrates variational learning, adversarial learning, wake-sleep algorithm, an…
BiDVL improves EBLVMs for visual tasks by optimizing two variational distributions.
problem Training EBLVMs is challenging due to intractable distributions.
method Bi-level doubly variational learning with two tractable distributions.
result BiDVL achieves impressive image generation and reconstruction performance.
CityTFT models urban building energy using a data-driven approach.
problem Current UBEM methods are time-consuming and based on physics.
method CityTFT uses a TFT framework with an augmented loss function.
result CityTFT predicts energy demands with high accuracy.
We study the problem of learning associative memory -- a system which is able to retrieve a remembered pattern based on its distorted or incomplete version. Attractor networks provide a sound model of associative memory: patterns are stored as attractors of the network dynamics and associative retrieval is performed by…
This paper proposes a multi-grid method for learning energy-based generative ConvNet models of images. For each grid, we learn an energy-based probabilistic model where the energy function is defined by a bottom-up convolutional neural network (ConvNet or CNN). Learning such a model requires generating synthesized exam…
Probabilistic models can be defined by an energy function, where the probability of each state is proportional to the exponential of the state's negative energy. This paper considers a generalization of energy-based models in which the probability of a state is proportional to an arbitrary positive, strictly decreasing…
New method trains energy-based models faster and more stably.
problem Training efficiency and stability of energy-based models.
method EBFlow with score-matching objectives.
result EBFlow achieves significant speedup and better performance.
Improved diffusion models using energy distillation and sequential Monte Carlo.
problem Training instability and inferior performance in energy parameterized diffusion models.
method Introduced a novel training regime for energy functions through distillation of pre-trained diffusion models, and cast the sampling procedure as a Feynman Kac model.
result Demonstrated improved performance and new sampling techniques.
ScoreGrad predicts multivariate time series with energy-based models, achieving state-of-the-art results.
problem Predicting multivariate time series with generative models while considering noise and distribution.
method ScoreGrad uses continuous energy-based generative models with a feature extraction and score matching module.
result ScoreGrad achieves state-of-the-art results on six real-world datasets.
Training energy-based probabilistic models is confronted with apparently intractable sums, whose Monte Carlo estimation requires sampling from the estimated probability distribution in the inner loop of training. This can be approximately achieved by Markov chain Monte Carlo methods, but may still face a formidable obs…
CEBMs learn flexible latent mappings from data.
problem Learning flexible latent mappings from data.
method CEBMs decompose joint density into tractable posterior over latent variables.
result CEBMs achieve competitive results in image modeling and latent space predictive power.
Proposes ENOs for learning PDE solutions that conserve energy.
problem Learning dynamics that obey physical laws, especially in super-resolution settings.
method Energy-consistent Neural Operators (ENOs) with a novel penalty function inspired by energy-based theory.
result ENOs outperform existing DNN models in predicting solutions from data, especially in super-resolution settings.
Improved generative models using overparametrized shallow neural networks.
problem Improving generative models for data with hidden low-dimensional structure.
method Using energy-based models with overparametrized shallow neural networks as approximators.
result Models trained in the 'active' regime outperform those in the 'lazy' or kernel regime, leading to better adaptivity to hidden structure.
Modern Hopfield networks help prevent forgetting in generative models after task changes.
problem How to prevent forgetting in generative models after task changes.
method Introduce intrinsic forgetting as an increase in Hopfield energy after task change, analyze memory replay effectiveness, and validate predictions in experiments.
result High-energy, outlier-like samples are more forgettable than cluster-like samples, and energy-based selection of replay samples mitigates forgetting.
Proposes linking energy and force uncertainty in deep learning potentials.
problem Uncertainty in predicted energies and forces in machine learning models.
method Introduces a spatially correlated noise process to link energy and force uncertainty.
result Demonstrates the approach on molecular datasets, linking energy and force uncertainties.
A method to automatically learn proposal distributions for energy-based regression models.
problem Manual design and initial estimate of proposal distributions for energy-based regression models.
method Introduces a method to learn an effective proposal distribution automatically, parameterized by a separate network head, and derives a unified training objective to minimize KL divergence and negative log-likelihood.
result Consistently outperforms conventional MDN training on four real-world regression tasks within computer vision.
This paper studies a training method to jointly estimate an energy-based model and a flow-based model, in which the two models are iteratively updated based on a shared adversarial value function. This joint training method has the following traits. (1) The update of the energy-based model is based on noise contrastive…
New sampler tackles complex discrete energy landscapes efficiently.
problem Stagnation in gradient-based discrete samplers for non-convex settings.
method DREXEL sampler with Replica Exchange and Adjusted Metropolis.
result Proves samplers satisfy detailed balance and converge to target distribution.
Graph neural networks are explained through energy gradient flow and framelet decomposition.
problem Understanding and improving graph neural networks.
method Viewing framelet-based models as gradient flows of energy, proposing a generalized energy via framelet decomposition.
result The proposed model leads to more flexible dynamics, enhancing graph neural networks.
Sustainability became the most important component of world development, as countries worldwide fight the battle against the climate change. To understand the effects of climate change, the ecological footprint, along with the biocapacity should be observed. The big part of the ecological footprint, the carbon footprin…