Paper proposes energy objective for training normalizing flows without determinants.
problem Challenges in training normalizing flows due to Jacobian determinants.
method Introduces energy objective based on proper scoring rules, determinant-free.
result Energy objective supports novel model families and competitive performance.
This paper improves normalizing flows by combining MLE and sliced-Wasserstein distance for better data fidelity.
problem Normalizing flows struggle with generating realistic data and detecting out-of-distribution data.
method Proposes a hybrid objective function combining MLE and sliced-Wasserstein distance.
result Shows better generative abilities and lower likelihood of out-of-distribution data.
Paper proposes methods to learn sub-manifolds and estimate densities in normalizing flows.
problem Normalizing flows struggle with finding sub-manifolds in high-dimensional data.
method Introduces per-pixel penalized log-likelihood and hierarchical training approaches.
result Validated superior performance in manifold learning and density estimation.
Normalizing flows optimize Jacobian determinant for unique likelihood objective.
problem Optimizing normalizing flows for unique likelihood.
method Showed Jacobian determinant is unique for given distributions, leading to a unique global optimum. Used eigenvalues of auto-correlation matrix for explicit likelihood expression.
result Explicit expression of likelihood for flows, independent of neural network parameterization, with theoretical optimal value.
Paper proposes efficient training for normalizing flows in Boltzmann generators.
problem Training normalizing flows for Boltzmann generators is computationally challenging and unstable.
method Regression Training of Normalizing Flows (RegFlow) using ℓ2-regression. result RegFlow enables efficient and stable training of normalizing flows for Boltzmann generators.
Flowification enriches neural networks with an inverse pass and likelihood monitoring.
problem Neural networks lack an inverse pass and likelihood monitoring, limiting their generative capabilities.
method Introduce flowification, enriching neural networks with a stochastic inverse pass and likelihood monitoring.
result Certain neural network architectures can be enriched to fall under the generalized notion of a normalizing flow.
SurVAE Flows combine VAEs and flows using surjective transformations.
problem Combining the strengths of VAEs and flows to model complex densities.
method Modular framework of composable deterministic and stochastic transformations.
result Exact likelihood computation and lower bound on likelihood.
SNPLA uses normalizing flows for efficient inference in implicit models.
problem Efficient inference in implicit models with complex likelihood and posterior learning.
method Sequential Neural Posterior and Likelihood Approximation (SNPLA) algorithm using normalizing flows.
result SNPLA achieves competitive performance with faster posterior draws compared to MCMC methods.
FF algorithm uses goodness as a likelihood-ratio test for scalar normalization.
problem Training each layer locally with scalar goodness.
method FF algorithm uses a likelihood-ratio test with squared goodness as the sufficient statistic.
result The FF algorithm generalizes to anisotropic and heavy-tailed populations.
Likelihood from a generative model is a natural statistic for detecting out-of-distribution (OoD) samples. However, generative models have been shown to assign higher likelihood to OoD samples compared to ones from the training distribution, preventing simple threshold-based detection rules. We demonstrate that OoD det…
The normalized maximized likelihood (NML) provides the minimax regret solution in universal data compression, gambling, and prediction, and it plays an essential role in the minimum description length (MDL) method of statistical modeling and estimation. Here we show that the normalized maximum likelihood has a Bayes-li…
Operational risk models commonly employ maximum likelihood estimation (MLE) to fit loss data to heavy-tailed distributions. Yet several desirable properties of MLE (e.g. asymptotic normality) are generally valid only for large sample-sizes, a situation rarely encountered in operational risk. In this paper, we study how…
A new method estimates marginal likelihood using normalizing flows.
problem Estimating marginal likelihood in Bayesian model selection.
method Learned harmonic mean estimator using normalizing flows.
result Normalizing flows avoid the exploding variance problem.
New method trains any neural network as a generative model.
problem Constrained design of normalizing flows due to analytical invertibility.
method Efficient gradient estimator for non-analytically invertible networks.
result Any dimension-preserving neural network can be used as a generative model.
Maximum likelihood training improves the performance of score-based diffusion models.
problem Training score-based diffusion models with maximum likelihood.
method Trained by minimizing a weighted combination of score matching losses, with a specific weighting scheme that bounds negative log-likelihood.
result Maximum likelihood training improves the log-likelihood of score-based diffusion models across multiple datasets.
A new base distribution for normalizing flows allows modeling complex distributions without sacrificing invertibility.
problem Normalizing flows struggle with complex, non-trivial distributions.
method Learned rejection sampling for base distribution, combined with optimization of log-likelihood and Kullback-Leibler divergence.
result The method effectively models complicated distributions without sacrificing invertibility.
Proposes LFGP for likelihood-free Gaussian process regression.
problem Inability to set likelihood functions in unknown probability models.
method Clusters and approximates likelihood using asymptotic normality.
result Reduces assumptions and computational costs for scalable problems.
New method for robust distribution alignment using log-likelihood ratio and normalizing flows.
problem Distribution alignment challenges in deep learning.
method Log-likelihood ratio statistic and normalizing flows.
result Minimizing the proposed objective yields robust domain alignment.
The paper strengthens the classical result of MLE convergence to a Gaussian distribution.
problem The classical result of MLE convergence to a Gaussian distribution.
method Sub-Gaussian concentration and entropic normality of the normalized MLE.
result Entropic central limit theorem for a smoothed version of the estimator.
Normalizing flows improve density estimation from noisy data.
problem Estimating underlying density from noisy samples.
method Use normalizing flows for density estimation with arbitrary noise distributions, using amortized variational inference.
result Normalizing flows can outperform Gaussian mixtures for density deconvolution.
A new path gradient estimator speeds up normalizing flows without sacrificing accuracy.
problem High computational cost and limited scalability of path gradient estimators for normalizing flows.
method Proposed a fast path gradient estimator that improves computational efficiency and scalability.
result The new estimator achieves superior performance and reduced variance across various applications.
Study shows limits of certain normalizing flows in higher dimensions.
problem Understanding the representation power of normalizing flows in different dimensions.
method Rigorously established bounds on expressive power of basic normalizing flows.
result Limited representation power in higher dimensions, especially with moderate depth.
Improves normalizing flows by incorporating data dependencies.
problem Current normalizing flow learning assumes independent data, leading to errors.
method Proposes a likelihood objective with dependencies and efficient learning algorithm.
result Improves density estimation and data generation on real-world data.
Stochastic normalizing flows use SDEs for efficient training and sampling.
problem Efficient maximum likelihood estimation and variational inference.
method Continuous normalizing flows extended with stochastic differential equations (SDEs) and rough path theory.
result Stochastic normalizing flows enable efficient training and sampling from complex distributions.
In Divide & Recombine (D&R), big data are divided into subsets, each analytic method is applied to subsets, and the outputs are recombined. This enables deep analysis and practical computational performance. An innovate D\&R procedure is proposed to compute likelihood functions of data-model (DM) parameters for big dat…
Normalizing flow regression approximates posterior distributions without additional sampling.
problem Bayesian inference with computationally expensive likelihood evaluations.
method Normalizing flow regression (NFR) for offline inference.
result NFR yields a tractable posterior approximation through regression on existing log-density evaluations.
A method for converting NIW parameters for better estimation.
problem Estimating parameters of multivariate normal distribution.
method Convergent procedure for converting mean parameters to natural parameters in NIW family.
result Maximum likelihood estimation of natural parameters from observed statistics.
Optimizing full likelihoods adapts loss scales and shapes for robust modeling.
problem Rigid loss functions limit model adaptability and robustness.
method Optimize full likelihoods with adjustable parameters.
result Adaptive tuning of loss scales and shapes improves model robustness.
New method trains EBMs using NFs for more accurate likelihood estimation.
problem Lack of statistical accuracy in EBMs likelihood estimation.
method Uses normalizing flows (NF) to fit an NF to an EBM during training.
result Accurate gradient for EBMs at all times, leading to a fast sampler.
FF algorithm uses goodness as a measure of input quality, derived from likelihood-ratio tests.
problem Training each layer locally with a goodness measure.
method FF algorithm uses a likelihood-ratio test to define goodness, which is the sum of squared activations normalized between layers.
result The goodness measure is a sufficient statistic for a likelihood-ratio test, explaining the FF algorithm's performance.
Paper proposes efficient method to calculate Fisher-Bingham distribution normalizing constant.
problem Efficiently calculating the normalizing constant of Fisher-Bingham distributions.
method Numerical integration with continuous Euler transform to Fourier-type integral representation.
result The method is fast and accurate, applicable to high-dimensional distributions.
This paper shows how to perform likelihood inference for complex graphical models efficiently.
problem Intractable normalizing constants in fully and partially observed exponential family graphical models.
method Using a technique from Geyer (1991), the paper estimates the normalizing constant and its gradient.
result Full likelihood-based analysis is feasible and computationally efficient for these models.
A new method normalizes EBM training by introducing a learnable parameter.
problem Training energy-based models with maximum likelihood is challenging due to intractable normalisation constants.
method Proposes a self-normalised log-likelihood (SNL) objective that introduces a learnable parameter representing the normalisation constant.
result The SNL objective is a lower bound of the log-likelihood and can be directly optimised using stochastic gradient techniques.
New method for efficient maximum likelihood estimation of p-generalized probit regression.
problem Efficient estimation of p-generalized probit regression models. method Combining sketching techniques with importance subsampling to obtain a coreset.
result Maximum likelihood estimator can be approximated efficiently up to a factor of (1+ε) on large data. A new layer, funnel, reduces dimensionality in flows for better performance.
problem Training high-dimensional models efficiently and accurately.
method Constructing dimension-reducing surjective flows using the funnel layer.
result The funnel layer improves model performance with a smaller latent space.
E-NFs generate molecules and their positions while preserving Euclidean symmetries.
problem Generating molecules with their positions while preserving Euclidean symmetries.
method Integrating E(n) graph neural networks into a differential equation to create an invertible equivariant function.
result E-NFs significantly outperform baselines and existing methods in log-likelihood for particle systems and molecules.
Paper proposes CoopFlow, a two-flow generator for energy-based models.
problem Training energy-based models with Langevin flow and normalizing flow.
method CoopFlow trains an energy-based model using a normalizing flow initialization and a short-run Langevin flow revision.
result CoopFlow converges to a moment matching estimator and synthesizes realistic images.
Bayesian inference for expensive likelihoods using Langevin Monte Carlo with NF.
problem Sampling from complex posterior distributions with expensive likelihoods.
method Deterministic Langevin equation with NF gradient, Metropolis-Hastings updates.
result Competitive performance compared to state-of-the-art methods.
We study asymptotic properties of maximum likelihood estimators of drift parameters for a jump-type Heston model based on continuous time observations, where the jump process can be any purely non-Gaussian Lévy process of not necessarily bounded variation with a Lévy measure concentrated on (−1,∞). We prove stro…
Bayesian synthetic likelihood (BSL) is a popular method for estimating the parameter posterior distribution for complex statistical models and stochastic processes that possess a computationally intractable likelihood function. Instead of evaluating the likelihood, BSL approximates the likelihood of a judiciously chose…
This research improves neural likelihood approximation for Bayesian inverse problems.
problem Challenges in modeling and inference for high-dimensional Bayesian inverse problems.
method Develops a strictly convex approximation framework for neural likelihood.
result Empirical minimizers converge to the true likelihood as sample size increases.
A new normalizing flow models continuous stochastic processes efficiently.
problem Efficient modeling of continuous stochastic processes.
method Dynamic normalizing flows driven by Wiener process.
result Rich time series model with efficient computation of likelihoods and marginals.
ManiFlow models manifold data by optimizing NFs on perturbed data.
problem Capturing manifold data with NFs' invertibility constraint.
method Train NFs on perturbed data to implicitly represent manifold.
result NFs implicitly model manifold in regions of maximum likelihood.
Paper extends LME models to allow sign constraints on coefficients with SDTN random effects.
problem Inference with sign constraints on random effects in LME models.
method Proposes SDTN distribution for random effects and develops likelihood-based approaches for estimation.
result Proposed constrained model improves real-world interpretations and achieves satisfactory performance.
An efficient LDP protocol for QMLE with improved practicality and theoretical guarantees.
problem Difficult implementation of existing LDP QMLE for large-scale surveys.
method Developed an alternative LDP protocol without long waiting time, high communication cost, and derivative boundedness assumptions.
result Sufficient conditions for consistency and asymptotic normality of the protocol.
Normalizing flows transform a latent distribution through an invertible neural network for a flexible and pleasingly simple approach to generative modelling, while preserving an exact likelihood. We propose FlowGMM, an end-to-end approach to generative semi supervised learning with normalizing flows, using a latent Gau…
Normalizing Flows (NFs) are able to model complicated distributions p(y) with strong inter-dimensional correlations and high multimodality by transforming a simple base density p(z) through an invertible neural network under the change of variables formula. Such behavior is desirable in multivariate structured predicti…
Normalizing Flows are a promising new class of algorithms for unsupervised learning based on maximum likelihood optimization with change of variables. They offer to learn a factorized component representation for complex nonlinear data and, simultaneously, yield a density function that can evaluate likelihoods and gene…