New metrics improve learning of Gaussian networks.
problem Lack of suitable metrics for continuous Bayesian network learning.
method Introduce three-part minimum description length and renormalized normalized maximum likelihood metrics.
result Proposed metrics outperform BIC/AIC metrics in accuracy and convergence.
This paper improves normalizing flows by combining MLE and sliced-Wasserstein distance for better data fidelity.
problem Normalizing flows struggle with generating realistic data and detecting out-of-distribution data.
method Proposes a hybrid objective function combining MLE and sliced-Wasserstein distance.
result Shows better generative abilities and lower likelihood of out-of-distribution data.
Paper proposes efficient training for normalizing flows in Boltzmann generators.
problem Training normalizing flows for Boltzmann generators is computationally challenging and unstable.
method Regression Training of Normalizing Flows (RegFlow) using ℓ2-regression. result RegFlow enables efficient and stable training of normalizing flows for Boltzmann generators.
Maximum likelihood training improves the performance of score-based diffusion models.
problem Training score-based diffusion models with maximum likelihood.
method Trained by minimizing a weighted combination of score matching losses, with a specific weighting scheme that bounds negative log-likelihood.
result Maximum likelihood training improves the log-likelihood of score-based diffusion models across multiple datasets.
Operational risk models commonly employ maximum likelihood estimation (MLE) to fit loss data to heavy-tailed distributions. Yet several desirable properties of MLE (e.g. asymptotic normality) are generally valid only for large sample-sizes, a situation rarely encountered in operational risk. In this paper, we study how…
Paper proposes energy objective for training normalizing flows without determinants.
problem Challenges in training normalizing flows due to Jacobian determinants.
method Introduces energy objective based on proper scoring rules, determinant-free.
result Energy objective supports novel model families and competitive performance.
New method for efficient maximum likelihood estimation of p-generalized probit regression.
problem Efficient estimation of p-generalized probit regression models. method Combining sketching techniques with importance subsampling to obtain a coreset.
result Maximum likelihood estimator can be approximated efficiently up to a factor of (1+ε) on large data. A new path gradient estimator speeds up normalizing flows without sacrificing accuracy.
problem High computational cost and limited scalability of path gradient estimators for normalizing flows.
method Proposed a fast path gradient estimator that improves computational efficiency and scalability.
result The new estimator achieves superior performance and reduced variance across various applications.
The normalized maximized likelihood (NML) provides the minimax regret solution in universal data compression, gambling, and prediction, and it plays an essential role in the minimum description length (MDL) method of statistical modeling and estimation. Here we show that the normalized maximum likelihood has a Bayes-li…
The paper strengthens the classical result of MLE convergence to a Gaussian distribution.
problem The classical result of MLE convergence to a Gaussian distribution.
method Sub-Gaussian concentration and entropic normality of the normalized MLE.
result Entropic central limit theorem for a smoothed version of the estimator.
A method for converting NIW parameters for better estimation.
problem Estimating parameters of multivariate normal distribution.
method Convergent procedure for converting mean parameters to natural parameters in NIW family.
result Maximum likelihood estimation of natural parameters from observed statistics.
Stochastic normalizing flows use SDEs for efficient training and sampling.
problem Efficient maximum likelihood estimation and variational inference.
method Continuous normalizing flows extended with stochastic differential equations (SDEs) and rough path theory.
result Stochastic normalizing flows enable efficient training and sampling from complex distributions.
Paper proves method for calculating NML code length works for continuous models.
problem Uncertainty in calculating NML code length for continuous models.
method Introduced a novel decomposition approach based on the coarea formula to prove correctness for continuous cases.
result Method accurately calculates NML code length for continuous models.
We present a variational renormalization group (RG) approach using a deep generative model based on normalizing flows. The model performs hierarchical change-of-variables transformations from the physical space to a latent space with reduced mutual information. Conversely, the neural net directly maps independent Gauss…
A new layer, funnel, reduces dimensionality in flows for better performance.
problem Training high-dimensional models efficiently and accurately.
method Constructing dimension-reducing surjective flows using the funnel layer.
result The funnel layer improves model performance with a smaller latent space.
A new method normalizes EBM training by introducing a learnable parameter.
problem Training energy-based models with maximum likelihood is challenging due to intractable normalisation constants.
method Proposes a self-normalised log-likelihood (SNL) objective that introduces a learnable parameter representing the normalisation constant.
result The SNL objective is a lower bound of the log-likelihood and can be directly optimised using stochastic gradient techniques.
Paper proposes efficient method to calculate Fisher-Bingham distribution normalizing constant.
problem Efficiently calculating the normalizing constant of Fisher-Bingham distributions.
method Numerical integration with continuous Euler transform to Fourier-type integral representation.
result The method is fast and accurate, applicable to high-dimensional distributions.
We study asymptotic properties of maximum likelihood estimators of drift parameters for a jump-type Heston model based on continuous time observations, where the jump process can be any purely non-Gaussian Lévy process of not necessarily bounded variation with a Lévy measure concentrated on (−1,∞). We prove stro…
A new Bayesian modeling method is proposed by combining the maximization of the marginal likelihood with a momentum-space renormalization group transformation for Gaussian graphical models. Moreover, we present a scheme for computint the statistical averages of hyperparameters and mean square errors in our proposed met…
Residual flows are shown to approximate MMD well.
problem Lack of theoretical understanding of normalizing flows' expressiveness.
method Proved residual flows are universal approximators in MMD.
result Residual flows can approximate MMD with a bounded number of blocks.
We study asymptotic properties of maximum likelihood estimators for Heston models based on continuous time observations of the log-price process. We distinguish three cases: subcritical (also called ergodic), critical and supercritical. In the subcritical case, asymptotic normality is proved for all the parameters, whi…
ACNML method improves uncertainty estimation for deep networks.
problem Uncertainty estimation and calibration for deep neural networks under distribution shift.
method Approximate Bayesian inference to approximate CNML distribution.
result ACNML compares favorably to prior techniques for uncertainty estimation.
We study a normalized version of the second order renormalization group flow on closed Riemannian surfaces. We discuss some general properties of this flow and establish several basic formulas. In particular, we focus on surfaces with zero and positive Euler characteristic.
A new model estimates complex densities without explicit normalizing constants.
problem Accurately estimating the normalizing constant of high-dimensional energy functions.
method Autoregressive Energy Machine (AEM) learns an unnormalized density and an importance-sampling estimate of the normalizing constant.
result Achieves state-of-the-art performance on density-estimation tasks.
We improve maximum likelihood for location estimation in finite samples.
problem Estimating a parameter from samples with unknown or varying distribution.
method Use smoothed Fisher information for finite sample size and varying distributions.
result Recover optimal estimation theory for finite n and arbitrary f. Paper proposes CoopFlow, a two-flow generator for energy-based models.
problem Training energy-based models with Langevin flow and normalizing flow.
method CoopFlow trains an energy-based model using a normalizing flow initialization and a short-run Langevin flow revision.
result CoopFlow converges to a moment matching estimator and synthesizes realistic images.
Unified detector calibration and simulation using MLE from generative models.
problem Combining detector calibration and simulation using traditional methods.
method Maximum likelihood estimation from conditional generative models.
result Prior-independent and non-Gaussian resolutions possible.
New method uses batch normalization to improve OoD detection.
problem Out-of-distribution samples are not reliably detected by generative models.
method Proposes exploiting in-batch dependencies for OoD detection.
result Empirical results show improved robustness for high-dimensional images.
Invertible ResNets enable classification, density estimation, and generation.
problem Enforcing invertibility in ResNets without architectural changes.
method Simple normalization during training to make ResNets invertible.
result Invertible ResNets achieve competitive performance with single architecture.
We consider a stable Cox--Ingersoll--Ross process driven by a standard Wiener process and a spectrally positive strictly stable Lévy process, and we study asymptotic properties of the maximum likelihood estimator (MLE) for its growth rate based on continuous time observations. We distinguish three cases: subcritical, c…
An efficient LDP protocol for QMLE with improved practicality and theoretical guarantees.
problem Difficult implementation of existing LDP QMLE for large-scale surveys.
method Developed an alternative LDP protocol without long waiting time, high communication cost, and derivative boundedness assumptions.
result Sufficient conditions for consistency and asymptotic normality of the protocol.
Normalizing flows improve density estimation from noisy data.
problem Estimating underlying density from noisy samples.
method Use normalizing flows for density estimation with arbitrary noise distributions, using amortized variational inference.
result Normalizing flows can outperform Gaussian mixtures for density deconvolution.
New method trains any neural network as a generative model.
problem Constrained design of normalizing flows due to analytical invertibility.
method Efficient gradient estimator for non-analytically invertible networks.
result Any dimension-preserving neural network can be used as a generative model.
Paper extends LME models to allow sign constraints on coefficients with SDTN random effects.
problem Inference with sign constraints on random effects in LME models.
method Proposes SDTN distribution for random effects and develops likelihood-based approaches for estimation.
result Proposed constrained model improves real-world interpretations and achieves satisfactory performance.
We study the evolution of the renormalized volume functional for asymptotically Poincare-Einstein metrics (M,g) which are evolving by normalized Ricci flow. In particular, we prove that the time derivative of the renormalized volume along the flow is the negative integral of scal(g(t)) + n(n-1) over the manifold. This …
Optimal downsampling improves GLM performance in imbalanced classification.
problem Improving GLM performance in imbalanced classification.
method Proposed a pseudo maximum likelihood estimator for optimal downsampling.
result The introduced estimator outperforms existing alternatives in both synthetic and empirical data.
Paper proposes a simple estimator for DPP correlation kernels.
problem Estimating the correlation kernel matrix of DPPs.
method Closed-form estimator for correlation kernel, easy to implement.
result Consistency and asymptotic normality of the estimator proved.
Paper proposes an alternative to MLE for GLMs with non-canonical link functions.
problem Challenges in MLE for GLMs with non-canonical link functions.
method Variational Inequality (VI) estimation framework.
result Established finite-sample error bounds and asymptotic normality for VI estimator.
Paper analyzes NCE method for unnormalized models, reducing asymptotic variance.
problem Estimating parameters of unnormalized models with high asymptotic variance.
method Proposes a method to reduce asymptotic variance by estimating auxiliary distribution parameters and analyzing objective function forms.
result NCE estimator is consistent and asymptotically normal, with reduced variance.
We consider a jump-type Cox--Ingersoll--Ross (CIR) process driven by a standard Wiener process and a subordinator, and we study asymptotic properties of the maximum likelihood estimator (MLE) for its growth rate. We distinguish three cases: subcritical, critical and supercritical. In the subcritical case we prove weak …
Proposes LFGP for likelihood-free Gaussian process regression.
problem Inability to set likelihood functions in unknown probability models.
method Clusters and approximates likelihood using asymptotic normality.
result Reduces assumptions and computational costs for scalable problems.
New method for robust distribution alignment using log-likelihood ratio and normalizing flows.
problem Distribution alignment challenges in deep learning.
method Log-likelihood ratio statistic and normalizing flows.
result Minimizing the proposed objective yields robust domain alignment.
Using the equivalence between the renormalized Euler characteristic of Ozsvath and Szabo, and the Turaev torsion normalized by the Casson-Walker invariant, we make calculations for Sp/q3(K). An alternative proof of a theorem by Ozsváth and Szabó on L-space surgery obstructions is provided.
A novel AIRLS algorithm for multiaffine variable relations in high-dimensional problems.
problem Challenges in Maximum Likelihood Estimation in high-dimensional settings with complex variable relations.
method Proposes an Alternating and Iteratively-Reweighted Least Squares (AIRLS) algorithm for multiaffine variable relations.
result Proves convergence for problems with Generalized Normal Distributions and shows empirically super-linear convergence rate.
M-flows learn data manifolds and densities, improving manifold learning and inference.
problem Representing datasets with manifold structure more faithfully.
method Combining normalizing flows, GANs, autoencoders, and energy-based models, with a new training algorithm.
result M-flows learn data manifolds better than standard flows and provide handles for dimensionality reduction.
New method trains energy-based models faster and more stably.
problem Training efficiency and stability of energy-based models.
method EBFlow with score-matching objectives.
result EBFlow achieves significant speedup and better performance.
New method for sequential probability assignment reduces regret using contextual Shtarkov sums.
problem Minimizing regret in sequential probability assignment with arbitrary hypothesis classes.
method Introducing contextual Shtarkov sum and contextual Normalized Maximum Likelihood (cNML) algorithm.
result The contextual Shtarkov sum characterizes minimax regret and provides a minimax optimal strategy.
EBMs are flexible but hard to train; this paper explains methods.
problem Training Energy-Based Models is difficult due to the unknown normalizing constant.
method Explains MCMC, SM, and NCE for training EBMs, highlighting connections.
result Provides a friendly introduction to modern EBM training methods.