This work studies the exact likelihood of DLVMs and its applications in inference.
problem The lack of attention to the exact likelihood of DLVMs and its implications for inference.
method Investigation of the properties of the exact likelihood, maximum likelihood estimation, and missing data imputation.
result The exact likelihood can be leveraged to ensure the existence of maximum likelihood estimates and improve missing data imputation.
PixelCNN models can achieve state-of-the-art results on CIFAR-10 with exact likelihood computation.
problem Dequantization gap in modeling discrete data like images.
method Introducing subset flows to allow exact computation of likelihoods for discrete data.
result PixelCNN models trained with exact likelihood computation achieve state-of-the-art results on CIFAR-10.
Unsupervised learning of probabilistic models is a central yet challenging problem in machine learning. Specifically, designing models with tractable learning, sampling, inference and evaluation is crucial in solving this task. We extend the space of such models using real-valued non-volume preserving (real NVP) transf…
Exact recovery method for community detection in Gaussian mixtures with dependent noise.
problem Community detection in Gaussian mixtures with dependent and heterogeneous noise.
method Maximum likelihood estimator (MLE) for constrained quadratic optimization problem, using Σ-whitened separation and local inequalities. result Sharp exact-recovery threshold and no-gap mechanism in the unknown-size setting.
Study exact community detection in k-community Gaussian mixtures with different intensities.
problem Community detection in k-community Gaussian mixtures with varying intensities.
method Explicitly find the threshold for exact recovery of maximum likelihood estimation.
result Threshold for exact recovery of maximum likelihood estimation is identified.
Glow uses invertible 1x1 convolutions to improve image generation and manipulation.
problem Efficient and realistic image generation and manipulation.
method Invertible 1x1 convolutions in generative flows.
result Significant improvement in log-likelihood and realistic image synthesis.
Neural networks estimate spatial process likelihoods efficiently.
problem Challenges in estimating spatial processes with slow or intractable likelihoods.
method Convolutional neural networks trained on a classification task to learn likelihood function.
result Neural likelihood surfaces provide fast and accurate parameter estimation.
Paper explores limits of exact inference in structured prediction models.
problem Exact recovery of true labels in graph-based structured prediction models.
method Analyzes necessary and sufficient conditions for exact recovery using maximum likelihood estimation.
result Derives tight conditions for exact recovery, revealing a gap with computationally tractable methods.
Recently Carr and Wu (2004, 2005) and also Huang and Wu (2004) show that most stochastic processes used in traditional option pricing models can be cast as special cases of time-changed Lévy processes. In particular these are models which can be tailored to exhibit correlated jumps in both the log price of assets and t…
Exact selective inference with randomization for Gaussian regression models.
problem Exact selective inference in Gaussian regression models.
method Introduces a pivot for exact selective inference with randomization, reducing the problem to a bivariate truncated Gaussian distribution.
result Our pivot leads to exact inference and produces narrower confidence intervals than related methods.
Exact learning improves naive Bayes classifier performance for small samples.
problem Improving naive Bayes classifier performance with small sample sizes.
method Proposes an exact learning augmented naive Bayes classifier (ANB) that ensures a class variable with no parents.
result The proposed ANB method outperforms other methods in comparison experiments.
Exact hierarchical clustering algorithms for data analysis.
problem Finding meaningful structures in data like phylogenetic trees.
method Novel dynamic-programming algorithms based on a trellis data structure.
result Exact computation of partition function, maximum likelihood hierarchy, and marginal probabilities.
This work improves neural likelihood surrogates for stochastic models with a score-augmented loss.
problem Efficient parameter inference for stochastic models with computationally expensive likelihood functions.
method Score-augmented loss function for neural network likelihood surrogates.
result Improves surrogate quality at a lower computational cost compared to generating more data.
We present a Bayesian model selection approach to estimate the intrinsic dimensionality of a high-dimensional dataset. To this end, we introduce a novel formulation of the probabilisitic principal component analysis model based on a normal-gamma prior distribution. In this context, we exhibit a closed-form expression o…
Markov chain Monte Carlo (MCMC) is a popular and successful general-purpose tool for Bayesian inference. However, MCMC cannot be practically applied to large data sets because of the prohibitive cost of evaluating every likelihood term at every iteration. Here we present Firefly Monte Carlo (FlyMC) an auxiliary variabl…
Models for which the likelihood function can be evaluated only up to a parameter-dependent unknown normalising constant, such as Markov random field models, are used widely in computer science, statistical physics, spatial statistics, and network analysis. However, Bayesian analysis of these models using standard Monte…
Adversarial learning of probabilistic models has recently emerged as a promising alternative to maximum likelihood. Implicit models such as generative adversarial networks (GAN) often generate better samples compared to explicit models trained by maximum likelihood. Yet, GANs sidestep the characterization of an explici…
Speeding up Markov Chain Monte Carlo (MCMC) for datasets with many observations by data subsampling has recently received considerable attention. A pseudo-marginal MCMC method is proposed that estimates the likelihood by data subsampling using a block-Poisson estimator. The estimator is a product of Poisson estimators,…
Paper uses SDP for community detection with side information.
problem Community detection in graphs with additional non-graph data.
method Formulates SDP relaxation for maximum likelihood node labeling with side information.
result SDP achieves same exact recovery threshold as maximum likelihood with side information.
A novel likelihood function for MRFs approximates marginal likelihoods and uses copulas to reconstruct the joint likelihood.
problem Intractable partition function for MRF likelihoods.
method Approximate marginal likelihoods through a modified coin-tossing scenario, then reconstruct the joint likelihood using copulas.
result Our approach outperforms Laplace approximation and pseudolikelihood, especially as MRF size increases.
The paper improves risk bounds for maximum likelihood estimation with arbitrary penalties.
problem Improving risk bounds for maximum likelihood estimation with arbitrary penalties.
method Developed a more general inequality for arbitrary penalties, leading to exact risk bounds of order 1/n.
result Derived exact risk bounds of order 1/n for iid parametric models, improving on previous bounds.
Proposes efficient Gaussian approximations for non-Gaussian likelihoods.
problem Computational challenges in learning and inference with non-Gaussian likelihoods.
method Variational inference and moment matching in transformed bases.
result Good approximation quality for binary and multiclass classification.
Efficient likelihood computation improves kernel learning accuracy for complex models.
problem Improving accuracy of kernel learning for complex models and sparse signals.
method Exact likelihood computation using Kalman filter and diagonalized state transition equation.
result Posterior mean with reference prior is more accurate for complex models and sparse sampling.
We consider training probabilistic classifiers in the case of a large number of classes. The number of classes is assumed too large to perform exact normalisation over all classes. To account for this we consider a simple approach that directly approximates the likelihood. We show that this simple approach works well o…
The Restricted Boltzmann Machines (RBM) can be used either as classifiers or as generative models. The quality of the generative RBM is measured through the average log-likelihood on test data. Due to the high computational complexity of evaluating the partition function, exact calculation of test log-likelihood is ver…
A new method for clustering heterogeneous data using likelihood-adjusted SDP.
problem Clustering heterogeneous data with different cluster shapes and sizes.
method Iterative likelihood-adjusted semidefinite programming (iLA-SDP) method.
result iLA-SDP achieves lower mis-clustering errors compared to other methods.
Set Flow models sets of data, learns dependencies, and achieves state-of-the-art likelihoods.
problem Modeling and sampling from finite, potentially high-dimensional, non-i.i.d. sets of data.
method Extends RealNVPs to handle finite sets, maintaining invertibility and exact log-likelihood evaluation.
result Achieves state-of-the-art likelihoods on 3D point clouds.
A new layer, funnel, reduces dimensionality in flows for better performance.
problem Training high-dimensional models efficiently and accurately.
method Constructing dimension-reducing surjective flows using the funnel layer.
result The funnel layer improves model performance with a smaller latent space.
Determinantal point processes (DPPs) are point process models that naturally encode diversity between the points of a given realization, through a positive definite kernel K. DPPs possess desirable properties, such as exact sampling or analyticity of the moments, but learning the parameters of kernel K through like…
New method uses diffusion models for unsupervised combinatorial optimization.
problem Learning to sample from intractable discrete distributions without training data.
method Lifts the restriction of generative models needing exact sample likelihoods using a loss that bounds reverse KL divergence.
result Achieves new state-of-the-art results in data-free Combinatorial Optimization.
New particle filter estimates model evidence without bias.
problem Unbiased estimation of marginal likelihood for model comparison.
method Particle filter with rejection control.
result Unbiased estimation of marginal likelihood.
Model counts interactions in dynamic networks using Poisson processes and clusters.
problem Counting interactions in dynamic networks with unknown cluster structure.
method Developed a model using non-homogeneous Poisson processes and block modeling. Truncated to discrete time for tractability. Used an exact integrated classification likelihood criterion for estimation.
result Estimates cluster memberships and number of clusters simultaneously.
Bayesian approach for inhomogeneous Poisson process intensity estimation.
problem Intractable integral in likelihood of Gaussian Cox process.
method Joint modeling of intensity and cumulative intensity as transformed Gaussian process; exact MCMC sampler.
result Exact posterior inference without approximations.
We have observed an interesting, yet unexplained, phenomenon: Semidefinite programming (SDP) based relaxations of maximum likelihood estimators (MLE) tend to be tight in recovery problems with noisy data, even when MLE cannot exactly recover the ground truth. Several results establish tightness of SDP based relaxations…
In many fields of science, generalized likelihood ratio tests are established tools for statistical inference. At the same time, it has become increasingly common that a simulator (or generative model) is used to describe complex processes that tie parameters θ of an underlying theory and measurement apparatus to hig…
FlowGMM uses normalizing flows for semi-supervised learning, showing promising results across various data types.
problem Semi-supervised learning with limited labeled data.
method Normalizing flows combined with latent Gaussian mixture models for generative modeling.
result FlowGMM achieves promising results on multiple data types, including text and tabular data.
We study statistical inference and distributionally robust solution methods for stochastic optimization problems, focusing on confidence intervals for optimal values and solutions that achieve exact coverage asymptotically. We develop a generalized empirical likelihood framework---based on distributional uncertainty se…
The study examines how side information quality and quantity affect community recovery in graphs.
problem Recovering a hidden community of size K=o(n) in a graph of size n. method Maximum likelihood detection and belief propagation are used to calculate necessary and sufficient conditions for exact and weak recovery. A local voting procedure is also designed and analyzed.
result Tight necessary and sufficient conditions for exact and weak recovery are derived, showing how side information needs to evolve with n to improve recovery thresholds. This work improves likelihood of score-based diffusion ODEs using high-order denoising score matching.
problem The gap between maximum likelihood and score matching objectives for score-based diffusion ODEs.
method High-order denoising score matching to maximize likelihood.
result Score-based diffusion ODEs achieve better likelihood on synthetic and CIFAR-10 data.
New variational bounds improve posterior covariances and likelihoods.
problem Improving variational inference with different divergence measures.
method Applying variational perturbation theory to construct new variational bounds.
result New variational bounds lead to more accurate posterior covariances and higher likelihoods.
A new parallel algorithm speeds up Hawkes process estimation.
problem Slow maximum likelihood estimation for Hawkes processes.
method Parallel prefix scan for sparse transition matrices.
result Massive speedup with O(N/P) complexity. A scalable method for estimating spatial data using VREML.
problem Costly computation of REML for large, sparse precision matrices in spatial data.
method Proposes VREML framework approximating marginal likelihood with Gaussian variational distribution and deriving a coordinate-ascent algorithm.
result Empirically shows VREML outperforms MLE and INLA.
New SMC methods improve likelihood estimates for doubly intractable models.
problem Bayesian inference for models with intractable partition functions.
method Marginal sequential Monte Carlo with adaptive likelihood estimates.
result Improved likelihood estimates lead to more accurate inference.
Framework for Bayesian inference using GP emulated MH sampler for noisy likelihoods.
problem Approximate Bayesian inference with limited noisy log-likelihood evaluations.
method Gaussian process emulates MH sampler for log-likelihood evaluations; sequential experimental design selects evaluation points.
result Approximate sampler is sample-efficient and robust to GP assumptions.
PVI seeks a posterior that makes predictions closer to true data, not approximating the Bayesian posterior.
problem Finding meaningful posterior distributions under model misspecification.
method Predictive variational inference (PVI) seeks an optimal posterior density for close predictive matching to true data.
result PVI learns a posterior that is not the same as the Bayesian posterior, but is closer to the true data generating process.
Paper extends LME models to allow sign constraints on coefficients with SDTN random effects.
problem Inference with sign constraints on random effects in LME models.
method Proposes SDTN distribution for random effects and develops likelihood-based approaches for estimation.
result Proposed constrained model improves real-world interpretations and achieves satisfactory performance.
Equivariant flows generate symmetric distributions for complex systems.
problem Generating symmetric distributions for complex systems with exact likelihood.
method Equivariant normalizing flows that preserve symmetries.
result Equivariant flows generate symmetric distributions that are invariant to symmetries in physical systems.
DALTON improves ODE parameter estimation by learning from noisy data.
problem High sensitivity to parameters in ODEs produces unreliable parameter estimates.
method Data-adaptive probabilistic likelihood approximation for ODEs.
result DALTON produces more accurate parameter estimates than existing methods.