The Restricted Boltzmann Machines (RBM) can be used either as classifiers or as generative models. The quality of the generative RBM is measured through the average log-likelihood on test data. Due to the high computational complexity of evaluating the partition function, exact calculation of test log-likelihood is ver…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “empirical log-likelihood”
Enhances VAEs for sharper image synthesis.
problem Blurriness in generated images from VAEs.
method Integrates a downscaled version of the original image into the VAE framework and uses it as input to the decoder.
result Improves FID score in image synthesis while maintaining similar log-likelihood performance.
Maximum likelihood training improves the performance of score-based diffusion models.
problem Training score-based diffusion models with maximum likelihood.
method Trained by minimizing a weighted combination of score matching losses, with a specific weighting scheme that bounds negative log-likelihood.
result Maximum likelihood training improves the log-likelihood of score-based diffusion models across multiple datasets.
We consider Bayesian inference when only a limited number of noisy log-likelihood evaluations can be obtained. This occurs for example when complex simulator-based statistical models are fitted to data, and synthetic likelihood (SL) method is used to form the noisy log-likelihood estimates using computationally costly …
The fate of scientific hypotheses often relies on the ability of a computational model to explain the data, quantified in modern statistical approaches by the likelihood function. The log-likelihood is the key element for parameter estimation and model evaluation. However, the log-likelihood of complex models in fields…
Expectation Maximization (EM) is among the most popular algorithms for maximum likelihood estimation, but it is generally only guaranteed to find its stationary points of the log-likelihood objective. The goal of this article is to present theoretical and empirical evidence that over-parameterization can help EM avoid …
Test log-likelihood comparisons can be misleading.
problem Misinterpretation of test log-likelihood in model comparison.
method Simple examples of model comparison and forecast accuracy.
result Test log-likelihood does not always correlate with model accuracy.
This work improves bounds on Bayesian coreset quality.
problem Limitations of existing theoretical analysis of Bayesian coresets.
method Develops general upper and lower bounds on KL divergence.
result Demonstrates flexibility of new theoretical bounds in various models.
Corrects pseudo log-likelihood method issues in various applications.
problem Log-likelihood function unbounded issues in pseudo log-likelihood methods.
method Provided a counterexample and corrected algorithms in previous literature.
result Ensured well-definedness of maximum pseudo log-likelihood estimation.
The variational autoencoder (VAE; Kingma, Welling (2014)) is a recently proposed generative model pairing a top-down generative network with a bottom-up recognition network which approximates posterior inference. It typically makes strong assumptions about posterior inference, for instance that the posterior distributi…
PEP improves deep network performance and calibration by perturbing optimal parameters.
problem Improving deep network performance and calibration.
method Parameter Ensembling by Perturbation (PEP) constructs an ensemble of parameter values as random perturbations of the optimal set, maximizing log-likelihood on validation data.
result PEP provides a small to substantial improvement in calibration and log-likelihood, and in some cases, classification accuracy.
New lower bounds improve logistic log-likelihood optimization and inference.
problem Designing computationally tractable lower bounds for logistic log-likelihoods.
method Developed a piece-wise quadratic lower bound that uniformly improves tangent quadratic minorizers.
result Improves the speed of convergence and accuracy of variational Bayes approximations.
A new VIS approach improves log-likelihood estimation in latent variable models.
problem Challenges in achieving high log-likelihood with VI for complex posterior distributions.
method Uses forward divergence to optimize proposal distribution for better log-likelihood estimation.
result Consistently outperforms state-of-the-art baselines in log-likelihood and parameter estimation.
Study uses multi-kernel Hawkes models to analyze high-frequency price dynamics.
problem Understanding responsive speeds of market participants in high-frequency trading.
method Multi-kernel Hawkes models with conditional Hessian analysis for optimization.
result Existence of multi-kernels (UHF, VHF, HF) in high-frequency price dynamics.
We simplify matrix computations for block matrices, especially useful for covariance and correlation matrices.
problem Complex computations for block matrices, especially for covariance and correlation matrices.
method Obtained a canonical representation for block matrices, facilitating computation of various matrix operations.
result Simplified computation of matrix operations for block matrices, particularly useful for covariance and correlation matrices.
New method optimizes clustering with better log-likelihood landscape.
problem Nonconvex log-likelihood optimization in model-based clustering.
method Entropic optimal transport loss for Sinkhorn-EM algorithm.
result New loss function avoids spurious local optima.
A new ABC method uses variational approximations for efficient inference.
problem Computational challenges in Bayesian inference for complex models.
method Variational approximation for log-posterior, empirical likelihood for estimating expected log-likelihood, differential entropy estimation.
result Posterior consistency established for the proposed method.
New method calculates DMN log-likelihood faster.
problem Precise and fast computation of DMN log-likelihood.
method Derived a closed form expression using gamma function properties.
result Closed form calculation is faster with same accuracy.
Partition Tree estimates conditional densities for mixed continuous and categorical variables.
problem Estimating conditional densities for mixed data types.
method Tree-based framework modeling conditional distributions as piecewise-constant densities on adaptive partitions, minimizing conditional negative log-likelihood.
result Improved probabilistic prediction compared to CART-style trees and state-of-the-art methods.
Survival regression method improves log-likelihood scores.
problem Improper scoring rules in survival regression models.
method SurvivalMonotonic-net (SuMo-net) with monotonic neural networks.
result SuMo-net achieves state-of-the-art log-likelihood scores.
A new method detects changes in multivariate data using random forests.
problem Detecting changes in multivariate data.
method A computationally feasible search method using random forests and class probability predictions.
result Consistently locates change points in simulations.
This research designs a data-driven partition to test independence between continuous variables.
problem Testing independence between continuous random variables.
method Empirical log-likelihood statistic and data-driven tree-structured partition.
result Strongly consistent test of independence over probability families.
This paper introduces a method to estimate log-likelihood in VAE models.
problem Difficulty in comparing models trained via ELBO due to lack of log-likelihood.
method Introduces a general upper bound to approximate model evidence.
result Efficiently approximates model evidence and compares to other bounds.
Paper introduces fair GLMs with convex penalty for equalizing GLM outcomes.
problem Achieving fairness in GLMs for practical use.
method Two fairness criteria based on GLM outcomes/log-likelihoods, achieved via a convex penalty on linear components.
result The fair GLM estimator is efficient and can handle various response variables.
Markov random fields (MRFs) are difficult to evaluate as generative models because computing the test log-probabilities requires the intractable partition function. Annealed importance sampling (AIS) is widely used to estimate MRF partition functions, and often yields quite accurate results. However, AIS is prone to ov…
Semi-supervised learning objectives as log-likelihoods in a generative model of data curationstat.ML
We explain SSL objectives as log-likelihoods in a data curation model.
problem Lack of understanding of SSL objectives as log-likelihoods.
method Formulate SSL objectives as a log-likelihood in a generative model of data curation.
result SSL methods can be understood as lower-bounds on a principled log-likelihood.
Quantum machine learning uses quantum cross entropy to minimize loss, but measurement loss affects this process.
problem Quantum machine learning's loss minimization through cross entropy is affected by measurement outcomes.
method Defined quantum cross entropy, proved its lower bounds, and investigated its relation to quantum fidelity and likelihood.
result Quantum cross entropy is lower-bounded by negative log-likelihood when derived from quantum data, but measurement outcomes can cause loss.
This work evaluates deep generative models using RD curves, providing a more comprehensive quality assessment.
problem Quantitative evaluation of deep generative models is challenging, especially for implicit models.
method Proposes using rate distortion (RD) curves to evaluate and compare deep generative models, approximating the entire curve with similar computations to log-likelihood estimation.
result Approximating the entire RD curve provides a more comprehensive quality assessment than scalar-valued metrics.
A new method normalizes EBM training by introducing a learnable parameter.
problem Training energy-based models with maximum likelihood is challenging due to intractable normalisation constants.
method Proposes a self-normalised log-likelihood (SNL) objective that introduces a learnable parameter representing the normalisation constant.
result The SNL objective is a lower bound of the log-likelihood and can be directly optimised using stochastic gradient techniques.
This work gives an in-depth derivation of the trainable evidence lower bound obtained from the marginal joint log-Likelihood with the goal of training a Multi-Modal Variational Autoencoder (MVAE).
Bayesian Neural Networks (BNNs) place priors over the parameters in a neural network. Inference in BNNs, however, is difficult; all inference methods for BNNs are approximate. In this work, we empirically compare the quality of predictive uncertainty estimates for 10 common inference methods on both regression and clas…
EM algorithm converges to global max in latent Gaussian tree models.
problem Optimizing log-likelihood in latent Gaussian tree models.
method Analyzed the optimization landscape and convergence of EM algorithm.
result EM algorithm converges to global maximum in latent Gaussian tree models.
ButterflyFlow uses butterfly matrices for efficient invertible layers in normalizing flows.
problem Building efficient invertible layers for complex probability distributions.
method Proposes butterfly layers for invertible linear layers, leveraging their ability to capture complex structures.
result ButterflyFlow achieves strong density estimation and significantly better log-likelihoods on various datasets.
Noise-Contrastive Estimation improves efficiency for estimating log-likelihood of complex point processes.
problem Estimating log-likelihood of complex multivariate point processes is computationally expensive.
method Noise-Contrastive Estimation adapted for multivariate point processes, with provable guarantees.
result Our method achieves similar log-likelihood with fewer evaluations and less time.
In this work, a deep learning-based method for log-likelihood ratio (LLR) lossy compression and quantization is proposed, with emphasis on a single-input single-output uncorrelated fading communication setting. A deep autoencoder network is trained to compress, quantize and reconstruct the bit log-likelihood ratios cor…
Flow-based generative models (Dinh et al., 2014) are conceptually attractive due to tractability of the exact log-likelihood, tractability of exact latent-variable inference, and parallelizability of both training and synthesis. In this paper we propose Glow, a simple type of generative flow using an invertible 1x1 con…
The MAP estimate's log-likelihood sub-optimality is hard to bound in general.
problem Bounding the expected log-likelihood sub-optimality of MAP for exponential families.
method Interpreting MAP as stochastic mirror descent and analyzing convergence rates.
result Current convergence results do not apply to standard examples of exponential families.
In this article we use rate-distortion theory, a branch of information theory devoted to the problem of lossy compression, to shed light on an important problem in latent variable modeling of data: is there room to improve the model? One way to address this question is to find an upper bound on the probability (equival…
The log-likelihood loss in heteroscedastic neural networks can lead to poor parameter estimates.
problem Capturing aleatoric uncertainty in deep learning models.
method Examine the log-likelihood loss in conjunction with gradient-based optimizers and propose an alternative formulation, -NLL.
result Using an appropriate largely mitigates the issue of poor parameter estimates.
ELU algorithm improves on EM for over-specified Gaussian mixtures.
problem Slow convergence of EM in over-specified Gaussian mixtures.
method Developed ELU algorithm for two-component mixtures, combining exponential location update and gradient descent.
result ELU converges to final statistical radius after logarithmic iterations, resolving open question.
We address the problem of detecting changes in multivariate datastreams, and we investigate the intrinsic difficulty that change-detection methods have to face when the data dimension scales. In particular, we consider a general approach where changes are detected by comparing the distribution of the log-likelihood of …
Log-Likelihood Ratio Minimizing Flows: Towards Robust and Quantifiable Neural Distribution Alignmentcs.LG
New method for robust distribution alignment using log-likelihood ratio and normalizing flows.
problem Distribution alignment challenges in deep learning.
method Log-likelihood ratio statistic and normalizing flows.
result Minimizing the proposed objective yields robust domain alignment.
A new UU-test decides unimodality of datasets.
problem Deciding on the unimodality of a dataset for better data analysis.
method UU-test operates on the empirical cumulative density function (ecdf) to build a piecewise linear approximation that models the data as a Uniform Mixture Model.
result The UU-test provides a statistical model of the data in the form of a Uniform Mixture Model.
Autoregressive (AR) models have become a popular tool for unsupervised learning, achieving state-of-the-art log likelihood estimates. We investigate the use of AR models as density estimators in two settings -- as a learning signal for image translation, and as an outlier detector -- and find that these density estimat…
Beyond Mixtures and Products for Ensemble Aggregation: A Likelihood Perspective on Generalized Meansstat.ML
This work studies a unified approach to ensemble aggregation using likelihood perspective.
problem Density aggregation in machine learning, focusing on improving ensemble predictions.
method Normalized generalized mean of order r in the log-likelihood framework.
result The optimal range for r is [0,1], providing a principled justification for linear and geometric pooling.
We discuss the connection between information and copula theories by showing that a copula can be employed to decompose the information content of a multivariate distribution into marginal and dependence components, with the latter quantified by the mutual information. We define the information excess as a measure of d…
Recursive Maximum Likelihood Estimation for Interacting Particle Systems using Virtual Particlesstat.ME
Estimates log-likelihood of interacting particle systems using virtual particles.
problem Inconsistent estimation of finite-particle log-likelihood in large particle systems.
method Stochastic gradient estimate using continuous trajectory and virtual particle systems.
result Convergence to stationary points of limiting mean-field system's log-likelihood.
On Expected Accuracycs.LG
We empirically investigate the (negative) expected accuracy as an alternative loss function to cross entropy (negative log likelihood) for classification tasks. Coupled with softmax activation, it has small derivatives over most of its domain, and is therefore hard to optimize. A modified, leaky version is evaluated on…