Corrects pseudo log-likelihood method issues in various applications.
problem Log-likelihood function unbounded issues in pseudo log-likelihood methods.
method Provided a counterexample and corrected algorithms in previous literature.
result Ensured well-definedness of maximum pseudo log-likelihood estimation.
Maximum likelihood training improves the performance of score-based diffusion models.
problem Training score-based diffusion models with maximum likelihood.
method Trained by minimizing a weighted combination of score matching losses, with a specific weighting scheme that bounds negative log-likelihood.
result Maximum likelihood training improves the log-likelihood of score-based diffusion models across multiple datasets.
Proposes a neural generator network for efficient energy-based model learning.
problem Challenges in maximum likelihood estimation of energy-based models.
method Uses a neural generator network to approximate the log-likelihood gradient and maximizes entropy of generated samples.
result Generates sharp images with competitive Inception and FID scores, and is robust to mode collapse.
This paper addresses the estimation of parameters of a Bayesian network from incomplete data. The task is usually tackled by running the Expectation-Maximization (EM) algorithm several times in order to obtain a high log-likelihood estimate. We argue that choosing the maximum log-likelihood estimate (as well as the max…
A new method normalizes EBM training by introducing a learnable parameter.
problem Training energy-based models with maximum likelihood is challenging due to intractable normalisation constants.
method Proposes a self-normalised log-likelihood (SNL) objective that introduces a learnable parameter representing the normalisation constant.
result The SNL objective is a lower bound of the log-likelihood and can be directly optimised using stochastic gradient techniques.
IBS efficiently estimates log-likelihood without bias.
problem Estimating log-likelihood for complex models.
method Inverse Binomial Sampling (IBS) for efficient, unbiased estimation.
result IBS produces lower error in estimated parameters and maximum log-likelihood values.
Study uses multi-kernel Hawkes models to analyze high-frequency price dynamics.
problem Understanding responsive speeds of market participants in high-frequency trading.
method Multi-kernel Hawkes models with conditional Hessian analysis for optimization.
result Existence of multi-kernels (UHF, VHF, HF) in high-frequency price dynamics.
The MAP estimate's log-likelihood sub-optimality is hard to bound in general.
problem Bounding the expected log-likelihood sub-optimality of MAP for exponential families.
method Interpreting MAP as stochastic mirror descent and analyzing convergence rates.
result Current convergence results do not apply to standard examples of exponential families.
Estimates log-likelihood of interacting particle systems using virtual particles.
problem Inconsistent estimation of finite-particle log-likelihood in large particle systems.
method Stochastic gradient estimate using continuous trajectory and virtual particle systems.
result Convergence to stationary points of limiting mean-field system's log-likelihood.
EM algorithm converges to global max in latent Gaussian tree models.
problem Optimizing log-likelihood in latent Gaussian tree models.
method Analyzed the optimization landscape and convergence of EM algorithm.
result EM algorithm converges to global maximum in latent Gaussian tree models.
TzK model learns from multiple datasets efficiently.
problem Efficiently learning from multiple heterogeneous datasets.
method Flow-based conditional generative model trained with maximum likelihood.
result Comparable log likelihood to state-of-the-art models.
Proves error bounds for PGD, extending log-Sobolev and Talagrand inequalities.
problem Maximum likelihood estimation of large latent variable models.
method Extending log-Sobolev and Talagrand inequalities to models with strongly concave log-likelihoods.
result Non-asymptotic error bounds for PGD in models satisfying LSI and PŁI.
Classifiers based on probabilistic graphical models are very effective. In continuous domains, maximum likelihood is usually used to assess the predictions of those classifiers. When data is scarce, this can easily lead to overfitting. In any probabilistic setting, Bayesian averaging (BA) provides theoretically optimal…
Maximum likelihood (ML) estimation using Newton's method in nonlinear state space models (SSMs) is a challenging problem due to the analytical intractability of the log-likelihood and its gradient and Hessian. We estimate the gradient and Hessian using Fisher's identity in combination with a smoothing algorithm. We exp…
Sparse logistic regression recovers any discrete pairwise graph model.
problem Recovering the Markov graph of discrete pairwise graphical models.
method Maximum conditional log-likelihood with convex optimization.
result The algorithm can recover any arbitrary discrete pairwise graphical model.
A new KDE model prevents singular solutions and accelerates optimization for probabilistic modeling.
problem Adapting to varying densities in data regions for probabilistic modeling.
method Adaptive KDE model with individual bandwidths, LOO-MLL criterion, and modified EM algorithm.
result The proposed models prevent singular solutions and have promising performance.
Noise-Contrastive Estimation improves efficiency for estimating log-likelihood of complex point processes.
problem Estimating log-likelihood of complex multivariate point processes is computationally expensive.
method Noise-Contrastive Estimation adapted for multivariate point processes, with provable guarantees.
result Our method achieves similar log-likelihood with fewer evaluations and less time.
The paper analyzes a recursive ML estimation method for non-linear state-space models.
problem Estimating maxima of the log-likelihood function in non-linear state-space models.
method Recursive maximum likelihood estimation using particle approximation to the optimal filter derivative.
result The algorithm accurately estimates maxima of the log-likelihood when the number of particles is sufficiently large.
New method for robust distribution alignment using log-likelihood ratio and normalizing flows.
problem Distribution alignment challenges in deep learning.
method Log-likelihood ratio statistic and normalizing flows.
result Minimizing the proposed objective yields robust domain alignment.
Researchers use Gaussian processes to approximate Lagrange multipliers for Maximum-Entropy distributions.
problem Finding Lagrange multipliers for Maximum-Entropy distributions is computationally challenging.
method Employed Gaussian processes to approximate the Lagrange multipliers as a map of moments. Optimized hyperparameters by maximizing log-likelihood.
result Data-driven Maximum-Entropy closure performs well in approximating non-equilibrium distributions.
Innovative game theory approach optimizes survival analysis metrics.
problem Survival analysis models trained with maximum likelihood do not directly optimize criteria like Brier score or Bernoulli log likelihood.
method Inverse-Weighted Survival Games: Construct objectives from re-weighted estimates featuring the other model, holding the latter fixed during training.
result Games optimize Brier score on simulations and real-world data.
Improved convergence rates for MLE in mixture models using penalized log-likelihood.
problem Convergence rates for MLE in finite mixture models.
method Penalizing log-likelihood to discourage vanishing mixing weights, using Wasserstein distance and new loss functions.
result Improved convergence rates for some mixture components, faster than traditional methods.
We propose a novel method for maximum likelihood-based parameter inference in nonlinear and/or non-Gaussian state space models. The method is an iterative procedure with three steps. At each iteration a particle filter is used to estimate the value of the log-likelihood function at the current parameter iterate. Using …
Adversarial learning of probabilistic models has recently emerged as a promising alternative to maximum likelihood. Implicit models such as generative adversarial networks (GAN) often generate better samples compared to explicit models trained by maximum likelihood. Yet, GANs sidestep the characterization of an explici…
A boosting method improves nonparametric density estimation without smoothing assumptions.
problem Overfitting in nonparametric data fitting.
method Introduces a boosting algorithm for univariate nonparametric maximum likelihood estimation.
result Demonstrates the effectiveness of the boosting approach through simulations and real data experiments.
Quantum machine learning uses quantum cross entropy to minimize loss, but measurement loss affects this process.
problem Quantum machine learning's loss minimization through cross entropy is affected by measurement outcomes.
method Defined quantum cross entropy, proved its lower bounds, and investigated its relation to quantum fidelity and likelihood.
result Quantum cross entropy is lower-bounded by negative log-likelihood when derived from quantum data, but measurement outcomes can cause loss.
Improved protein identification in mass spectrometry data.
problem Expanding peptide scoring capabilities in tandem mass spectrometry.
method Deriving concave emission distributions for dynamic Bayesian networks.
result Efficiently learned scoring function outperforms state-of-the-art.
DNLL loss improves deep LDA accuracy and consistency.
problem Pathological solutions in unconstrained Deep LDA.
method Introducing Discriminative Negative Log-Likelihood (DNLL) loss.
result Deep LDA trained with DNLL produces clean latent spaces and better calibrated probabilities.
An efficient LDP protocol for QMLE with improved practicality and theoretical guarantees.
problem Difficult implementation of existing LDP QMLE for large-scale surveys.
method Developed an alternative LDP protocol without long waiting time, high communication cost, and derivative boundedness assumptions.
result Sufficient conditions for consistency and asymptotic normality of the protocol.
In this paper we present nonparametric estimators for coefficients in stochastic differential equation if the data are described by independent, identically distributed random variables. The problem is formulated as a nonlinear ill-posed operator equation with a deterministic forward operator described by the Fokker-Pl…
Unbiased gradient estimation improves VAE performance.
problem Training VAEs via maximum likelihood is difficult due to intractable integrals.
method Introduced unbiased estimators of the log-likelihood gradient using coupled Markov chains.
result Unbiased estimators lead to better predictive performance in VAEs.
Recently, there has been focus on penalized log-likelihood covariance estimation for sparse inverse covariance (precision) matrices. The penalty is responsible for inducing sparsity, and a very common choice is the convex l1 norm. However, the best estimator performance is not always achieved with this penalty. The …
New method tracks time-varying parameters in data.
problem Tracking unknown time-varying parameters in data.
method Stochastic gradient descent-based recursive scheme with log-likelihood as gain function.
result Convergence in mean-square error in a suitable neighborhood of the unknown parameter.
TraDE uses self-attention for better density estimation of tabular and image data.
problem Improving density estimation for tabular and image data.
method Self-attention-based architecture trained with a penalized maximum likelihood objective.
result TraDE produces significantly better density estimates than existing methods.
Deterministic training improves generative autoencoder performance.
problem Stochastic training limits generative autoencoder performance.
method Invertible layers for deterministic training.
result AEFs outperform VAEs in log-likelihood and sample quality.
New lower bounds improve logistic log-likelihood optimization and inference.
problem Designing computationally tractable lower bounds for logistic log-likelihoods.
method Developed a piece-wise quadratic lower bound that uniformly improves tangent quadratic minorizers.
result Improves the speed of convergence and accuracy of variational Bayes approximations.
Geodesic descent optimizes likelihood in dually flat spaces.
problem Maximum likelihood estimation in exponential families.
method m-geodesic and e-geodesic updates on dually flat spaces.
result Geodesic updates can reach maximum likelihood estimator in one step.
Sparse matrices simplify computation of GP variances and likelihoods.
problem Efficient computation of posterior variance and log-likelihood for additive Matérn GPs.
method Represented posterior mean, variance, log-likelihood, and gradient using sparse matrices.
result Efficient computation of posterior mean, variance, log-likelihood, and gradient in O(nlogn) time. Efficiently learns perturb-and-map models using weighted log-likelihood.
problem Structured output prediction with weighted Hamming losses.
method Generalizes perturb-and-MAP framework, uses dynamic graph cuts for MAP inference, and double stochastic gradient descent for efficient learning.
result Shows efficiency in learning log-supermodular models with weak supervision.
Study on likelihood functions, associative equations, and Frobenius manifolds.
problem Maximum likelihood estimation and associativity equations in statistical models.
method Analyzes the cone of concentration matrices, log-likelihood function, and Frobenius manifolds.
result Maximum likelihood degree is indexed by components of Frobenius residuals.
Partition Tree estimates conditional densities for mixed continuous and categorical variables.
problem Estimating conditional densities for mixed data types.
method Tree-based framework modeling conditional distributions as piecewise-constant densities on adaptive partitions, minimizing conditional negative log-likelihood.
result Improved probabilistic prediction compared to CART-style trees and state-of-the-art methods.
We show that univariate and symmetric multivariate Hawkes processes are only weakly causal: the true log-likelihoods of real and reversed event time vectors are almost equal, thus parameter estimation via maximum likelihood only weakly depends on the direction of the arrow of time. In ideal (synthetic) conditions, test…
Paper proposes efficient and accurate initialization and EM algorithm for PL mixture models.
problem Initialization issues and combinatorial complexity in PL likelihood maximization.
method Initialization algorithm and EM algorithm for true log-likelihood maximization.
result Proposed algorithm provides accurate initial estimates and efficiently maximizes true log-likelihood.
Over-parameterization helps EM avoid local optima in Gaussian mixture models.
problem Finding global optima in non-convex optimization problems with EM.
method Using an over-parameterized model with unknown mixing weights.
result Over-parameterization improves EM's ability to find the global maximizer of the log-likelihood.
The study examines how side information quality and quantity affect community recovery in graphs.
problem Recovering a hidden community of size K=o(n) in a graph of size n. method Maximum likelihood detection and belief propagation are used to calculate necessary and sufficient conditions for exact and weak recovery. A local voting procedure is also designed and analyzed.
result Tight necessary and sufficient conditions for exact and weak recovery are derived, showing how side information needs to evolve with n to improve recovery thresholds. We consider log-supermodular models on binary variables, which are probabilistic models with negative log-densities which are submodular. These models provide probabilistic interpretations of common combinatorial optimization tasks such as image segmentation. In this paper, we focus primarily on parameter estimation in…
A new policy improvement method using CEM for Actor-Critic.
problem Improving policy efficiency and robustness in reinforcement learning.
method Greedy Actor-Critic (Greedy AC) using Conditional Cross-Entropy Method (CCEM).
result Greedy AC outperforms Soft Actor-Critic and is less sensitive to entropy regularization.
VAEs improve representation learning by inverting the data-generating process through self-consistency.
problem VAEs struggle to invert the data-generating process, yet often succeed in representation learning.
method Studied VAEs in the limit of near-deterministic decoders, proving self-consistency and showing ELBO convergence to a regularized log-likelihood.
result VAEs can perform independent mechanism analysis (IMA), recovering true latent factors under specific conditions.