Bayesian networks with latent variables are characterized and their likelihoods compared.
problem Characterizing and comparing likelihoods of Bayesian networks with latent variables.
method Characterized likelihood function and empirical Bayesian network. Proved dominance of global maximum likelihood from empirical model.
result The global maximum likelihood of the original Bayesian network is attained if and only if parameters are consistent with empirical model.
Invertibility conditions for observation-driven time series models often fail to be guaranteed in empirical applications. As a result, the asymptotic theory of maximum likelihood and quasi-maximum likelihood estimators may be compromised. We derive considerably weaker conditions that can be used in practice to ensure t…
Maximum likelihood training improves the performance of score-based diffusion models.
problem Training score-based diffusion models with maximum likelihood.
method Trained by minimizing a weighted combination of score matching losses, with a specific weighting scheme that bounds negative log-likelihood.
result Maximum likelihood training improves the log-likelihood of score-based diffusion models across multiple datasets.
This work explores maximum likelihood optimization of neural networks through hypernetworks. A hypernetwork initializes the weights of another network, which in turn can be employed for typical functional tasks such as regression and classification. We optimize hypernetworks to directly maximize the conditional likelih…
Paper quantifies label shift robustly.
problem Quantifying label shift in datasets.
method Robust estimators of label distribution.
result Maximum Likelihood Estimator is a robust estimator.
We propose an efficient algorithm for approximate computation of the profile maximum likelihood (PML), a variant of maximum likelihood maximizing the probability of observing a sufficient statistic rather than the empirical sample. The PML has appealing theoretical properties, but is difficult to compute exactly. Inspi…
Proposes a guaranteed regularization method for maximum likelihood estimation using gauge symmetry in Kullback-Leibler divergence.
problem Overfitting in maximum likelihood estimation.
method Introduces a regularization approach based on gauge symmetry in Kullback-Leibler divergence.
result The method provides a theoretically guaranteed optimal model without frequent hyperparameter tuning.
We propose a robust estimator to improve maximum likelihood in probabilistic models.
problem Overfitting and sensitivity to noise in maximum likelihood estimation.
method Distributionally robust maximum likelihood estimator that minimizes worst-case expected log-loss.
result The robust estimator is statistically consistent and performs well in regression and classification tasks.
New estimators outperform maximum likelihood without hyper-parameter estimation.
problem Improving system identification performance without hyper-parameter estimation.
method Developed generalized Bayes and closed-form biased estimators using excess MSE.
result New estimators have comparable performance to empirical-Bayes-based regularized estimator.
Proposes a new approach to approximate maximum likelihood for complex models.
problem Intractable likelihood functions in complex parametric models.
method Simulation-based constrained approximation to the structural model.
result Estimators nearly as efficient as maximum likelihood, feasible in many cases.
New algorithms improve linear bandit performance with low computation.
problem Optimizing reward in linear stochastic bandits.
method Reward-biased maximum likelihood method modified for linear and generalized linear bandits.
result New policies achieve order-optimality and competitive empirical performance.
This work improves likelihood of score-based diffusion ODEs using high-order denoising score matching.
problem The gap between maximum likelihood and score matching objectives for score-based diffusion ODEs.
method High-order denoising score matching to maximize likelihood.
result Score-based diffusion ODEs achieve better likelihood on synthetic and CIFAR-10 data.
A new method optimizes neural sequence models for better task performance.
problem Training neural sequence models with maximum likelihood estimation ignores task losses.
method Maximum likelihood guided parameter search (MGS) in the parameter space.
result MGS optimizes sequence-level losses, reducing repetition and non-termination.
Advocates for MLE in regression and forecasting for better inductive biases and post-hoc optimization.
problem Designing effective loss functions for regression and forecasting.
method Maximum Likelihood Estimation (MLE) approach for regression and forecasting.
result MLE approach outperforms direct empirical risk minimization under certain conditions and for various datasets.
A scalable method for estimating spatial data using VREML.
problem Costly computation of REML for large, sparse precision matrices in spatial data.
method Proposes VREML framework approximating marginal likelihood with Gaussian variational distribution and deriving a coordinate-ascent algorithm.
result Empirically shows VREML outperforms MLE and INLA.
The coefficient of determination, known as R2, is commonly used as a goodness-of-fit criterion for fitting linear models. R2 is somewhat controversial when fitting nonlinear models, although it may be generalised on a case-by-case basis to deal with specific models such as the logistic model. Assume we are fittin…
Paper proposes efficient training for normalizing flows in Boltzmann generators.
problem Training normalizing flows for Boltzmann generators is computationally challenging and unstable.
method Regression Training of Normalizing Flows (RegFlow) using ℓ2-regression. result RegFlow enables efficient and stable training of normalizing flows for Boltzmann generators.
We study the Nonparametric Maximum Likelihood Estimator (NPMLE) for estimating Gaussian location mixture densities in d-dimensions from independent observations. Unlike usual likelihood-based methods for fitting mixtures, NPMLEs are based on convex optimization. We prove finite sample results on the Hellinger accurac…
Quantum ML predicts data with improved speed and accuracy.
problem Predicting data using maximum likelihood in a quantum setting.
method Quantum states embedding and minimization of quantum relative entropy.
result Unified framework for classical and quantum LLMs with performance guarantees.
We investigate the problem of semi-parametric maximum likelihood under constraints on summary statistics. Such a procedure results in a discrete probability distribution that maximises the likelihood among all such distributions under the specified constraints (called estimating equations), and is an approximation to t…
Alternative hypothesis tests for class-conditional noise using local maximum likelihood.
problem Assessing label noise in supervised learning datasets.
method Proposes hypothesis tests based on local maximum likelihood estimation for nonparametric logistic regression.
result Shows improved applicability and flexibility of the proposed tests compared to parametric approaches.
A new method of moments estimator goes beyond data reweighting.
problem Estimation of moment restrictions and conditional moment restrictions.
method Kernel Method of Moments (KMM) based on maximum mean discrepancy.
result KMM achieves competitive performance on conditional moment restriction tasks.
The study simplifies assessing overlap in logistic regression models using empirical likelihood.
problem Assessing overlap in multidimensional logistic regression models.
method Translation of Silvapulle's condition to empirical likelihood maximization, mechanized with R code.
result Minimal overlapping structures are cataloged in dimensions less than four, providing rules for higher dimensions.
Maximum likelihood estimation fails to be well-posed in Gaussian process regression.
problem Establishing well-posedness of maximum likelihood estimation in Gaussian process regression.
method Analyzing the conditions under which maximum likelihood estimation is not Lipschitz in the data with respect to the Hellinger distance.
result Maximum likelihood estimation is not well-posed in the noiseless data setting for any Gaussian process with a stationary covariance function whose lengthscale parameter is estimated using maximum likelihood.
Nonparametric neural-network estimation of current-status data
problem Estimation of conditional cumulative distribution function with current-status data
method Neural-network sieve maximum likelihood estimator
result Explicit convergence rate for Hölder smoothness
Despite the ubiquity of the Gaussian process regression model, few theoretical results are available that account for the fact that parameters of the covariance kernel typically need to be estimated from the dataset. This article provides one of the first theoretical analyses in the context of Gaussian process regressi…
A new path gradient estimator speeds up normalizing flows without sacrificing accuracy.
problem High computational cost and limited scalability of path gradient estimators for normalizing flows.
method Proposed a fast path gradient estimator that improves computational efficiency and scalability.
result The new estimator achieves superior performance and reduced variance across various applications.
Adversarial learning of probabilistic models has recently emerged as a promising alternative to maximum likelihood. Implicit models such as generative adversarial networks (GAN) often generate better samples compared to explicit models trained by maximum likelihood. Yet, GANs sidestep the characterization of an explici…
New efficient algorithm for approximate PML distribution.
problem Computing the profile maximum likelihood (PML) distribution efficiently.
method Exploiting sparsity structure and new matrix rounding algorithm.
result First provable computationally efficient implementation of PseudoPML.
Modeling preference rankings with salient features to explain irrational choices.
problem Estimating rankings from noisy pairwise comparisons with irrational choices.
method Salient feature preference model with maximum likelihood estimation.
result Strong performance of maximum likelihood estimation on synthetic and real data.
Residual flows are shown to approximate MMD well.
problem Lack of theoretical understanding of normalizing flows' expressiveness.
method Proved residual flows are universal approximators in MMD.
result Residual flows can approximate MMD with a bounded number of blocks.
Paper shows robust estimators converge to true risk minimizers at optimal rates.
problem Understanding asymptotic properties of robust risk minimizers.
method Investigates robust analogues of empirical risk minimization, focusing on median of means estimator.
result Robust minimizers converge to true minimizers at optimal rates and have similar asymptotic variance.
Quantum machine learning uses quantum cross entropy to minimize loss, but measurement loss affects this process.
problem Quantum machine learning's loss minimization through cross entropy is affected by measurement outcomes.
method Defined quantum cross entropy, proved its lower bounds, and investigated its relation to quantum fidelity and likelihood.
result Quantum cross entropy is lower-bounded by negative log-likelihood when derived from quantum data, but measurement outcomes can cause loss.
Algorithm identifies and corrects noisy labels using Gaussian process regression.
problem Detecting and correcting real-valued noisy labels from mixed data.
method Gaussian process regression with heteroscedastic noise model and leave-one-out cross-validation.
result The method can pinpoint corrupted sample points and improve regression models.
This study examine the theoretical and empirical perspectives of the symmetric Hawkes model of the price tick structure. Combined with the maximum likelihood estimation, the model provides a proper method of volatility estimation specialized in ultra-high-frequency analysis. Empirical studies based on the model using t…
Corrects pseudo log-likelihood method issues in various applications.
problem Log-likelihood function unbounded issues in pseudo log-likelihood methods.
method Provided a counterexample and corrected algorithms in previous literature.
result Ensured well-definedness of maximum pseudo log-likelihood estimation.
A new method for efficient inference and model selection in SBMs using OT.
problem Efficient inference and model selection in stochastic block models.
method Interpreting MLVI as srGW with entropic regularization, then unregularizing for sparse solutions, and adding a sparsity-promoting regularizer.
result The method consistently recovers SBM parameters and selects the number of clusters in finite samples.
Deep models memorize training data in geophysical inversion, leading to biased posterior distributions.
problem Memorization of training data biases learned priors in geophysical inverse problems.
method Casting generative models' training as maximum likelihood, we show memorization results in a reweighted empirical distribution for diffusion models, leading to Gaussian mixture priors and posteriors.
result Memorization leads to posterior distributions that are likelihood-weighted lookup among stored training examples, affecting full waveform inversion outcomes.
New method learns latent energy models using particle algorithms.
problem Learning latent variable models with energy priors.
method Continuous-time SDEs for MMLE, particle-based discretization.
result Practical algorithm converges to solve MMLE problem.
We present the particle stochastic approximation EM (PSAEM) algorithm for learning of dynamical systems. The method builds on the EM algorithm, an iterative procedure for maximum likelihood inference in latent variable models. By combining stochastic approximation EM and particle Gibbs with ancestor sampling (PGAS), PS…
Paper proposes a fair grading method for randomized exams.
problem Ensuring fairness in grading for randomized exams.
method Maximum-likelihood estimator for Bradley-Terry-Luce model on student-question graph.
result Maximum-likelihood estimator is consistent and outperforms simple averaging in fairness and accuracy.
This paper studies the partial estimation of Gaussian graphical models from high-dimensional empirical observations. We derive a convex formulation for this problem using ℓ1-regularized maximum-likelihood estimation, which can be solved via a block coordinate descent algorithm. Statistical estimation performance …
A fundamental problem arising in many areas of machine learning is the evaluation of the likelihood of a given observation under different nominal distributions. Frequently, these nominal distributions are themselves estimated from data, which makes them susceptible to estimation errors. We thus propose to replace each…
In this paper, we empirically study models for pricing Italian sovereign bonds under a reduced form framework, by assuming different dynamics for the short-rate process. We analyze classical Cox-Ingersoll-Ross and Vasicek multi-factor models, with a focus on optimization algorithms applied in the calibration exercise. …
Proposes adjusting neural network errors for time series forecasting.
problem Autocorrelated errors in neural networks for time series.
method Jointly learn autocorrelation coefficient with model parameters.
result Enhances performance in almost all time series forecasting cases.
Paper offers a framework for estimating symmetric properties efficiently.
problem Estimating symmetric properties of distributions from samples.
method General framework using profile maximum likelihood (PML) distribution.
result Optimal sample complexity for many properties, practical algorithms.
Reward augmented maximum likelihood (RAML), a simple and effective learning framework to directly optimize towards the reward function in structured prediction tasks, has led to a number of impressive empirical successes. RAML incorporates task-specific reward by performing maximum-likelihood updates on candidate outpu…
Graphical lasso may fail to fit models when data points are insufficient.
problem When does graphical lasso fail to select and fit a graphical model?
method Computational experiments with graphical lasso.
result Graphical lasso may fail when the number of data points is less than the maximum likelihood threshold.