R package for Bayesian empirical likelihood sampling using HMC.
problem Sampling from non-convex Bayesian empirical likelihood posteriors.
method Hamiltonian Monte Carlo (HMC) algorithm for numerical integration.
result Efficient HMC sampling from BayesEL posteriors.
A new ABC method uses variational approximations for efficient inference.
problem Computational challenges in Bayesian inference for complex models.
method Variational approximation for log-posterior, empirical likelihood for estimating expected log-likelihood, differential entropy estimation.
result Posterior consistency established for the proposed method.
Bayesian framework uses AI-generated data to improve parameter estimation.
problem Parameter estimation in models with unknown or unspecified likelihood.
method Exponentially tilted empirical likelihood with Dirichlet process posterior.
result AI-generated data can provide useful regularization for parameter estimation.
Proposes a method for valid inference in GPLSIMs with longitudinal data.
problem Challenges in longitudinal data inference due to within-subject correlation and unstable variance estimation.
method Profile estimating-equation approach using spline approximation and block empirical likelihood.
result Block empirical likelihood ratio statistic with Wilks-type chi-square limit for joint inference.
Bayesian networks with latent variables are characterized and their likelihoods compared.
problem Characterizing and comparing likelihoods of Bayesian networks with latent variables.
method Characterized likelihood function and empirical Bayesian network. Proved dominance of global maximum likelihood from empirical model.
result The global maximum likelihood of the original Bayesian network is attained if and only if parameters are consistent with empirical model.
The Restricted Boltzmann Machines (RBM) can be used either as classifiers or as generative models. The quality of the generative RBM is measured through the average log-likelihood on test data. Due to the high computational complexity of evaluating the partition function, exact calculation of test log-likelihood is ver…
The study simplifies assessing overlap in logistic regression models using empirical likelihood.
problem Assessing overlap in multidimensional logistic regression models.
method Translation of Silvapulle's condition to empirical likelihood maximization, mechanized with R code.
result Minimal overlapping structures are cataloged in dimensions less than four, providing rules for higher dimensions.
A new two-step MH method for Bayesian EL computation.
problem Complex likelihood support in Bayesian EL.
method Hierarchical Metropolis Hastings with reversible jump MCMC.
result Improved sampling from BayesEL posteriors.
New method improves estimation of complex models from conditional moment restrictions.
problem Estimation of complex models from conditional moment restrictions.
method Functional Generalized Empirical Likelihood (GEL) with a practical method.
result The method achieves state-of-the-art performance on two problems.
Many scientifically well-motivated statistical models in natural, engineering and environmental sciences are specified through a generative process, but in some cases it may not be possible to write down a likelihood for these models analytically. Approximate Bayesian computation (ABC) methods, which allow Bayesian inf…
Develops an empirical likelihood framework for random forests and ensembles.
problem Quantifying the statistical uncertainty of random forests and ensembles.
method Empirical likelihood framework exploiting the incomplete U-statistic structure of ensemble predictions. result Modified empirical likelihood statistic achieves accurate coverage and practical reliability.
Proposes Likelihood Regret for VAEs to improve OOD detection.
problem VAEs can assign high likelihoods to OOD samples, making traditional likelihood thresholds unreliable.
method Introduces Likelihood Regret, a new OOD score for VAEs.
result Empirical results show Likelihood Regret outperforms existing methods for VAEs.
We seek to infer the parameters of an ergodic Markov process from samples taken independently from the steady state. Our focus is on non-equilibrium processes, where the steady state is not described by the Boltzmann measure, but is generally unknown and hard to compute, which prevents the application of established eq…
New estimators outperform maximum likelihood without hyper-parameter estimation.
problem Improving system identification performance without hyper-parameter estimation.
method Developed generalized Bayes and closed-form biased estimators using excess MSE.
result New estimators have comparable performance to empirical-Bayes-based regularized estimator.
Flow-based generative models leverage invertible generator functions to fit a distribution to the training data using maximum likelihood. Despite their use in several application domains, robustness of these models to adversarial attacks has hardly been explored. In this paper, we study adversarial robustness of flow-b…
This paper connects masked pre-training to Bayesian model selection.
problem Understanding the success of masked pre-training and its generalization.
method The paper shows masked pre-training corresponds to maximizing the marginal likelihood.
result Masked pre-training with a suitable scoring function maximizes the marginal likelihood.
New method for efficient inference in large datasets.
problem Statistical inference in massive datasets.
method Combines divide-and-conquer method and empirical likelihood.
result Reduces computation burden and demonstrates effectiveness.
Research shows deep generative models' likelihoods are unreliable for anomaly detection.
problem Anomaly detection using deep generative models' likelihoods is unreliable.
method Examined the behavior of distribution densities through reparametrization.
result The likelihoods used for anomaly detection rely on strong and implicit hypotheses.
New SMC sampler improves diffusion model sampling efficiency.
problem Sampling generative diffusion models efficiently.
method Constructs correlated observation paths and designs a sampler.
result Improved statistical efficiency, especially under outlier conditions.
Empirical Bayes method improves Gaussian sequence model inference.
problem Estimating parameters in correlated Gaussian sequence models.
method Maximum Composite Marginal Likelihood (CML) estimator, leveraging geometric Brascamp-Lieb inequality.
result CML estimator converges at rate \( n_*^{-1/2} \) in weighted Hellinger distance.
Parameters defined via General Estimating Equations (GEE) can be estimated by maximizing the Empirical Likelihood (EL). Newey and Smith (2004) have recently shown that this EL estimator exhibits desirable higher-order asymptotic properties, namely, that its O(n^-1) bias is small and that bias-corrected EL is higher-ord…
Parameters defined via general estimating equations (GEE) can be estimated by maximizing the empirical likelihood (EL). Newey and Smith [Econometrica 72 (2004) 219--255] have recently shown that this EL estimator exhibits desirable higher-order asymptotic properties, namely, that its O(n−1) bias is small and that …
A new method of moments estimator goes beyond data reweighting.
problem Estimation of moment restrictions and conditional moment restrictions.
method Kernel Method of Moments (KMM) based on maximum mean discrepancy.
result KMM achieves competitive performance on conditional moment restriction tasks.
EnVAE uses energy score for likelihood-free VAEs, improving image reconstructions.
problem Likelihood misspecification in VAEs leads to blurry reconstructions and poor data fidelity.
method Deterministic decoder, energy score as reconstruction loss, fast variant for efficiency.
result EnVAE achieves superior reconstruction and generation quality compared to likelihood-based baselines.
In this study, we consider an empirical Bayes method for Boltzmann machines and propose an algorithm for it. The empirical Bayes method allows estimation of the values of the hyperparameters of the Boltzmann machine by maximizing a specific likelihood function referred to as the empirical Bayes likelihood function in t…
Differentiable structure learning addresses DAGs with multiple global minimizers.
problem Identify the true DAG from global minimizers of acyclicity-constrained optimization problems.
method Carefully regularize the likelihood to identify the sparsest model in the Markov equivalence class.
result Regularization of the likelihood defines a score that identifies the sparsest model in general models and likelihoods.
Likelihood from a generative model is a natural statistic for detecting out-of-distribution (OoD) samples. However, generative models have been shown to assign higher likelihood to OoD samples compared to ones from the training distribution, preventing simple threshold-based detection rules. We demonstrate that OoD det…
Neural Empirical Bayes estimates source distributions from noisy simulations.
problem Estimating source distributions from noisy, simulated data.
method Uses neural density estimators to estimate a prior or source distribution over uncorrupted samples, then performs posterior inference.
result Recovering ground truth source distributions up to symmetries.
Invertibility conditions for observation-driven time series models often fail to be guaranteed in empirical applications. As a result, the asymptotic theory of maximum likelihood and quasi-maximum likelihood estimators may be compromised. We derive considerably weaker conditions that can be used in practice to ensure t…
A new method for training diffusion models using likelihood matching.
problem Training efficient and accurate diffusion models.
method Likelihood Matching approach, quasi-likelihood approximation, score and Hessian estimation.
result Consistent matching of first two transitional moments between diffusion steps.
Bayesian method infers contextual bandit policies robustly.
problem Inference of contextual bandit policies in small sample sizes.
method Empirical likelihood for Bayesian inference.
result Accurate uncertainty measurements and policy comparison.
This work improves likelihood of score-based diffusion ODEs using high-order denoising score matching.
problem The gap between maximum likelihood and score matching objectives for score-based diffusion ODEs.
method High-order denoising score matching to maximize likelihood.
result Score-based diffusion ODEs achieve better likelihood on synthetic and CIFAR-10 data.
CoinDICE estimates confidence intervals for unknown behavior policies in reinforcement learning.
problem Estimating value of a target policy using only behavior policy data.
method Function space embedding, generalized empirical likelihood method, Lagrangian optimization.
result Valid confidence intervals with tighter and more accurate estimates than existing methods.
A new principle and method improve out-of-distribution detection in generative models.
problem Out-of-distribution detection in deep generative models often fails due to poor likelihood estimates.
method Introducing the Likelihood Path (LPath) principle and new theoretical tools for OOD detection.
result Non-asymptotic provable OOD detection guarantees for variational autoencoders (VAEs).
Advocates for MLE in regression and forecasting for better inductive biases and post-hoc optimization.
problem Designing effective loss functions for regression and forecasting.
method Maximum Likelihood Estimation (MLE) approach for regression and forecasting.
result MLE approach outperforms direct empirical risk minimization under certain conditions and for various datasets.
PEP improves deep network performance and calibration by perturbing optimal parameters.
problem Improving deep network performance and calibration.
method Parameter Ensembling by Perturbation (PEP) constructs an ensemble of parameter values as random perturbations of the optimal set, maximizing log-likelihood on validation data.
result PEP provides a small to substantial improvement in calibration and log-likelihood, and in some cases, classification accuracy.
SCALLOP improves likelihood flow maps for efficient Boltzmann generation.
problem Efficient estimation of model likelihood in flow-based generative models.
method SCALLOP introduces a Hutchinson-free likelihood distillation objective for scalable flow-based models.
result SCALLOP achieves up to 10x inference speedup while improving performance.
We investigate the problem of semi-parametric maximum likelihood under constraints on summary statistics. Such a procedure results in a discrete probability distribution that maximises the likelihood among all such distributions under the specified constraints (called estimating equations), and is an approximation to t…
Nonparametric density deconvolution and denoising using simulation-based inference
problem Learning latent signals and their distributions in the presence of measurement noise
method Convolutional maximum mean discrepancy (convMMD) loss and likelihood-free framework
result Learn a latent generative model matching observed data distribution
This work explores maximum likelihood optimization of neural networks through hypernetworks. A hypernetwork initializes the weights of another network, which in turn can be employed for typical functional tasks such as regression and classification. We optimize hypernetworks to directly maximize the conditional likelih…
We propose a robust estimator to improve maximum likelihood in probabilistic models.
problem Overfitting and sensitivity to noise in maximum likelihood estimation.
method Distributionally robust maximum likelihood estimator that minimizes worst-case expected log-loss.
result The robust estimator is statistically consistent and performs well in regression and classification tasks.
Introduces BPEL for EL, enhancing flexibility and using MCMC for inference.
problem Computational challenges in EL methods.
method Bayesian Penalized Empirical Likelihood (BPEL) framework with MCMC sampling.
result Enhanced flexibility and practicality of EL methods with MCMC.
The coefficient of determination, known as R2, is commonly used as a goodness-of-fit criterion for fitting linear models. R2 is somewhat controversial when fitting nonlinear models, although it may be generalised on a case-by-case basis to deal with specific models such as the logistic model. Assume we are fittin…
Expectation Maximization (EM) is among the most popular algorithms for maximum likelihood estimation, but it is generally only guaranteed to find its stationary points of the log-likelihood objective. The goal of this article is to present theoretical and empirical evidence that over-parameterization can help EM avoid …
We propose an estimator and confidence interval for computing the value of a policy from off-policy data in the contextual bandit setting. To this end we apply empirical likelihood techniques to formulate our estimator and confidence interval as simple convex optimization problems. Using the lower bound of our confiden…
Paper proposes efficient training for normalizing flows in Boltzmann generators.
problem Training normalizing flows for Boltzmann generators is computationally challenging and unstable.
method Regression Training of Normalizing Flows (RegFlow) using ℓ2-regression. result RegFlow enables efficient and stable training of normalizing flows for Boltzmann generators.
Variational autoencoders learn distributions of high-dimensional data. They model data with a deep latent-variable model and then fit the model by maximizing a lower bound of the log marginal likelihood. VAEs can capture complex distributions, but they can also suffer from an issue known as "latent variable collapse," …
Complex computer simulators are increasingly used across fields of science as generative models tying parameters of an underlying theory to experimental observations. Inference in this setup is often difficult, as simulators rarely admit a tractable density or likelihood function. We introduce Adversarial Variational O…