Direct neural ratio estimator for likelihood-free inference.
problem Efficient likelihood estimation for complex models.
method Amortized likelihood ratio estimation using neural networks.
result DNRE often outperforms previous ratio estimators.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “likelihood estimation”
A new method for training diffusion models using likelihood matching.
problem Training efficient and accurate diffusion models.
method Likelihood Matching approach, quasi-likelihood approximation, score and Hessian estimation.
result Consistent matching of first two transitional moments between diffusion steps.
Corrects pseudo log-likelihood method issues in various applications.
problem Log-likelihood function unbounded issues in pseudo log-likelihood methods.
method Provided a counterexample and corrected algorithms in previous literature.
result Ensured well-definedness of maximum pseudo log-likelihood estimation.
Maximum likelihood estimation fails to be well-posed in Gaussian process regression.
problem Establishing well-posedness of maximum likelihood estimation in Gaussian process regression.
method Analyzing the conditions under which maximum likelihood estimation is not Lipschitz in the data with respect to the Hellinger distance.
result Maximum likelihood estimation is not well-posed in the noiseless data setting for any Gaussian process with a stationary covariance function whose lengthscale parameter is estimated using maximum likelihood.
New method estimates marginal likelihood for deep learning models using training data alone.
problem Estimation difficulties in marginal likelihood for model selection in deep learning.
method Scalable marginal likelihood estimation based on Laplace's method and Gauss-Newton approximations.
result Estimate outperforms cross-validation and manual tuning on various datasets.
The fate of scientific hypotheses often relies on the ability of a computational model to explain the data, quantified in modern statistical approaches by the likelihood function. The log-likelihood is the key element for parameter estimation and model evaluation. However, the log-likelihood of complex models in fields…
New method improves causal structure discovery with Prior-Fitted Networks.
problem Errors in likelihood estimation limit proper causal structure discovery.
method Amortized causal discovery with Prior-Fitted Networks.
result Significant gains in structure recovery compared to baselines.
A new method reduces dimensionality for better likelihood-free parameter estimation.
problem Estimating parameters from data with no closed-form likelihood.
method Combines reconstruction map estimation with dimension-reduction techniques.
result The proposed method outperforms existing techniques in accuracy and efficiency.
Stochastic Gradient Annealed Importance Sampling for Efficient Online Marginal Likelihood Estimationstat.ML
We consider estimating the marginal likelihood in settings with independent and identically distributed (i.i.d.) data. We propose estimating the predictive distributions in a sequential factorization of the marginal likelihood in such settings by using stochastic gradient Markov Chain Monte Carlo techniques. This appro…
Neural networks estimate spatial process likelihoods efficiently.
problem Challenges in estimating spatial processes with slow or intractable likelihoods.
method Convolutional neural networks trained on a classification task to learn likelihood function.
result Neural likelihood surfaces provide fast and accurate parameter estimation.
We consider two connected aspects of maximum likelihood estimation of the parameter for high-dimensional discrete graphical models: the existence of the maximum likelihood estimate (mle) and its computation. When the data is sparse, there are many zeros in the contingency table and the maximum likelihood estimate of th…
SUMO provides unbiased log marginal likelihood estimation for latent variable models.
problem Biased estimates of log marginal likelihood in latent variable models.
method Randomized truncation of infinite series for unbiased estimation.
result Models trained with SUMO give better test-set likelihoods than standard methods.
The Restricted Boltzmann Machines (RBM) can be used either as classifiers or as generative models. The quality of the generative RBM is measured through the average log-likelihood on test data. Due to the high computational complexity of evaluating the partition function, exact calculation of test log-likelihood is ver…
Likelihood-based inference and forecasting for trawl processes: a stochastic optimization approachstat.ME
Develops likelihood-based methods for trawl processes, improving forecasting accuracy.
problem Statistical modeling of trawl processes with heavy tails and long memory.
method Composite likelihood estimation as a stochastic optimization problem, using gradient descent methods.
result New gradient estimators with significantly reduced variance for trawl processes.
A new VIS approach improves log-likelihood estimation in latent variable models.
problem Challenges in achieving high log-likelihood with VI for complex posterior distributions.
method Uses forward divergence to optimize proposal distribution for better log-likelihood estimation.
result Consistently outperforms state-of-the-art baselines in log-likelihood and parameter estimation.
Likelihood-free (a.k.a. simulation-based) inference problems are inverse problems with expensive, or intractable, forward models. ODE inverse problems are commonly treated as likelihood-free, as their forward map has to be numerically approximated by an ODE solver. This, however, is not a fundamental constraint but jus…
In high-dimensional data analysis, penalized likelihood estimators are shown to provide superior results in both variable selection and parameter estimation. A new algorithm, APPLE, is proposed for calculating the Approximate Path for Penalized Likelihood Estimators. Both the convex penalty (such as LASSO) and the nonc…
Recursive Maximum Likelihood Estimation for Interacting Particle Systems using Virtual Particlesstat.ME
Estimates log-likelihood of interacting particle systems using virtual particles.
problem Inconsistent estimation of finite-particle log-likelihood in large particle systems.
method Stochastic gradient estimate using continuous trajectory and virtual particle systems.
result Convergence to stationary points of limiting mean-field system's log-likelihood.
Proposes a new approach to approximate maximum likelihood for complex models.
problem Intractable likelihood functions in complex parametric models.
method Simulation-based constrained approximation to the structural model.
result Estimators nearly as efficient as maximum likelihood, feasible in many cases.
SNVI combines likelihood estimation with variational inference for efficient Bayesian inference.
problem Bayesian inference in models with intractable likelihoods.
method Sequential Neural Variational Inference (SNVI) that combines likelihood-estimation with variational inference.
result SNVI is more computationally efficient than previous algorithms without sacrificing accuracy.
A boosting method improves nonparametric density estimation without smoothing assumptions.
problem Overfitting in nonparametric data fitting.
method Introduces a boosting algorithm for univariate nonparametric maximum likelihood estimation.
result Demonstrates the effectiveness of the boosting approach through simulations and real data experiments.
We propose a robust estimator to improve maximum likelihood in probabilistic models.
problem Overfitting and sensitivity to noise in maximum likelihood estimation.
method Distributionally robust maximum likelihood estimator that minimizes worst-case expected log-loss.
result The robust estimator is statistically consistent and performs well in regression and classification tasks.
Paper develops methods for estimating and forecasting integer-valued trawl processes.
problem Estimation and forecasting of continuous-time integer-valued trawl processes.
method Composite likelihood methods, focusing on pairwise likelihood.
result Consistency and asymptotic normality of the estimator in the short memory case.
New method uses path signatures for efficient likelihood estimation in time-series data.
problem Intractable likelihood functions in complex dynamic models.
method Kernel classifier based on path signatures for sequential data.
result Path signatures yield highly performant classifiers, even with low sample numbers.
Noise-Contrastive Estimation improves efficiency for estimating log-likelihood of complex point processes.
problem Estimating log-likelihood of complex multivariate point processes is computationally expensive.
method Noise-Contrastive Estimation adapted for multivariate point processes, with provable guarantees.
result Our method achieves similar log-likelihood with fewer evaluations and less time.
New machine learning methods for inference from simulated data.
problem Modeling score and likelihood ratio functions from sampled data.
method InferoStatic Networks (ISN), Kernel Score Estimation (KSE), Kernel Likelihood Ratio Estimation (KLRE).
result Improved inference methods for complex models.
Novel neural likelihood ratio estimation for negative data in particle physics.
problem Estimating likelihood ratios with negative probability densities and weights.
method Introducing a novel loss function and a new model architecture based on signed mixture models.
result Demonstrated improved estimation on a real-world example from particle physics.
ALFI improves likelihood-free inference for black-box generators.
problem Limitations of likelihood-free inference on black-box generators.
method Adversarial Likelihood-Free Inference (ALFI) to estimate posterior distributions.
result ALFI achieves best parameter estimation accuracy with limited simulation.
Online Centralized Non-parametric Change-point Detection via Graph-based Likelihood-ratio Estimationstat.ML
Paper detects changes in graph-based data streams using likelihood-ratios.
problem Detecting changes in synchronized graph-based data streams.
method Kernel-based likelihood-ratio estimation over graph nodes.
result Effective detection and localization of change-points.
Bayesian synthetic likelihood (BSL) is a popular method for estimating the parameter posterior distribution for complex statistical models and stochastic processes that possess a computationally intractable likelihood function. Instead of evaluating the likelihood, BSL approximates the likelihood of a judiciously chose…
A common problem in disciplines of applied Statistics research such as Astrostatistics is of estimating the posterior distribution of relevant parameters. Typically, the likelihoods for such models are computed via expensive experiments such as cosmological simulations of the universe. An urgent challenge in these rese…
Paper presents a method for estimating Hawkes process parameters.
problem Estimating parameters of Hawkes processes with self-excitation or inhibition.
method Maximum likelihood estimation for Hawkes processes with self-excitation or inhibition.
result The proposed estimator provides more accurate estimations in the inhibition context.
The paper proposes a method to construct confidence sets using likelihood ratios for sequential decision-making.
problem Constructing valid uncertainty estimates for unknown quantities in sequential decision-making.
method The method uses likelihood ratios to create any-time valid confidence sequences without specialized treatment for each application.
result The proposed confidence sets maintain the prescribed coverage in a model-agnostic manner and their size depends on the choice of estimator sequence.
New method improves simulation-based inference by avoiding model misspecification.
problem Inefficient parameter estimation for models with intractable likelihoods.
method Proposes a robust SNL method with additional adjustment parameters.
result Demonstrates more accurate point estimates and uncertainty quantification.
Unbiased gradient estimation improves VAE performance.
problem Training VAEs via maximum likelihood is difficult due to intractable integrals.
method Introduced unbiased estimators of the log-likelihood gradient using coupled Markov chains.
result Unbiased estimators lead to better predictive performance in VAEs.
In this short note we provide an unbiased multilevel Monte Carlo estimator of the log marginal likelihood and discuss its application to variational Bayes.
Directly estimates Fisher score for likelihood maximization.
problem Intractable likelihood functions with model simulations.
method Gradient-based optimization using local score matching and linear parameterization.
result Efficient approximation of Fisher score improves likelihood maximization.
Minimization of Functions on Dually Flat Spaces Using Geodesic Descent Based on Dual Connectionsstat.CO
Geodesic descent optimizes likelihood in dually flat spaces.
problem Maximum likelihood estimation in exponential families.
method m-geodesic and e-geodesic updates on dually flat spaces.
result Geodesic updates can reach maximum likelihood estimator in one step.
We consider the problem of sparse estimation in a factor analysis model. A traditional estimation procedure in use is the following two-step approach: the model is estimated by maximum likelihood method and then a rotation technique is utilized to find sparse factor loadings. However, the maximum likelihood estimates c…
On the Consistency of Maximum Likelihood Estimation of Probabilistic Principal Component Analysisstat.ML
New approach resolves ambiguity in PPCA model's maximum likelihood estimation.
problem Ambiguity in maximum likelihood estimation of PPCA model due to rotational symmetry.
method Using quotient topological spaces, the approach resolves ambiguity and shows consistency of the maximum likelihood solution.
result Maximum likelihood solution is consistent in an appropriate quotient Euclidean space.
I consider two problems in machine learning and statistics: the problem of estimating the joint probability density of a collection of random variables, known as density estimation, and the problem of inferring model parameters when their likelihood is intractable, known as likelihood-free inference. The contribution o…
A new method normalizes EBM training by introducing a learnable parameter.
problem Training energy-based models with maximum likelihood is challenging due to intractable normalisation constants.
method Proposes a self-normalised log-likelihood (SNL) objective that introduces a learnable parameter representing the normalisation constant.
result The SNL objective is a lower bound of the log-likelihood and can be directly optimised using stochastic gradient techniques.
SSNL improves simulation-based inference for high-dimensional data.
problem Performance degradation in neural likelihood estimation for high-dimensional data.
method Surjective Sequential Neural Likelihood (SSNL) using surjective normalizing flow models.
result SSNL avoids manual crafting of summary statistics and outperforms state-of-the-art methods.
A new method improves text generation quality and diversity.
problem Exposure bias in Maximum Likelihood Estimation for text generation.
method ψ-MLE, a new training scheme based on density ratio estimation.
result ψ-MLE outperforms Maximum Likelihood Estimation and other models in text generation quality and diversity.
Improved MMD estimator for likelihood-free inference.
problem Computational challenges in estimating MMD for likelihood-free inference.
method Optimally-weighted MMD estimator with improved sample complexity.
result Significantly improved sample complexity for accurate MMD estimation.
Maximum likelihood (ML) estimation using Newton's method in nonlinear state space models (SSMs) is a challenging problem due to the analytical intractability of the log-likelihood and its gradient and Hessian. We estimate the gradient and Hessian using Fisher's identity in combination with a smoothing algorithm. We exp…
R package for Bayesian empirical likelihood sampling using HMC.
problem Sampling from non-convex Bayesian empirical likelihood posteriors.
method Hamiltonian Monte Carlo (HMC) algorithm for numerical integration.
result Efficient HMC sampling from BayesEL posteriors.
New method trains EBMs using NFs for more accurate likelihood estimation.
problem Lack of statistical accuracy in EBMs likelihood estimation.
method Uses normalizing flows (NF) to fit an NF to an EBM during training.
result Accurate gradient for EBMs at all times, leading to a fast sampler.