arXiv research

R package for Bayesian empirical likelihood sampling using HMC.

Bayesian networks with latent variables are characterized and their likelihoods compared.

A new two-step MH method for Bayesian EL computation.

Develops an empirical likelihood framework for random forests and ensembles.

A new ABC method uses variational approximations for efficient inference.

The study simplifies assessing overlap in logistic regression models using empirical likelihood.

Proposes a method for valid inference in GPLSIMs with longitudinal data.

New method for efficient inference in large datasets.

In this study, we consider an empirical Bayes method for Boltzmann machines and propose an algorithm for it. The empirical Bayes method allows estimation of the values of the hyperparameters of the Boltzmann machine by maximizing a specific likelihood function referred to as the empirical Bayes likelihood function in t…

Many scientifically well-motivated statistical models in natural, engineering and environmental sciences are specified through a generative process, but in some cases it may not be possible to write down a likelihood for these models analytically. Approximate Bayesian computation (ABC) methods, which allow Bayesian inf…

The Restricted Boltzmann Machines (RBM) can be used either as classifiers or as generative models. The quality of the generative RBM is measured through the average log-likelihood on test data. Due to the high computational complexity of evaluating the partition function, exact calculation of test log-likelihood is ver…

Neural Empirical Bayes estimates source distributions from noisy simulations.

Bayesian framework uses AI-generated data to improve parameter estimation.

Invertibility conditions for observation-driven time series models often fail to be guaranteed in empirical applications. As a result, the asymptotic theory of maximum likelihood and quasi-maximum likelihood estimators may be compromised. We derive considerably weaker conditions that can be used in practice to ensure t…

Bayesian method infers contextual bandit policies robustly.

We investigate the problem of semi-parametric maximum likelihood under constraints on summary statistics. Such a procedure results in a discrete probability distribution that maximises the likelihood among all such distributions under the specified constraints (called estimating equations), and is an approximation to t…

This work explores maximum likelihood optimization of neural networks through hypernetworks. A hypernetwork initializes the weights of another network, which in turn can be employed for typical functional tasks such as regression and classification. We optimize hypernetworks to directly maximize the conditional likelih…

Introduces BPEL for EL, enhancing flexibility and using MCMC for inference.

Empirical Bayes method improves Gaussian sequence model inference.

New estimators outperform maximum likelihood without hyper-parameter estimation.

New method improves estimation of complex models from conditional moment restrictions.

We propose an estimator and confidence interval for computing the value of a policy from off-policy data in the contextual bandit setting. To this end we apply empirical likelihood techniques to formulate our estimator and confidence interval as simple convex optimization problems. Using the lower bound of our confiden…

We propose an efficient algorithm for approximate computation of the profile maximum likelihood (PML), a variant of maximum likelihood maximizing the probability of observing a sufficient statistic rather than the empirical sample. The PML has appealing theoretical properties, but is difficult to compute exactly. Inspi…

We seek to infer the parameters of an ergodic Markov process from samples taken independently from the steady state. Our focus is on non-equilibrium processes, where the steady state is not described by the Boltzmann measure, but is generally unknown and hard to compute, which prevents the application of established eq…

Parameters defined via General Estimating Equations (GEE) can be estimated by maximizing the Empirical Likelihood (EL). Newey and Smith (2004) have recently shown that this EL estimator exhibits desirable higher-order asymptotic properties, namely, that its O(n^-1) bias is small and that bias-corrected EL is higher-ord…

Parameters defined via general estimating equations (GEE) can be estimated by maximizing the empirical likelihood (EL). Newey and Smith [Econometrica 72 (2004) 219--255] have recently shown that this EL estimator exhibits desirable higher-order asymptotic properties, namely, that its $O(n^{-1})$ bias is small and that …

Maximum likelihood training improves the performance of score-based diffusion models.

A new method for training diffusion models using likelihood matching.

Proposes efficient Gaussian approximations for non-Gaussian likelihoods.

Neural networks approximate likelihood ratios for complex models.

We study the Nonparametric Maximum Likelihood Estimator (NPMLE) for estimating Gaussian location mixture densities in $d$-dimensions from independent observations. Unlike usual likelihood-based methods for fitting mixtures, NPMLEs are based on convex optimization. We prove finite sample results on the Hellinger accurac…

PEP improves deep network performance and calibration by perturbing optimal parameters.

A fundamental problem arising in many areas of machine learning is the evaluation of the likelihood of a given observation under different nominal distributions. Frequently, these nominal distributions are themselves estimated from data, which makes them susceptible to estimation errors. We thus propose to replace each…

A new method of moments estimator goes beyond data reweighting.

Paper quantifies label shift robustly.

Directly estimates Fisher score for likelihood maximization.

Proposes Likelihood Regret for VAEs to improve OOD detection.

This paper studies the estimation of low-rank Markov chains from empirical trajectories. We propose a non-convex estimator based on rank-constrained likelihood maximization. Statistical upper bounds are provided for the Kullback-Leiber divergence and the $\ell_2$ risk between the estimator and the true transition matri…

This paper connects masked pre-training to Bayesian model selection.

New SMC sampler improves diffusion model sampling efficiency.

Proposes a method to optimize neural network initialization using marginal likelihood maximization.

Research shows deep generative models' likelihoods are unreliable for anomaly detection.

We consider Bayesian inference when only a limited number of noisy log-likelihood evaluations can be obtained. This occurs for example when complex simulator-based statistical models are fitted to data, and synthetic likelihood (SL) method is used to form the noisy log-likelihood estimates using computationally costly …

The coefficient of determination, known as $R^2$, is commonly used as a goodness-of-fit criterion for fitting linear models. $R^2$ is somewhat controversial when fitting nonlinear models, although it may be generalised on a case-by-case basis to deal with specific models such as the logistic model. Assume we are fittin…

Bayesian Quadrature improves ensembling for neural networks with dispersed likelihood peaks.

In this paper, we empirically study models for pricing Italian sovereign bonds under a reduced form framework, by assuming different dynamics for the short-rate process. We analyze classical Cox-Ingersoll-Ross and Vasicek multi-factor models, with a focus on optimization algorithms applied in the calibration exercise. …

Python package for estimating Hurst exponent in fBm.

Flow-based generative models leverage invertible generator functions to fit a distribution to the training data using maximum likelihood. Despite their use in several application domains, robustness of these models to adversarial attacks has hardly been explored. In this paper, we study adversarial robustness of flow-b…