Develops methods for constructing likelihoods and priors for Bayesian networks.
problem Learning parameters and structure of Bayesian networks from limited data.
method Introduces assumptions for constructing likelihoods and priors from small assessments.
result Allows construction of likelihoods and priors for a wide range of network structures.
KELFI improves inference accuracy in likelihood-free settings with limited simulations.
problem Intractable likelihood evaluations in likelihood-free inference.
method Kernel embedding likelihood-free inference (KELFI) learns model hyperparameters to balance accuracy and efficiency.
result Improved accuracy and efficiency on challenging inference problems in ecology.
Invertibility conditions for observation-driven time series models often fail to be guaranteed in empirical applications. As a result, the asymptotic theory of maximum likelihood and quasi-maximum likelihood estimators may be compromised. We derive considerably weaker conditions that can be used in practice to ensure t…
Paper proposes a new method for estimating conditional densities using logistic regressions.
problem Estimating conditional densities for complex distributions.
method Parametric conditional density estimation via weighted logistic regressions.
result Maximum likelihood estimates can be obtained efficiently via a block-wise alternating maximization scheme and local case-control sampling.
A new method trains and samples from energy-based models using diffusion recovery likelihood.
problem Training and sampling high-dimensional datasets with energy-based models is challenging.
method Trains EBMs with a diffusion recovery likelihood method, maximizing conditional probabilities of data at different noise levels.
result Generates high-fidelity images with low FID and inception scores, and accurately estimates normalized data density.
Improved autoregressive models generate higher quality images and are more robust to noise.
problem Generating high-quality images from autoregressive models.
method Noise conditional maximum likelihood estimation (MLE) with score-based sampling.
result Models trained with noise conditional MLE achieve better test likelihoods and generate higher quality images.
Generalizes moment-matching for exponential families with conditioning or hidden data.
problem Generalizing moment-matching conditions for exponential families with conditioning or hidden data.
method First-principles explanation and self-contained derivation of generalized moment-matching conditions.
result Derives generalized moment-matching conditions for conditional exponential families and hidden data.
We study likelihood-based methods for distribution regression with deep generative models.
problem Distribution regression with high-dimensional responses concentrated on a low-dimensional manifold.
method Likelihood-based approach using conditional deep generative models.
result Convergence rates for estimating conditional distributions in Hellinger and Wasserstein metrics.
The paper addresses ill-conditioning in large spatial data, proposing solutions for prediction and likelihood estimation.
problem Ill-conditioning of the kernel matrix in large spatial data sets.
method Introduction of various optimality criteria and solutions for managing large spatial data.
result Solutions for managing large spatial data, addressing ill-conditioning and improving prediction and likelihood estimation.
In this paper, we describe the "implicit autoencoder" (IAE), a generative autoencoder in which both the generative path and the recognition path are parametrized by implicit distributions. We use two generative adversarial networks to define the reconstruction and the regularization cost functions of the implicit autoe…
CNFs model conditional densities efficiently and avoid mode collapse.
problem Capturing strong correlations between output dimensions in structured prediction tasks.
method Conditional normalizing flows (CNFs) with a likelihood-based objective.
result CNFs achieve competitive performance in super-resolution and vessel segmentation tasks.
New method for conditional sampling using M-GANs, likely-free inference.
problem Conditional sampling of probability measures.
method Developed a novel computational approach called M-GANs based on block triangular transport.
result Accurate sampling of conditional measures in various applications.
Two synthetic likelihood methods learn EBM of likelihood from simulator data for SBI.
problem Conduct inference from experimental observations using high-fidelity simulators.
method Learn conditional EBM of likelihood using synthetic data conditioned on parameters.
result Learned likelihood combined with prior yields posterior estimate for sampling.
New method for fitting graphical models with latent variables using regularized conditional likelihood.
problem Graphical modeling with latent variables and confounding dependencies.
method Regularized conditional likelihood for exponential family graphical models.
result Framework applicable to broader settings without knowing latent variables' distribution.
Paper introduces c-Glow for efficient structured output learning.
problem Intractable computation of conditional likelihood in structured prediction models.
method Conditional Glow (c-Glow) - a conditional generative flow that computes p(y|x) exactly and efficiently.
result c-Glow outperforms state-of-the-art baselines in structured prediction tasks.
New SMC sampler improves diffusion model sampling efficiency.
problem Sampling generative diffusion models efficiently.
method Constructs correlated observation paths and designs a sampler.
result Improved statistical efficiency, especially under outlier conditions.
Develops quasi-likelihood analysis for marked point processes and applies it to Hawkes processes.
problem Analyzing multivariate marked point processes and their applications.
method Quasi-likelihood analysis for a general class of multivariate marked point processes, with focus on marked Hawkes processes.
result The quasi-likelihood analysis for marked Hawkes processes provides explicit conditions for ergodicity and Markovian transformation.
A new sampler tackles high-dimensional models with intractable likelihoods.
problem Statistical inference for models with computationally intractable likelihoods and high-dimensional parameters.
method Likelihood-free approximate Gibbs sampler focusing on lower-dimensional conditional distributions estimated by flexible regression models.
result The sampler enables fitting models with 13,140 parameters that are otherwise impossible with standard ABC techniques.
We consider two connected aspects of maximum likelihood estimation of the parameter for high-dimensional discrete graphical models: the existence of the maximum likelihood estimate (mle) and its computation. When the data is sparse, there are many zeros in the contingency table and the maximum likelihood estimate of th…
This work explores maximum likelihood optimization of neural networks through hypernetworks. A hypernetwork initializes the weights of another network, which in turn can be employed for typical functional tasks such as regression and classification. We optimize hypernetworks to directly maximize the conditional likelih…
Classifiers based on probabilistic graphical models are very effective. In continuous domains, maximum likelihood is usually used to assess the predictions of those classifiers. When data is scarce, this can easily lead to overfitting. In any probabilistic setting, Bayesian averaging (BA) provides theoretically optimal…
We extend Bayes' theorem for upper probabilities considering likelihood uncertainty.
problem Addressing uncertainty in likelihood for upper probability bounds.
method Generalization of Wasserman and Kadane's result, considering both prior and likelihood uncertainty.
result A sufficient condition for the upper bound to become an equality.
Researchers develop a method to learn CTBN structure using penalized maximum likelihood.
problem Learning the structure of continuous-time Bayesian networks (CTBNs) when the dependence structure is unknown.
method Penalized maximum likelihood method.
result The algorithm can accurately identify the network structure with high probability under certain conditions.
Bayesian framework uses AI-generated data to improve parameter estimation.
problem Parameter estimation in models with unknown or unspecified likelihood.
method Exponentially tilted empirical likelihood with Dirichlet process posterior.
result AI-generated data can provide useful regularization for parameter estimation.
Paper proposes an optimistic likelihood approximation for nonparametric likelihoods.
problem Computational intractability of evaluating likelihood functions in Bayesian statistics.
method Non-parametric approximation using distributionally robust optimization.
result Optimistic likelihood can be solved as a convex optimization problem with analytical expressions.
A new method for training diffusion models using likelihood matching.
problem Training efficient and accurate diffusion models.
method Likelihood Matching approach, quasi-likelihood approximation, score and Hessian estimation.
result Consistent matching of first two transitional moments between diffusion steps.
The study simplifies assessing overlap in logistic regression models using empirical likelihood.
problem Assessing overlap in multidimensional logistic regression models.
method Translation of Silvapulle's condition to empirical likelihood maximization, mechanized with R code.
result Minimal overlapping structures are cataloged in dimensions less than four, providing rules for higher dimensions.
Improved protein identification in mass spectrometry data.
problem Expanding peptide scoring capabilities in tandem mass spectrometry.
method Deriving concave emission distributions for dynamic Bayesian networks.
result Efficiently learned scoring function outperforms state-of-the-art.
Implicit probabilistic models are models defined naturally in terms of a sampling procedure and often induces a likelihood function that cannot be expressed explicitly. We develop a simple method for estimating parameters in implicit models that does not require knowledge of the form of the likelihood function or any d…
New method improves estimation of complex models from conditional moment restrictions.
problem Estimation of complex models from conditional moment restrictions.
method Functional Generalized Empirical Likelihood (GEL) with a practical method.
result The method achieves state-of-the-art performance on two problems.
Unified contrastive learning for likelihood-free inference.
problem Parameter inference in models with intractable likelihood.
method Unified contrastive learning scheme for both density ratio and direct posterior estimation.
result Unified approach clarifies method selection and comparison.
Research investigates limitations of likelihood-based generative models for robust classification.
problem Investigating robustness of class-conditional generative models for classification.
method Theoretical and practical analysis focusing on robustness properties of adversarial and ambiguous inputs.
result It is impossible to guarantee detectability of adversarially-perturbed inputs for robust classification.
We derive an explicit formula for likelihood function for Gaussian VARMA model conditioned on initial observables where the moving-average (MA) coefficients are scalar. For fixed MA coefficients the likelihood function is optimized in the autoregressive variables Φ's by a closed form formula generalizing regression c…
Likelihood-free inference refers to inference when a likelihood function cannot be explicitly evaluated, which is often the case for models based on simulators. Most of the literature is based on sample-based `Approximate Bayesian Computation' methods, but recent work suggests that approaches based on deep neural condi…
A universal method for hypothesis tests and confidence sets without regularity conditions.
problem Difficult inference in irregular statistical models.
method Modified likelihood ratio statistic (split LRT).
result Works for any parametric and some nonparametric models.
FlowSDR learns a low-dimensional projection preserving the response's conditional distribution.
problem Learning a low-dimensional projection that captures the response's conditional distribution.
method FlowSDR uses conditional log-likelihood maximization with monotone rational-quadratic spline flows to learn the projection and conditional density.
result FlowSDR outperforms existing SDR methods in various simulation settings and a face-age prediction task.
The likelihood function of a finite mixture model is a non-convex function with multiple local maxima and commonly used iterative algorithms such as EM will converge to different solutions depending on initial conditions. In this paper we ask: is it possible to assess how far we are from the global maximum of the likel…
New estimator for tensor weights with improved bias.
problem Estimating tensor weights from noisy data.
method Random matrix theory and KKT conditions.
result Asymptotically unbiased estimator for tensor rank.
The paper analyzes the power of MX CI tests and finds likelihood-based statistics most powerful.
problem Testing conditional independence under model-X assumptions.
method Conditional randomization test (CRT) and MX knockoffs.
result Likelihood-based statistics are most powerful in MX CI tests.
Many algorithms have been proposed for fitting network models with communities, but most of them do not scale well to large networks, and often fail on sparse networks. Here we propose a new fast pseudo-likelihood method for fitting the stochastic block model for networks, as well as a variant that allows for an arbitr…
Improved estimation of VAR-Moving Average models using GLS.
problem Estimating VAR-Moving Average models efficiently.
method Use generalized least squares (GLS) for each fixed moving average polynomial.
result Likelihood function format similar to VAR models, allowing maximum likelihood estimation.
Paper proposes an alternative to MLE for GLMs with non-canonical link functions.
problem Challenges in MLE for GLMs with non-canonical link functions.
method Variational Inequality (VI) estimation framework.
result Established finite-sample error bounds and asymptotic normality for VI estimator.
It has been argued that in supervised classification tasks, in practice it may be more sensible to perform model selection with respect to some more focused model selection score, like the supervised (conditional) marginal likelihood, than with respect to the standard marginal likelihood criterion. However, for most Ba…
Bayesian deconditional embeddings solve complex function recovery.
problem Recovering original functions from conditional mean observations.
method Formalizes deconditional kernel mean embeddings as Bayesian inference, connects to task-transformed Gaussian processes.
result Establishes deconditional kernel means as posterior predictive mean, providing Bayesian interpretations and uncertainty.
Partition Tree estimates conditional densities for mixed continuous and categorical variables.
problem Estimating conditional densities for mixed data types.
method Tree-based framework modeling conditional distributions as piecewise-constant densities on adaptive partitions, minimizing conditional negative log-likelihood.
result Improved probabilistic prediction compared to CART-style trees and state-of-the-art methods.
Alternative hypothesis tests for class-conditional noise using local maximum likelihood.
problem Assessing label noise in supervised learning datasets.
method Proposes hypothesis tests based on local maximum likelihood estimation for nonparametric logistic regression.
result Shows improved applicability and flexibility of the proposed tests compared to parametric approaches.
There has been much recent, exciting work on combining the complementary strengths of latent variable models and deep learning. Latent variable modeling makes it easy to explicitly specify model constraints through conditional independence properties, while deep learning makes it possible to parameterize these conditio…
Approximate Bayesian computation (ABC) and synthetic likelihood (SL) techniques have enabled the use of Bayesian inference for models that may be simulated, but for which the likelihood cannot be evaluated pointwise at values of an unknown parameter θ. The main idea in ABC and SL is to, for different values of θ (u…