A new method for estimating random utility models using rank-breaking and composite marginal likelihood.
problem Estimating random utility models efficiently and accurately.
method Rank-breaking-then-composite-marginal-likelihood (RBCML) framework.
result RBCML achieves better statistical efficiency and computational efficiency than existing methods.
We consider two connected aspects of maximum likelihood estimation of the parameter for high-dimensional discrete graphical models: the existence of the maximum likelihood estimate (mle) and its computation. When the data is sparse, there are many zeros in the contingency table and the maximum likelihood estimate of th…
The paper introduces multimodal generative models to improve data marginal likelihood.
problem Improving data marginal likelihood in multimodal settings.
method Derives variational bounds on the evidence for multimodal deep generative models, generalizes objectives for different model types, and benchmarks across various datasets.
result Multimodal VAEs excel in image, label, and text datasets with and without weak supervision.
Empirical Bayes method improves Gaussian sequence model inference.
problem Estimating parameters in correlated Gaussian sequence models.
method Maximum Composite Marginal Likelihood (CML) estimator, leveraging geometric Brascamp-Lieb inequality.
result CML estimator converges at rate \( n_*^{-1/2} \) in weighted Hellinger distance.
Efficient logistic regression for aggregated data reduces computation time.
problem Inference for logistic regression models is computationally expensive for large datasets.
method Adapted symbolic data analysis to summarise predictor variables into histograms and use composite likelihoods.
result The method achieves comparable classification rates to full data analysis but at lower computational cost.
Automating statistical modelling is a challenging problem in artificial intelligence. The Automatic Statistician takes a first step in this direction, by employing a kernel search algorithm with Gaussian Processes (GP) to provide interpretable statistical models for regression problems. However this does not scale due …
Estimates log marginal likelihood using multilevel Monte Carlo.
problem Estimating log marginal likelihood accurately.
method Unbiased multilevel Monte Carlo estimator.
result Validates application in variational Bayes.
A new model combines deep learning and Gaussian Processes with hyperdata learning.
problem Combining deep learning and Gaussian Processes for expressive and robust learning.
method Conditional Deep Gaussian Process (DGP) with hyperdata learning and approximate inference.
result Conditional DGP offers better expressiveness and robustness compared to existing methods.
Bayesian evidence helps compare models but can overfit.
problem Comparing hypotheses consistent with observations.
method Marginal likelihood, Occam's razor, PAC-Bayes bounds.
result Marginal likelihood can negatively correlate with generalization.
Efficiently estimates marginal likelihood using SGAIS.
problem Estimating marginal likelihood in i.i.d. data settings.
method Stochastic Gradient Annealed Importance Sampling (SGAIS).
result Significantly faster and more accurate estimates of marginal likelihood.
New method resolves nonidentifiability in mixture models.
problem Nonidentifiability in marginal models of mixture models.
method Introducing an effective temperature to generalize the marginal likelihood.
result Maximization of the generalized likelihood leads to unique results.
Derives M2VAE objective from marginal joint log-likelihood.
problem Training Multi-Modal Variational Autoencoders (M2VAEs). method Derives trainable evidence lower bound from marginal joint log-likelihood.
result Derives M2VAE objective from marginal joint log-likelihood. New method estimates marginal likelihood for deep learning models using training data alone.
problem Estimation difficulties in marginal likelihood for model selection in deep learning.
method Scalable marginal likelihood estimation based on Laplace's method and Gauss-Newton approximations.
result Estimate outperforms cross-validation and manual tuning on various datasets.
A novel likelihood function for MRFs approximates marginal likelihoods and uses copulas to reconstruct the joint likelihood.
problem Intractable partition function for MRF likelihoods.
method Approximate marginal likelihoods through a modified coin-tossing scenario, then reconstruct the joint likelihood using copulas.
result Our approach outperforms Laplace approximation and pseudolikelihood, especially as MRF size increases.
Paper develops methods for estimating and forecasting integer-valued trawl processes.
problem Estimation and forecasting of continuous-time integer-valued trawl processes.
method Composite likelihood methods, focusing on pairwise likelihood.
result Consistency and asymptotic normality of the estimator in the short memory case.
Composite likelihood inference of fractional Gaussian processes with sequentially optimal subset selection
problem Estimating parameters in time series
method Composite likelihood method
result The method reduces computational cost
Bayesian models use marginal likelihood; non-Bayesian use cross-validation, shown equivalent.
problem Comparing Bayesian and non-Bayesian models for evaluation.
method Showed marginal likelihood is equivalent to leave-p-out cross-validation, with log posterior predictive as scoring rule.
result Marginal likelihood and cross-validation are formally equivalent under data exchangeability.
Bayesian approach learns invariances from data alone, but last layer approximation is not always sufficient.
problem Learning invariances in neural networks using only training data.
method Bayesian marginal likelihood for last layer, custom optimisation routine, new lower bound.
result Partial success on standard benchmarks and medical imaging dataset, failure on CIFAR10.
Learn invariances in models using the marginal likelihood.
problem Generalizing well in supervised learning tasks.
method Learn invariances in model structure using the marginal likelihood.
result Demonstrated for Gaussian process models, reducing complexity of invariant models.
Conditional DGP learns effective kernels from low-fidelity data.
problem Learning effective kernels for multi-fidelity regression.
method Conditional DGP with moment matching for implicit kernel approximation.
result Effective kernels are learned from lower-fidelity data, improving multi-fidelity regression.
Improved GP models for scalable large data sets.
problem Computational infeasibility of Gaussian process models for large datasets.
method Composite likelihood approach with recursive computation and hyper-parameter learning.
result The derived composite GP model provides accurate predictions and hyper-parameter learning.
New particle filter estimates model evidence without bias.
problem Unbiased estimation of marginal likelihood for model comparison.
method Particle filter with rejection control.
result Unbiased estimation of marginal likelihood.
Unified tractability conditions for various compositional inference queries.
problem Analyzing tractability of probabilistic and causal inference queries.
method Algebraic perspective on circuits, focusing on semiring operators.
result Unified sufficient conditions for tractable composition of operators.
SUMO provides unbiased log marginal likelihood estimation for latent variable models.
problem Biased estimates of log marginal likelihood in latent variable models.
method Randomized truncation of infinite series for unbiased estimation.
result Models trained with SUMO give better test-set likelihoods than standard methods.
We study losses for binary classification and class probability estimation and extend the understanding of them from margin losses to general composite losses which are the composition of a proper loss with a link function. We characterise when margin losses can be proper composite losses, explicitly show how to determ…
Proposes a method to optimize neural network initialization using marginal likelihood maximization.
problem Optimizing hyperparameters for neural network initialization.
method Leverages the connection between neural networks and Gaussian processes to infer optimal hyperparameters.
result Marginal likelihood maximization provides near-optimal prediction performance on MNIST classification tasks.
Improved Gaussian process regression with tighter log marginal likelihood bounds.
problem Improving predictive performance in Gaussian process regression models.
method Lower bound on log marginal likelihood using conjugate gradients.
result Improved predictive performance compared to other conjugate gradient based approaches.
We present and implement two algorithms for analytic asymptotic evaluation of the marginal likelihood of data given a Bayesian network with hidden nodes. As shown by previous work, this evaluation is particularly hard for latent Bayesian network models, namely networks that include hidden variables, where asymptotic ap…
Develops likelihood-based methods for trawl processes, improving forecasting accuracy.
problem Statistical modeling of trawl processes with heavy tails and long memory.
method Composite likelihood estimation as a stochastic optimization problem, using gradient descent methods.
result New gradient estimators with significantly reduced variance for trawl processes.
A fast Bayesian optimization method using threshold-guided marginal likelihood maximization.
problem Efficiently optimizing models with Gaussian process regression.
method Guided marginal likelihood maximization with a pre-defined threshold to reduce model selection steps.
result Significantly reduces execution time without compromising optimization quality.
The paper discusses the impact of prior densities on Bayesian model selection.
problem The sensitivity of marginal likelihood to prior choice in Bayesian model selection.
method Analyzes the role of prior densities in model selection, discusses improper priors, and proposes solutions.
result Marginal likelihood can be sensitive to prior choice, but improper priors can still be used with caution.
Improves hyperparameter learning in GP models with non-conjugate likelihoods.
problem Hyperparameter learning entangled with approximate inference in GP models.
method Hybrid training procedure combining VI for inference and EP-like marginal likelihood approximation for hyperparameter learning.
result Empirically demonstrates the effectiveness of the proposed training procedure across various data sets.
The paper derives a formula for factorizing categorical data to improve Bayes classifiers.
problem Improving the accuracy of Bayes classifiers by effectively factoring multidimensional data.
method Derives an explicit formula for calculating the marginal likelihood of a factorized categorical dataset.
result The derived formula can be used to select the best factorization for constructing a Bayes classifier.
New variational bounds improve posterior covariances and likelihoods.
problem Improving variational inference with different divergence measures.
method Applying variational perturbation theory to construct new variational bounds.
result New variational bounds lead to more accurate posterior covariances and higher likelihoods.
Improves Gaussian process regression without bias.
problem Bias in Gaussian process regression estimates.
method Adaptive computation selection to minimize bias.
result Guaranteed small bias in log marginal likelihood estimates.
Warm starts improve Gaussian process regression by up to 16x.
problem Optimizing hyperparameters for Gaussian processes.
method Iterative Gaussian processes with warm start optimization.
result Warm starts achieve the same results as conventional methods but significantly speed up computations.
It has been argued that in supervised classification tasks, in practice it may be more sensible to perform model selection with respect to some more focused model selection score, like the supervised (conditional) marginal likelihood, than with respect to the standard marginal likelihood criterion. However, for most Ba…
New method improves variational inference for likelihood-free models.
problem Efficiently approximate posterior distributions in likelihood-free models.
method Forward amortized inference using joint-contrastive variational loss.
result Forward amortized inference optimizes exact posterior marginals in mean-field approximations.
New algorithm improves latent variable model estimation.
problem Estimating parameters in latent variable models.
method Jarzynski-adjusted Langevin algorithm (JALA) for SMC methods.
result JALA-EM provides maximum marginal likelihood estimate.
Improved likelihood-free inference by localizing and refining low-dimensional approximations.
problem Poor performance of common likelihood-free methods in high-dimensional models.
method Localisation followed by refinement of low-dimensional summaries.
result Improved accuracy in marginal posteriors through localized and refined approximations.
Deep Gaussian processes provide a flexible approach to probabilistic modelling of data using either supervised or unsupervised learning. For tractable inference approximations to the marginal likelihood of the model must be made. The original approach to approximate inference in these models used variational compressio…
Identifies interpretable generative model for multivariate data.
problem Black-box architectures of deep generative models are often unidentified and difficult to interpret.
method Introduces Deep Discrete Encoder (DDE) Copula, a hierarchical binary latent variable model inside a copula framework.
result Establishes conditions for identification of DDE copula parameters and proves posterior consistency.
We consider discrete graphical models Markov with respect to a graph G and propose two distributed marginal methods to estimate the maximum likelihood estimate of the canonical parameter of the model. Both methods are based on a relaxation of the marginal likelihood obtained by considering the density of the variable…
Bayesian inference in the presence of an intractable likelihood function is computationally challenging. When following a Markov chain Monte Carlo (MCMC) approach to approximate the posterior distribution in this context, one typically either uses MCMC schemes which target the joint posterior of the parameters and some…
Adjustment reduces bias in widely applicable Bayesian information criterion.
problem Overestimation of widely applicable Bayesian information criterion.
method Identified and adjusted an overestimating term in the criterion.
result Asymptotically unbiased estimator of log marginal likelihood.
The study examines volatility models and finds decoupling of short- and long-term correlation structures.
problem Understanding the dynamic of volatility at different time scales.
method Developed a composite likelihood estimation framework for parametric continuous-time stationary Gaussian processes.
result The short- and long-term correlation structures of stochastic volatility are decoupled.
Proposes a more efficient knot selection method for sparse Gaussian processes.
problem Optimizing marginal likelihood for knot selection leads to suboptimal and inefficient placement of knots.
method Uses Bayesian optimization to propose knots one at a time, avoiding multimodal surface issues.
result Improves both accuracy and speed of knot selection compared to current methods.
Optimizes AIS hyperparameters for efficient marginal likelihood estimation.
problem Limited computation budget affects AIS performance.
method Flexible intermediary distributions defined by residual density, parameter sharing, and fix linear schedule.
result Optimized-Path AIS reduces sampling iterations and improves performance.