A method merges two pretrained diffusion experts to improve image quality and likelihood.
problem Trade-off between image quality and data likelihood in diffusion models.
method Combining two pretrained diffusion experts by switching between them along the denoising trajectory.
result The merged model consistently matches or outperforms its base components, improving or preserving both likelihood and sample quality.
Develops efficient method for nonconvex problems using Regula Falsi.
problem Nonconvex inverse problems with likelihood constraints.
method Regula Falsi root-finding techniques applied to level-set formulations.
result Proves extension of level-set methods to nonconvex problems.
New methods improve anomaly detection in deep networks by leveraging hierarchical likelihoods and multi-scale features.
problem Challenges in detecting anomalies in high-level features due to model bias and domain prior.
method Two methods: 1) Log likelihood ratios between in-distribution and general distribution models, 2) Multi-scale likelihood contribution.
result Strong anomaly detection performance in unsupervised setting, slightly underperforming supervised methods.
A new method trains and samples from energy-based models using diffusion recovery likelihood.
problem Training and sampling high-dimensional datasets with energy-based models is challenging.
method Trains EBMs with a diffusion recovery likelihood method, maximizing conditional probabilities of data at different noise levels.
result Generates high-fidelity images with low FID and inception scores, and accurately estimates normalized data density.
Hierarchical VAEs detect out-of-distribution data by identifying low-level in-distribution features.
problem Out-of-distribution data often has in-distribution low-level features, leading to misleading likelihood estimates in deep generative models.
method Developed a fast, scalable, unsupervised likelihood-ratio score for out-of-distribution detection based on hierarchical variational autoencoders.
result Achieved state-of-the-art results on out-of-distribution detection across various data and model combinations.
In this paper, we describe the "implicit autoencoder" (IAE), a generative autoencoder in which both the generative path and the recognition path are parametrized by implicit distributions. We use two generative adversarial networks to define the reconstruction and the regularization cost functions of the implicit autoe…
New RL algorithm GDPO improves DLM reasoning efficiency.
problem Adapting RL to DLMs for efficient, unbiased likelihood estimation.
method Group Diffusion Policy Optimization (GDPO) using semi-deterministic Monte Carlo.
result GDPO outperforms existing methods on math, reasoning, and coding benchmarks.
We introduce the multiresolution recurrent neural network, which extends the sequence-to-sequence framework to model natural language generation as two parallel discrete stochastic processes: a sequence of high-level coarse tokens, and a sequence of natural language tokens. There are many ways to estimate or learn the …
A new method optimizes neural sequence models for better task performance.
problem Training neural sequence models with maximum likelihood estimation ignores task losses.
method Maximum likelihood guided parameter search (MGS) in the parameter space.
result MGS optimizes sequence-level losses, reducing repetition and non-termination.
A diffusion model estimates data manifold dimension by tracking likelihood increases.
problem Estimating the intrinsic dimension of data manifolds.
method Trained diffusion model approximates score function, revealing manifold directionality.
result Diffusion model provides an approximation of the tangent space's dimension.
AVICA estimates noise levels for better group ICA source recovery.
problem Estimating shared independent sources from multiple noisy views.
method AVICA models each view as a linear mixture of shared sources with additive noise, optimizing noise levels alongside sources.
result AVICA yields better source estimates than other methods, especially in real-world applications like MEG and fMRI.
This paper proposes a hierarchical feature extractor for non-stationary streaming time series based on the concept of switching observable Markov chain models. The slow time-scale non-stationary behaviors are considered to be a mixture of quasi-stationary fast time-scale segments that are exhibited by complex dynamical…
We consider a problem of ecological inference, in which individual-level covariates are known, but labeled data is available only at the aggregate level. The intended application is modeling voter preferences in elections. In Rosenman and Viswanathan (2018), we proposed modeling individual voter probabilities via a log…
Proposes SGM for modeling complex dependencies in high-dimensional systems.
problem Limited pairwise interactions in PGMs for high-dimensional systems.
method Simplicial Gaussian model (SGM) using discrete Hodge theory and independent random components.
result Maximum-likelihood inference algorithm for parameter recovery and conditional dependence structure.
Bayesian inference uses Stein discrepancy for robustness in intractable likelihoods.
problem Intractable likelihoods in Bayesian inference.
method Generalised Bayesian inference with Stein discrepancy as the loss function.
result Robust generalised posteriors with closed form or accessible using MCMC.
DTM improves dLLM fine-tuning stability and performance.
problem Intractable sequence-level marginal likelihoods for masked diffusion models.
method Discrete Tilt Matching (DTM) recasts dLLM fine-tuning as state-level matching of local unmasking posteriors under reward tilting.
result DTM yields strong gains on Sudoku and Countdown while remaining competitive on MATH500 and GSM8K.
New findings suggest deep generative models can misclassify outliers, requiring new evaluation methods.
problem Deep generative models often assign higher likelihood to outliers, challenging existing outlier detection methods.
method Analyzed the typical set and high-density region of DGMs, proposing a novel outlier test.
result Existing likelihood-based outlier tests may fail due to model calibration issues, not just misclassification.
Efficient neural Bayes estimators for censored peaks-over-threshold models improve inference speed and accuracy.
problem Computational burden in inference with spatial extremal dependence models due to intractable or censored likelihoods.
method Developed neural Bayes estimators using data augmentation techniques to encode censoring information.
result Significant gains in computational and statistical efficiency compared to traditional methods.
PEP improves deep network performance and calibration by perturbing optimal parameters.
problem Improving deep network performance and calibration.
method Parameter Ensembling by Perturbation (PEP) constructs an ensemble of parameter values as random perturbations of the optimal set, maximizing log-likelihood on validation data.
result PEP provides a small to substantial improvement in calibration and log-likelihood, and in some cases, classification accuracy.
Improved autoregressive models generate higher quality images and are more robust to noise.
problem Generating high-quality images from autoregressive models.
method Noise conditional maximum likelihood estimation (MLE) with score-based sampling.
result Models trained with noise conditional MLE achieve better test likelihoods and generate higher quality images.
Noise-Contrastive Estimation improves efficiency for estimating log-likelihood of complex point processes.
problem Estimating log-likelihood of complex multivariate point processes is computationally expensive.
method Noise-Contrastive Estimation adapted for multivariate point processes, with provable guarantees.
result Our method achieves similar log-likelihood with fewer evaluations and less time.
PD curve calibration refers to the transformation of a set of rating grade level probabilities of default (PDs) to another average PD level that is determined by a change of the underlying portfolio-wide PD. This paper presents a framework that allows to explore a variety of calibration approaches and the conditions un…
A scalable method for estimating spatial data using VREML.
problem Costly computation of REML for large, sparse precision matrices in spatial data.
method Proposes VREML framework approximating marginal likelihood with Gaussian variational distribution and deriving a coordinate-ascent algorithm.
result Empirically shows VREML outperforms MLE and INLA.
Enhances neural forecasting for hierarchically organized time series data.
problem Probabilistic coherent forecasting of time series data across different levels of aggregation.
method Proposes a coherent multivariate mixture output for neural forecasting architectures, optimizing with a composite likelihood objective.
result 13.2% average accuracy improvements on most datasets compared to state-of-the-art baselines.
Graph-based LRE estimates likelihood-ratios collaboratively for nodes.
problem Comparing unknown pdfs at graph nodes with graph structure.
method Graph-based Relative Unconstrained Least-squares Importance Fitting (GRULSIF).
result Collaborative estimation improves performance compared to independent methods.
TaRP predicts missing relations in KGs using type and instance-level info.
problem Missing relations in KGs.
method Type-augmented relation prediction (TaRP) combining type and instance-level info.
result Significantly better performance on benchmark datasets.
Proposes a method for valid inference in GPLSIMs with longitudinal data.
problem Challenges in longitudinal data inference due to within-subject correlation and unstable variance estimation.
method Profile estimating-equation approach using spline approximation and block empirical likelihood.
result Block empirical likelihood ratio statistic with Wilks-type chi-square limit for joint inference.
This work tackles the challenge of aligning generative models without explicit reward signals.
problem Aligning generative models without explicit reward signals.
method A Bilevel Optimization framework where the reward function is treated as the optimization variable of an outer-level problem.
result Theoretical analysis and insights generalize to tabular classification and model-based reinforcement learning.
Text documents are structured on multiple levels of detail: individual words are related by syntax, but larger units of text are related by discourse structure. Existing language models generally fail to account for discourse structure, but it is crucial if we are to have language models that reward coherence and gener…
We investigate the choice of tuning parameters for a Bayesian multi-level group lasso model developed for the joint analysis of neuroimaging and genetic data. The regression model we consider relates multivariate phenotypes consisting of brain summary measures (volumetric and cortical thickness values) to single nucleo…
Warm starts improve Gaussian process regression by up to 16x.
problem Optimizing hyperparameters for Gaussian processes.
method Iterative Gaussian processes with warm start optimization.
result Warm starts achieve the same results as conventional methods but significantly speed up computations.
Paper proposes a dual-level approach for multi-step forecasting of dynamical systems.
problem Accurate multi-step forecasting of time series systems for automatic control and optimization.
method Hybrid input forecasting using LSTM-STMs and physics-informed neural networks (PINNs).
result Hybrid models achieve higher log-likelihood and lower MSE compared to conventional methods.
We consider a particular instance of a common problem in recommender systems: using a database of book reviews to inform user-targeted recommendations. In our dataset, books are categorized into genres and sub-genres. To exploit this nested taxonomy, we use a hierarchical model that enables information pooling across a…
We propose flow-based likelihoods to accurately capture non-Gaussian data.
problem Bypassing the Gaussian assumption in scientific analyses.
method Use optimization targets of flow-based generative models to reconstruct likelihoods.
result Flow-based likelihoods can accurately capture non-Gaussian data, improving parameter constraints.
Proposes a method to use external machine-learning predictions in multinomial logistic regression.
problem Improving statistical inference using summary-level external machine-learning predictions.
method Empirical-likelihood framework incorporating moment constraints from external nonparametric machine-learning predictions.
result Fused estimator achieves strict efficiency gain over primary-only estimator under mild conditions.
The subtle and unique imprint of dark matter substructure on extended arcs in strong lensing systems contains a wealth of information about the properties and distribution of dark matter on small scales and, consequently, about the underlying particle physics. However, teasing out this effect poses a significant challe…
Synthesizes sensor likelihoods to enforce accuracy constraints in uncertain systems.
problem Designing sensing architectures for systems with uncertain or unavailable sensor models and accuracy requirements.
method Inverts the design flow, synthesizing measurement likelihoods that minimize Kullback-Leibler divergence from the prior while enforcing an accuracy bound.
result The method synthesizes a maximum-entropy posterior and induced likelihood, accommodating various discrepancy metrics.
A new MCMC method for GPs tackles computational burden and intractable likelihoods.
problem High computational burden and intractable likelihoods in Gaussian process models.
method Combines variationally sparse Gaussian processes with pseudo-marginal MCMC.
result Asymptotically exact inference with computational gains for large datasets.
The paper proposes an efficient nested simulation design using likelihood ratio method.
problem Designing nested simulations with fixed outer scenarios and minimizing simulation effort.
method Proposes a bi-level optimization problem to decide inner replications and pooling strategies.
result Optimized design achieves $\cO(Γ^{-1})$ mean squared error of estimators.
Paper develops unbiased gradient estimator for continuous-time models.
problem Estimating unbiased gradient of log-likelihood for continuous-time models.
method Doubly randomized scheme with coupled conditional particle filter (CCPF).
result Unbiased gradient estimate facilitates gradient-based algorithms.
Neural text generation is a key tool in natural language applications, but it is well known there are major problems at its core. In particular, standard likelihood training and decoding leads to dull and repetitive outputs. While some post-hoc fixes have been proposed, in particular top-k and nucleus sampling, they …
We study the Nonparametric Maximum Likelihood Estimator (NPMLE) for estimating Gaussian location mixture densities in d-dimensions from independent observations. Unlike usual likelihood-based methods for fitting mixtures, NPMLEs are based on convex optimization. We prove finite sample results on the Hellinger accurac…
BiDVL improves EBLVMs for visual tasks by optimizing two variational distributions.
problem Training EBLVMs is challenging due to intractable distributions.
method Bi-level doubly variational learning with two tractable distributions.
result BiDVL achieves impressive image generation and reconstruction performance.
Bayesian approach sparsifies neural networks efficiently.
problem Efficiently pruning neural networks to save resources.
method Sparsifiability via the Marginal likelihood (SpaM) framework.
result Prunes neural networks effectively without significant loss in performance.
The Split-Session Cluster GARCH model captures tail heterogeneity in overnight and intraday returns.
problem Capturing tail behavior and dependence in multivariate asset returns.
method Convolution-t distributions, session and sector clustering, block-structured correlation matrices. result Session-specific and sector-level tail parameters improve model fit and out-of-sample performance.
Revises logistic-softmax likelihood for Bayesian meta-learning in few-shot classification.
problem Inherent uncertainty in logistic-softmax leads to suboptimal performance in meta-learning.
method Redesigns logistic-softmax likelihood with a temperature parameter for better control of prior confidence.
result Achieves well-calibrated uncertainty estimates and comparable/superior performance on benchmark datasets.
Improved model-based estimation through tempered Bayes filter.
problem Improving predictive accuracy in partially-observable stochastic systems.
method Developed tempered Bayes filter combining likelihood and full posterior tempering.
result Tempered Bayes filter achieves improved predictive performance over the Bayes filter baseline.
Improves generation of minority samples using diffusion models.
problem Generating minority samples on low-density regions of a data manifold.
method Introduces minority guidance to focus diffusion models on minority samples.
result Significantly improves generation of high-quality minority samples.