We present Sequential Neural Likelihood (SNL), a new method for Bayesian inference in simulator models, where the likelihood is intractable but simulating data from the model is possible. SNL trains an autoregressive flow on simulated data in order to learn a model of the likelihood in the region of high posterior dens…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “sequential neural likelihood”
SNPLA uses normalizing flows for efficient inference in implicit models.
problem Efficient inference in implicit models with complex likelihood and posterior learning.
method Sequential Neural Posterior and Likelihood Approximation (SNPLA) algorithm using normalizing flows.
result SNPLA achieves competitive performance with faster posterior draws compared to MCMC methods.
New method improves simulation-based inference by avoiding model misspecification.
problem Inefficient parameter estimation for models with intractable likelihoods.
method Proposes a robust SNL method with additional adjustment parameters.
result Demonstrates more accurate point estimates and uncertainty quantification.
Improved likelihood-free inference using preconditioned neural posterior estimation.
problem Inaccurate posterior estimation in likelihood-free inference methods.
method Preconditioned Neural Posterior Estimation (PNPE) and Sequential PNPE (PSNPE) methods.
result PNPE and PSNPE improve posterior estimation accuracy over NPE and SNPE.
Improved neural likelihood estimation for SSMs with truncated-SNL.
problem Challenges in parameter inference for state-space models.
method Truncated-SNL: a novel inference algorithm addressing SNL's limitations.
result Truncated-SNL is more accurate, scalable, and sample-efficient.
A new method improves SNPE for intractable likelihood models.
problem Simulation-based models with intractable likelihoods.
method Adaptive calibration kernel and variance reduction techniques.
result The proposed method provides a better approximation of the posterior.
Likelihood-free inference refers to inference when a likelihood function cannot be explicitly evaluated, which is often the case for models based on simulators. Most of the literature is based on sample-based `Approximate Bayesian Computation' methods, but recent work suggests that approaches based on deep neural condi…
Improves likelihood-free inference by using a new sampling approach to avoid biased data collection.
problem Efficient Bayesian inference without likelihood evaluation for real-world datasets.
method Introduces Neural Proposal (NP) to sample simulation inputs i.i.d. for unbiased posterior inference.
result Demonstrates improved performance, especially for multi-modal posteriors, through experiments.
Pseudo-Likelihood Inference improves ABC for high-dimensional Bayesian inference.
problem Intractable likelihood in Bayesian system identification.
method PLI combines neural approximation with integral probability metrics and adaptive bandwidth.
result PLI outperforms SNPE on challenging tasks, especially with more data.
LSBI approximates likelihood with linear functions for cosmological parameter estimation.
problem Estimating cosmological parameters from complex data.
method Sequential Linear Simulation-based Inference (LSBI) using Gaussian approximations.
result LSBI achieves convergence after 4-5 rounds of simulations, comparable to neural methods.
Leveraging Nested MLMC for Sequential Neural Posterior Estimation with Intractable Likelihoodsstat.CO
Paper proposes nested MLMC for SNPE with intractable likelihoods.
problem Estimating posterior distributions from intractable likelihoods.
method Nested MLMC for loss function and gradients, with convergence results.
result Effective methods for approximating complex multimodal posteriors.
New method uses path signatures for efficient likelihood estimation in time-series data.
problem Intractable likelihood functions in complex dynamic models.
method Kernel classifier based on path signatures for sequential data.
result Path signatures yield highly performant classifiers, even with low sample numbers.
SSNL improves simulation-based inference for high-dimensional data.
problem Performance degradation in neural likelihood estimation for high-dimensional data.
method Surjective Sequential Neural Likelihood (SSNL) using surjective normalizing flow models.
result SSNL avoids manual crafting of summary statistics and outperforms state-of-the-art methods.
Hidden Markov model (HMM) has been successfully used for sequential data modeling problems. In this work, we propose to power the modeling capacity of HMM by bringing in neural network based generative models. The proposed model is termed as GenHMM. In the proposed GenHMM, each HMM hidden state is associated with a neu…
SPRT-TANDEM improves sequential classification accuracy with fewer samples.
problem Efficiently classifying sequential data with high accuracy and low sampling cost.
method Deep neural network-based SPRT algorithm that estimates log-likelihood ratio of two hypotheses.
result SPRT-TANDEM achieves statistically significantly better classification accuracy than other classifiers with fewer samples.
This work explains why GANs are less used for NLP tasks.
problem Why adversarial approaches like GANs are not widely used for NLP tasks.
method Theoretical analysis and reductions showing that maximizing likelihood is equivalent to minimizing distinguishability for certain models.
result Maximizing likelihood is as effective as minimizing distinguishability for NLP tasks.
Model change points in time-series data with neural SDEs and variational autoencoders.
problem Modeling change points in time-series data with neural stochastic differential equations.
method Proposes a novel model formulation and training procedure based on the variational autoencoder framework, alternating between updating neural SDE parameters and change points.
result Demonstrates the expressive power of the proposed model in modeling both classical parametric SDEs and real datasets with distribution shifts.
Unified statistical framework for LSTM model selection.
problem Model selection and hyperparameter tuning in LSTM networks is heuristic and computationally expensive.
method Proposes a statistical framework extending classical model selection ideas to LSTM networks.
result Improved performance of the proposed framework demonstrated on biomedical data.
A new method for Bayesian inference using diffusion models.
problem Bayesian inference in simulator-based models.
method Score-based diffusion models trained with a sequential training procedure.
result Comparable or superior performance compared to existing methods.
A universal framework for constructing confidence sets using sequential likelihood mixing.
problem Constructing reliable confidence sets for realizable likelihood functions.
method Sequential likelihood mixing, integrating Bayesian inference and regret inequalities.
result Establishes fundamental connections and provable coverage guarantees for various inference techniques.
NAS-X improves inference and model learning for SLVMs.
problem Challenges in analytic inference and model learning for flexible SLVMs.
method NAS-X combines reweighted wake-sleep and smoothing sequential Monte Carlo.
result NAS-X provides low-bias and low-variance gradient estimates.
SFSVI uses Gaussian mixtures to approximate neural network outputs for continual learning.
problem Learning new tasks without forgetting old ones in neural networks.
method Sequential function-space variational inference with Gaussian mixture approximation.
result Gaussian mixture SFSVI outperforms other methods in continual learning.
A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithfulstat.ML
Simulation-based inference methods can produce unreliable posterior approximations.
problem Reliability of simulation-based inference methods for scientific use cases.
method Benchmarked algorithms including Neural Posterior Estimation, Neural Ratio Estimation, Sequential Neural Likelihood, and Approximate Bayesian Computation.
result Ensembling posterior surrogates provides more reliable approximations.
How can one perform Bayesian inference on stochastic simulators with intractable likelihoods? A recent approach is to learn the posterior from adaptively proposed simulations using neural network-based conditional density estimators. However, existing methods are limited to a narrow range of proposal distributions or r…
New model generates graphs with tighter likelihood bounds and better quality.
problem Intractable likelihood of autoregressive graph models.
method Derive exact joint probability, approximate node orderings, variational inference.
result Lower bound on log-likelihood is significantly tighter than previous methods.
We build on auto-encoding sequential Monte Carlo (AESMC): a method for model and proposal learning based on maximizing the lower bound to the log marginal likelihood in a broad family of structured probabilistic models. Our approach relies on the efficiency of sequential Monte Carlo (SMC) for performing inference in st…
Bayesian neural networks with dependent weights converge to Gaussian mixtures.
problem Limitations of standard Gaussian priors in neural networks.
method Posterior analysis with Gaussian likelihood for networks with dependent weights.
result Posterior distribution identified in the wide-width limit, ensuring invertibility of random covariance matrix.
Improved SBI with neural networks for complex models.
problem Accurate inference for complex models with intractable likelihood.
method Structured mixtures of probability distributions for likelihood and posterior approximation.
result Accurate posterior inference with smaller computational footprint.
The paper proposes a method to construct confidence sets using likelihood ratios for sequential decision-making.
problem Constructing valid uncertainty estimates for unknown quantities in sequential decision-making.
method The method uses likelihood ratios to create any-time valid confidence sequences without specialized treatment for each application.
result The proposed confidence sets maintain the prescribed coverage in a model-agnostic manner and their size depends on the choice of estimator sequence.
Continual learning is the ability to sequentially learn over time by accommodating knowledge while retaining previously learned experiences. Neural networks can learn multiple tasks when trained on them jointly, but cannot maintain performance on previously learned tasks when tasks are presented one at a time. This pro…
New SMC sampler improves diffusion model sampling efficiency.
problem Sampling generative diffusion models efficiently.
method Constructs correlated observation paths and designs a sampler.
result Improved statistical efficiency, especially under outlier conditions.
Sequential hypothesis testing is a desirable decision making strategy in any time sensitive scenario. Compared with fixed sample-size testing, sequential testing is capable of achieving identical probability of error requirements using less samples in average. For a binary detection problem, it is well known that for k…
Composite likelihood inference of fractional Gaussian processes with sequentially optimal subset selection
problem Estimating parameters in time series
method Composite likelihood method
result The method reduces computational cost
SNVI combines likelihood estimation with variational inference for efficient Bayesian inference.
problem Bayesian inference in models with intractable likelihoods.
method Sequential Neural Variational Inference (SNVI) that combines likelihood-estimation with variational inference.
result SNVI is more computationally efficient than previous algorithms without sacrificing accuracy.
The likelihood for the parameters of a generalized linear mixed model involves an integral which may be of very high dimension. Because of this intractability, many approximations to the likelihood have been proposed, but all can fail when the model is sparse, in that there is only a small amount of information availab…
A new method learns state and proposal dynamics in state-space models using neural networks.
problem Inference in non-linear state-space models.
method StateMixNN method using neural networks for proposal and transition distributions.
result Significantly improved recovery of hidden state, especially in highly non-linear scenarios.
We consider Bayesian inference when only a limited number of noisy log-likelihood evaluations can be obtained. This occurs for example when complex simulator-based statistical models are fitted to data, and synthetic likelihood (SL) method is used to form the noisy log-likelihood estimates using computationally costly …
Efficient methods for answering complex probabilistic queries in sequential data.
problem Complex probabilistic queries in sequential data.
method Broad class of novel approximation techniques for marginalization in sequential models.
result Efficient techniques for answering long-range probabilistic queries.
New method for sequential probability assignment reduces regret using contextual Shtarkov sums.
problem Minimizing regret in sequential probability assignment with arbitrary hypothesis classes.
method Introducing contextual Shtarkov sum and contextual Normalized Maximum Likelihood (cNML) algorithm.
result The contextual Shtarkov sum characterizes minimax regret and provides a minimax optimal strategy.
A benchmark for simulation-based inference methods.
problem Lack of a public benchmark for 'likelihood-free' algorithms.
method Provided a benchmark with tasks and performance metrics, including neural networks and ABC methods.
result State-of-the-art algorithms have room for improvement, and neural network-based approaches generally perform better.
We study the neural-linear bandit model for solving sequential decision-making problems with high dimensional side information. Neural-linear bandits leverage the representation power of deep neural networks and combine it with efficient exploration mechanisms, designed for linear contextual bandits, on top of the last…
This paper introduces NPR, a technique to improve Bayesian inference for multi-modal, high-dimensional simulations.
problem Challenges in Bayesian inference for multi-modal, high-dimensional simulations.
method Introduces Neural Posterior Regularization (NPR) to enforce exploration of input parameter space.
result Empirically validated that NPR significantly improves performance on various simulation tasks.
Review of diffusion models for SBI in non-ideal data scenarios.
problem Inference of parameters from complex simulation outputs with intractable likelihoods.
method Diffusion models for likelihood-free inference, addressing model misspecification, unstructured observations, and missing data.
result Improved robustness and efficiency in SBI methods for non-ideal data scenarios.
The paper introduces a method for fitting complex models using simulation and optimization.
problem Fitting models with intractable likelihood or moments.
method Sequential sampling and local smoothing, combining global and local search phases.
result The proposed method outperforms alternative approaches in fitting complex models.
cvHM framework speeds up GP inference for neural spike train analysis.
problem Scalability issue in approximate inference for latent GP models.
method cvHM framework using Hida-Matérn kernels and conjugate computation variational inference (CVI).
result Linear time inference for latent neural trajectories.
This paper presents a sequential method to identify the topological ordering of causal DAGs using likelihood ratio scores.
problem Identifying the causal relationships in a data mining scenario with ambiguity of causal directions.
method A general sequential sorting procedure that orders variables one at a time, starting at root nodes, followed by children of the root nodes, and so on until completion. Simple likelihood ratio scores are used to decide the next node to append to the current partial ordering.
result The population version of the procedure provably identifies a true ordering of the underlying DAG under mild assumptions.
BI-EqNO: Generalized Approximate Bayesian Inference with an Equivariant Neural Operator Frameworkstat.ML
BI-EqNO improves Bayesian inference with flexible neural operators.
problem Inaccurate estimation of marginal likelihoods in approximate Bayesian methods.
method Equivariant neural operator framework for generalized approximate Bayesian inference.
result BI-EqNO enhances both deterministic and stochastic approaches to Bayesian inference.
LatentTrack generates model parameters online for nonstationary data.
problem Online probabilistic prediction under nonstationary dynamics.
method Sequential neural architecture with latent filtering and amortized inference.
result Consistently lower negative log-likelihood and mean squared error than baselines.