New methods for inferring, predicting, and estimating continuous-time, discrete-event processes.
problem Inferring, predicting, and estimating entropy rate of continuous-time, discrete-event processes.
method Bayesian structural inference extended with neural networks.
result Methods are competitive for prediction and entropy-rate estimation with state-of-the-art.
Exact Bayesian inference for discrete models using probability generating functions.
problem Discrete statistical models with infinite support and continuous priors.
method Probabilistic programming language with automatic differentiation and probability generating functions.
result Genfer tool provides exact solutions for a wide range of inference problems.
A method for efficient approximate inference on discrete distributions.
problem Applying SVGD to discrete distributions.
method Transforming discrete distributions to piecewise continuous distributions for SVGD application.
result Outperforms traditional algorithms and ensemble methods on discrete graphical models.
A new method for efficient inference in probabilistic programs with mixed support.
problem Challenges in inference for programs with both continuous and discrete latent variables.
method Stochastic gradient Markov Chain Monte Carlo algorithms.
result Outperforms existing composing inference baselines and works almost as well as inference in marginalized versions.
Develops a new Bayesian inference method for discrete data.
problem Computational challenges in discrete state spaces, especially intractable likelihoods.
method Uses a discrete Fisher divergence to update beliefs about model parameters, circumventing the intractable normalising constant.
result Establishes statistical properties of the generalised posterior and proposes a calibration approach.
This study benchmarks likelihood-free inference methods for models with heavy-tailed or discrete data.
problem Comparing likelihood-free inference methods for models with structural features like heavy-tails or discreteness.
method Four approaches: MLE, NBE, EOT, and AW-NBE are evaluated using simulations.
result The choice of evaluation tools is crucial for models with extremes and discrete data.
Novel framework combines tree-based discretization and ILP matching for causal inference.
problem Challenges in identifying causal relationships from observational data.
method Combines tree-based discretization and ILP matching for causal inference.
result Yields computational efficiency and less biased ATT estimates.
In this work we present Discrete Attend Infer Repeat (Discrete-AIR), a Recurrent Auto-Encoder with structured latent distributions containing discrete categorical distributions, continuous attribute distributions, and factorised spatial attention. While inspired by the original AIR model andretaining AIR model's capabi…
Bayesian inference of discrete component states in civil infrastructures using PGMs and GNNs.
problem Inferring discrete states of civil infrastructure components from measurable responses is an ill-posed inverse problem.
method The study proposes a novel Bayesian inversion paradigm based on Probabilistic Graphical Models (PGMs) and Graph Neural Networks (GNNs). PGMs are used to model the problem, with parameters learned from data and structural topology prior. Inference is accomplished by GNNs, and a graph property-based training strategy is developed.
result The proposed framework effectively solves the challenges of inferring the posterior PDF for discrete variables in high-dimensional problems.
A new method for categorical variational inference using discrete normalizing flows.
problem Challenges in optimizing variational approximations for discrete latent variables.
method Differentiable reparameterization using a mixture of discrete normalizing flows.
result Improves optimization of evidence lower bound and reduces sensitivity to hyperparameters.
New method improves inference for discrete diffusion models, achieving better quality and efficiency.
problem High dimensionality of discrete diffusion models causes inference challenges.
method Developed high-order numerical inference schemes for discrete diffusion models.
result Second-order accuracy of the θ-Trapezoidal method in KL divergence. Efficiently infers coupled hidden Markov models with noisy discrete observations.
problem Intractable inference for coupled continuous-time Markov chains with discrete observations.
method Latent Interacting Particle Systems, look-ahead functions, twisted Sequential Monte Carlo sampling.
result Demonstrated effectiveness on latent SIRS model and wildfire spread dynamics.
Robust Bayesian inference improves model performance on discrete data.
problem Misspecification of discrete-valued models leads to poor inference and prediction.
method Total Variation Distance (TVD) for discrepancy, efficient estimator and inference method.
result Our approach significantly improves predictive performance on various data.
Discrete choice models are commonly used by applied statisticians in numerous fields, such as marketing, economics, finance, and operations research. When agents in discrete choice models are assumed to have differing preferences, exact inference is often intractable. Markov chain Monte Carlo techniques make approximat…
Undirected graphical models are applied in genomics, protein structure prediction, and neuroscience to identify sparse interactions that underlie discrete data. Although Bayesian methods for inference would be favorable in these contexts, they are rarely used because they require doubly intractable Monte Carlo sampling…
Gradient-based MCMC for discrete spaces improves sampling performance.
problem Sampling in discrete spaces using traditional methods is challenging.
method Introduced new discrete Metropolis-Hastings samplers inspired by MALA, with a novel preconditioning technique.
result Demonstrated strong empirical performance across various challenging sampling problems.
Theory broadens GFlowNets to handle continuous spaces.
problem Limitation of GFlowNets to discrete spaces.
method Developed a theory for generalized GFlowNets.
result Empirical results show strong performance in continuous cases.
We apply belief propagation to a Bayesian bipartite graph composed of discrete independent hidden variables and discrete visible variables. The network is the Discrete counterpart of Independent Component Analysis (DICA) and it is manipulated in a factor graph form for inference and learning. A full set of simulations …
Proposes a non-parametric method for deep discrete latent variable models.
problem Learning sparse discrete latent representations in deep models.
method Iterative algorithm with Beta-Bernoulli process prior and local data scaling.
result Improves sparsity and scalability of deep discrete latent variable models.
We introduce the thermodynamic variational objective (TVO) for learning in both continuous and discrete deep generative models. The TVO arises from a key connection between variational inference and thermodynamic integration that results in a tighter lower bound to the log marginal likelihood than the standard variatio…
A new algorithm infers causal networks from data using topological thresholds.
problem Inferring causal networks from data.
method Two methods for determining topological thresholds: one to leave no disconnected nodes, the other to find a causal large connected component.
result The novel algorithm is faster and more accurate than the PC algorithm.
Drawing a sample from a discrete distribution is one of the building components for Monte Carlo methods. Like other sampling algorithms, discrete sampling suffers from the high computational burden in large-scale inference problems. We study the problem of sampling a discrete random variable with a high degree of depen…
Profile entropy measures learnability and compressibility of discrete distributions.
problem Understanding the learnability and compressibility of discrete distributions.
method Investigates profile entropy, showing its role in estimation, inference, and compression.
result Profile entropy is a fundamental measure unifying estimation, inference, and compression.
Non-autoregressive method speeds up protein folding prediction 23 times.
problem Generating protein sequences with higher order interactions.
method Discrete diffusion conditioned on 3D structure using ProteinMPNN.
result 23 times speed up in inference without performance loss.
NCDSSM models irregularly sampled time series with improved imputation and forecasting.
problem Accurate modeling of irregularly sampled time series with missing observations.
method Neural Continuous-Discrete State Space Model (NCDSSM) with amortized inference for auxiliary variables and flexible dynamic state parameterizations.
result Improved imputation and forecasting performance on multiple benchmark datasets.
Particle-based variational inference offers a flexible way of approximating complex posterior distributions with a set of particles. In this paper we introduce a new particle-based variational inference method based on the theory of semi-discrete optimal transport. Instead of minimizing the KL divergence between the po…
We address the problem of unsupervised disentanglement of discrete and continuous explanatory factors of data. We first show a simple procedure for minimizing the total correlation of the continuous latent variables without having to use a discriminator network or perform importance sampling, via cascading the informat…
Type system captures CI relationships for probabilistic models.
problem Challenges in inference for models with mixed discrete and continuous parameters.
method Information flow type system for probabilistic programming.
result Well-typed programs guarantee certain CI relationships.
A new method uncovers discrete and continuous factors in gene expression data.
problem Jointly identifying discrete and continuous factors of variability without supervision.
method cpl-mixVAE framework using multiple interacting networks.
result The method successfully uncovers discrete and continuous factors in gene expression data.
A new method uses Coordinate Descent to optimize ResNet networks for private inference.
problem Reducing ReLU count in ResNet networks for private inference.
method Directly optimizing in the discrete domain using Coordinate Descent.
result Our method yields a sparse solution and is state-of-the-art.
Bayesian inference for Levy density with Gibbs posterior in discrete sampling.
problem Inference on Levy density for financial models with jumps.
method Gibbs posterior framework using a loss function for intractable likelihood.
result Gibbs posterior achieves nearly optimal rate of convergence under certain conditions.
Paper develops Bayesian inference for discrete-choice mnp models with Gaussian priors.
problem Estimating parameters of discrete-choice multinomial probit models with Gaussian priors.
method Adapts Fasano and Durante's results to a specific mnp model with zero mean and independent Gaussian priors, simplifying posterior distribution parameters and providing a new variational algorithm.
result Simplified expressions for posterior distribution parameters and a novel variational algorithm.
Discrete diffusion models improve text and image inference.
problem Challenges in posterior sampling with discrete diffusion models.
method Anchored Posterior Sampling (APS) with quantized expectation and anchored remasking.
result APS achieves state-of-the-art performance on various tasks.
New method uses joint stochastic approximation to improve learning of discrete latent models.
problem Challenges in learning discrete latent variable models, especially with inference model gradients and log-likelihood optimization.
method Proposes a new method based on stochastic approximation theory that directly maximizes the target log-likelihood and minimizes the posterior-inference model divergence.
result Consistently outperforms recent competitive algorithms in generative modeling and structured prediction tasks.
A new RG approach connects discrete and continuous time descriptions of Gaussian processes.
problem Discretization of continuous stochastic processes for accurate simulation or model inference.
method Renormalization Group (RG) approach for Gaussian time series generated by auto-regressive models.
result RG fixed points correspond to discretizations of linear SDEs, providing insights into process accuracy.
In this paper, we deal with the problem of inferring causal directions when the data is on discrete domain. By considering the distribution of the cause P(X) and the conditional distribution mapping cause to effect P(Y∣X) as independent random variables, we propose to infer the causal direction via comparing the di…
GraphBSI generates graphs by refining a belief in continuous space, outperforming existing models.
problem Generating discrete, unordered graph data is challenging for traditional models.
method GraphBSI uses Bayesian Sample Inference (BSI) to iteratively refine a belief over graph distribution parameters.
result GraphBSI outperforms existing one-shot graph generative models on molecular and synthetic graph generation benchmarks.
In this letter we borrow from the inference techniques developed for unbounded state-cardinality (nonparametric) variants of the HMM and use them to develop a tuning-parameter free, black-box inference procedure for Explicit-state-duration hidden Markov models (EDHMM). EDHMMs are HMMs that have latent states consisting…
Proposes ABC method for discrete data, improving likelihood-free inference.
problem Discrete data likelihood-free inference problems.
method Population-based MCMC ABC framework with a new Markov kernel inspired by Differential Evolution.
result High potential and superiority of the new Markov kernel demonstrated.
This thesis studies two problems in modern statistics. First, we study selective inference, or inference for hypothesis that are chosen after looking at the data. The motiving application is inference for regression coefficients selected by the lasso. We present the Condition-on-Selection method that allows for valid s…
Bayesian framework for semiparametric regression of discrete data.
problem Complex distributional features of discrete data.
method Semiparametric modeling with nonparametric marginal and latent linear regression.
result Posterior consistency and analytical/posterior predictive distributions.
The paper develops methods to estimate frequencies in large discrete data sets with improved coverage and robustness.
problem Estimating frequencies in large, discrete data sets with valid coverage and robustness.
method Conformal inference methods using discrete sketches, marginal coverage for queries, and novel conformal calibration.
result Improved empirical performance compared to existing methods in simulations and real data.
Remasking improves the quality of discrete diffusion models for natural language and image generation.
problem Limited iterative refinement in masked discrete diffusion models.
method Introducing ReMDM sampler that allows remasking during inference.
result Remasking enables better quality outputs with increased sampling steps.
PGMax automates PGM inference on GPUs, improving quality and speed.
problem Efficient inference in complex discrete PGMs.
method Factor graph specification and loopy belief propagation in JAX.
result Higher-quality inference with up to 3x speedups.
New algorithm for aggregate inference in HMMs with continuous observations.
problem Inference in large populations with indistinguishable individuals and continuous measurements.
method Continuous observation collective forward-backward algorithm extending existing discrete case algorithm.
result Efficacy demonstrated through numerical experiments.
High-dimensional inference for sparse spectral precision matrices
problem Inference on the spectral precision matrix at a fixed frequency
method Full likelihood-based inference using neighboring discrete Fourier transforms
result Simultaneous control of regularization, finite-sample truncation, and smoothing biases
We show how to use a variational approximation to the logistic function to perform approximate inference in Bayesian networks containing discrete nodes with continuous parents. Essentially, we convert the logistic function to a Gaussian, which facilitates exact inference, and then iteratively adjust the variational par…
Graph Energy Matching improves generation quality for molecular graphs.
problem Discrete energy-based models struggle with efficient and high-quality sampling for graph generation.
method Inspired by transport-map optimization, Graph Energy Matching learns a permutation-invariant potential energy to guide sampling.
result GEM matches or surpasses discrete diffusion baselines on molecular graph benchmarks.