SUMO provides unbiased log marginal likelihood estimation for latent variable models.
problem Biased estimates of log marginal likelihood in latent variable models.
method Randomized truncation of infinite series for unbiased estimation.
result Models trained with SUMO give better test-set likelihoods than standard methods.
A new method reduces variance in training discrete latent variable models.
problem High variance in stochastic gradient estimators for discrete latent variable models.
method Double control variates for score function estimators using Taylor expansions.
result Our method can have lower variance compared to other estimators.
Adaptive multi-stage density ratio estimation improves learning of latent space EBM.
problem Learning energy-based models in latent space is computationally expensive and challenging.
method Adaptive multi-stage density ratio estimation using NCE to bridge the gap between prior and posterior densities.
result The method enables more expressive prior models and sharpens the latent space EBM.
Proposes a transfer learning method for accurate latent variable estimation.
problem Accurate estimation of latent variables in networks with large parameter spaces.
method Leverages information from similar networks to improve estimation accuracy.
result The proposed methods improve estimation accuracy and are validated on real datasets.
A method estimates causal parameters using a latent variable recovery.
problem Estimating causal parameters in contexts with multiple causes and unobserved confounding.
method Substitute adjustment via recovery of latent variables.
result Substitute adjustment estimates adjusted regression parameters under certain conditions.
GCAE uses density estimation to achieve reliable disentanglement in latent space.
problem Disentangled learning representations suffer from reliability issues.
method GCAE uses Gaussian Channel Autoencoder with Dual Total Correlation (DTC) to avoid the curse of dimensionality.
result GCAE achieves highly competitive and reliable disentanglement scores.
Paper models non-linear dynamics from time series data.
problem Modeling non-linear dynamical systems from time series data.
method Introduces latent state modeling and a novel alternating minimization algorithm.
result LaNoLem achieves competitive performance in dynamics estimation and prediction.
Paper proposes a novel auto-encoder for latent density estimation.
problem Challenges of learning generative probabilistic models due to curse of dimensionality.
method Joint dimensionality reduction and non-parametric density estimation framework using a novel estimator.
result Proposed model achieves promising results on various datasets.
A new gradient estimator reduces variance near boundaries for binary latent variables.
problem Explosive gradient variance near boundaries in binary latent variable models.
method Introduces a new gradient estimator (bitflip-1) and an aggregated estimator (UGC) that uses either bitflip-1 or DisARM for each coordinate.
result UGC has uniformly lower variance than DisARM and achieves optimal optimization objectives.
Random geometric graphs are a popular choice for a latent points generative model for networks. Their definition is based on a sample of n points X1,X2,⋯,Xn on the Euclidean sphere~Sd−1 which represents the latent positions of nodes of the network. The connection probabilities between the node…
Study on predicting graph labels at nodes using local averaging and distance estimation.
problem Predicting graph labels at nodes given observations at other nodes.
method Local averaging and distance estimation methods for graph regression.
result Alternative methods can achieve standard nonparametric rates even when graph neighborhoods are too large or small.
New estimators reduce variance in training variational autoencoders with discrete latent variables.
problem Training variational autoencoders with discrete latent variables requires efficient gradient estimation.
method Introduce ReinMax-Rao and ReinMax-CV estimators using Rao-Blackwellisation and control variates.
result Demonstrate superior performance on training variational autoencoders with discrete latent spaces.
The paper proposes a method to estimate latent structures in multivariate data without assuming their existence.
problem Estimating latent structures in multivariate distributions that are difficult to identify and reflect the data generating mechanism.
method A model-free approach using a multiscale nonparametric maximum likelihood estimator.
result The method captures meaningful discrete structure at different scales and integrates them to yield an interpretable discrete representation.
Framework estimates multiple plausible solutions with uncertainty measures.
problem Machine learning models need to propose multiple plausible solutions with meaningful uncertainty.
method Discrete latent variables model one-to-many mappings, allowing effective conditional probability estimation.
result Framework outperforms state-of-the-art in uncertainty estimation and is practical.
In this paper, we treat the image generation task using an autoencoder, a representative latent model. Unlike many studies regularizing the latent variable's distribution by assuming a manually specified prior, we approach the image generation task using an autoencoder by directly estimating the latent distribution. To…
This paper shows how to estimate distances in latent space of random graphs using entropic OT.
problem Estimating distances between groups of nodes in latent space of random graphs.
method Entropic Optimal Transport (OT) with stability results for perturbations of the cost matrix.
result Consistent estimation of entropic OT distances between groups of nodes in latent space.
GPIRT uses Gaussian processes to estimate latent traits and IRFs from binary responses.
problem Nonparametric IRT models struggle to estimate flexible IRFs and latent traits simultaneously.
method GPIRT employs Gaussian process priors to relax IRF assumptions while estimating latent traits.
result GPIRT provides a flexible solution to IRT challenges, including active learning.
Proposes MD-LiNA for multi-domain latent factor causal discovery.
problem Discovering causal structures among latent factors from multi-domain data.
method Multi-Domain Linear Non-Gaussian Acyclic Models (MD-LiNA) with an integrated two-phase algorithm.
result Locally consistent estimators of causal structure among shared latent factors.
A novel disentangled graph autoencoder improves treatment effect estimation from networked observational data.
problem Treatment effect estimation from observational data is challenging due to unconfoundedness assumption and latent confounders.
method Proposes a disentangled variational graph autoencoder to disentangle latent factors and enforce factor independence.
result Extensive experiments show superior performance compared to state-of-the-art approaches.
New algorithm improves latent variable model estimation.
problem Estimating parameters in latent variable models.
method Jarzynski-adjusted Langevin algorithm (JALA) for SMC methods.
result JALA-EM provides maximum marginal likelihood estimate.
A new estimator improves training of probabilistic models with latent Gaussian variables.
problem Improving gradient estimation for models with latent Gaussian variables.
method Rao-Blackwellised Reparameterisation Gradients (R2-G2)
result R2-G2 consistently yields better performance in models with multiple applications of the reparameterisation trick.
StepMix estimates mixture models with covariates for social science applications.
problem Estimating latent classes with covariates in social science models.
method Pseudo-likelihood estimation using one-, two-, and three-step approaches.
result Unified framework for expectation-maximization subroutines.
Improved noise estimation in latent neural SDEs enhances model accuracy.
problem Latent neural SDEs underestimate noise, limiting their stochastic dynamics modeling.
method Explicit additional noise regularization in the loss function.
result Model accurately captures diffusion component of stochastic time series data.
Deep equilibrium models estimate latent variables from data.
problem Estimating latent variables from data.
method Generalized exponential family models, deep equilibrium networks.
result Deep equilibrium models solve MAP estimates for latent and transformation parameters.
In data science and machine learning, hierarchical parametric models, such as mixture models, are often used. They contain two kinds of variables: observable variables, which represent the parts of the data that can be directly measured, and latent variables, which represent the underlying processes that generate the d…
Estimates latent dimensionality for prediction tasks using mutual information.
problem Estimating the latent dimensionality needed for accurate prediction.
method Formulates the problem as an Information Bottleneck question and uses neural mutual information estimators with a hybrid critic to preserve latent geometry.
result The hybrid critic method provides a more accurate estimation of task-relevant dimensionality.
Hierarchical probabilistic models, such as Gaussian mixture models, are widely used for unsupervised learning tasks. These models consist of observable and latent variables, which represent the observable data and the underlying data-generation process, respectively. Unsupervised learning tasks, such as cluster analysi…
The paper optimizes hyperplanes for binary classification in high-dimensional data with latent Gaussian mixtures.
problem Binary classification in high-dimensional data with latent Gaussian mixtures.
method Generalized least squares estimator for estimating the direction of the optimal separating hyperplane. Simple correction for intercept estimation.
result The procedure is minimax optimal in many scenarios and can retain the interpolation property.
New method for efficient marginalization of discrete latent variables in neural networks.
problem Computational challenges in training models with discrete latent variables.
method Parameterizing discrete distributions using sparse mappings (sparsemax and structured variants) to reduce support and enable efficient marginalization.
result Achieved good performance in various tasks with efficient and practical training.
We consider learning the possible causal direction of two observed variables in the presence of latent confounding variables. Several existing methods have been shown to consistently estimate causal direction assuming linear or some type of nonlinear relationship and no latent confounders. However, the estimation resul…
Hierarchical parametric models consisting of observable and latent variables are widely used for unsupervised learning tasks. For example, a mixture model is a representative hierarchical model for clustering. From the statistical point of view, the models can be regular or singular due to the distribution of data. In …
Estimates latent norms and Gram matrices for graphs on Euclidean balls.
problem Estimating latent points and their relationships in graphs on Euclidean balls.
method Estimates latent norms and Gram matrices using observed graph data.
result Graphs on Euclidean balls can have power-law degree distributions.
Graphical models are commonly used tools for modeling multivariate random variables. While there exist many convenient multivariate distributions such as Gaussian distribution for continuous data, mixed data with the presence of discrete variables or a combination of both continuous and discrete variables poses new cha…
We decode latent states in Block MDPs and learn near-optimal policies.
problem Model estimation and reward-free learning in Block MDPs.
method Information-theoretical lower bound and efficient model estimation algorithm.
result Our algorithm approaches the information-theoretical limit for latent state decoding and converges to optimal policies.
Deep Discrete Encoders (DDEs) tackle interpretable generative models for rich data with discrete latent layers.
problem Overparametrized, non-identifiable, and uninterpretable deep generative models in high-stakes applications.
method Directed graphical model with multiple binary latent layers, transparent identifiability conditions, scalable estimation pipeline.
result Transparent identifiability conditions and scalable estimation pipeline for interpretable DDEs.
Fractal Flow enhances normalizing flows with interpretable latent space and hierarchical modeling.
problem High-dimensional density estimation and generative modeling challenges.
method Integrates topic modeling (LDA) and fractal strategy into normalizing flows.
result Achieves latent clustering, controllable generation, and superior estimation accuracy.
DisARM improves gradient estimation for binary latent variables.
problem Challenges in training models with discrete latent variables.
method Uses antithetic sampling over continuous augmentation.
result DisARM consistently outperforms ARM and baseline methods in log-likelihood and variance.
Improved robust latent variable estimation for neural dynamics.
problem Inconsistent results due to noise and nonlinearity in existing models.
method Probabilistic approach to latent variable estimation in decomposed models.
result More accurate latent variable inference in nonlinear systems with diverse noise conditions.
Unbiased gradient estimation improves VAE performance.
problem Training VAEs via maximum likelihood is difficult due to intractable integrals.
method Introduced unbiased estimators of the log-likelihood gradient using coupled Markov chains.
result Unbiased estimators lead to better predictive performance in VAEs.
We propose an unsupervised object matching method for relational data, which finds matchings between objects in different relational datasets without correspondence information. For example, the proposed method matches documents in different languages in multi-lingual document-word networks without dictionaries nor ali…
Estimating the individual treatment effect (ITE) from observational data is essential in medicine. A central challenge in estimating the ITE is handling confounders, which are factors that affect both an intervention and its outcome. Most previous work relies on the unconfoundedness assumption, which posits that all th…
New approach for causal inference with interdependent, time-varying latent confounders.
problem Estimating causal effects with interdependent, time-varying latent confounders.
method Variational estimation with a representer theorem and random input space.
result Demonstrates effectiveness on various temporal datasets.
GAME improves matrix completion by considering subgroup-specific latent structures.
problem Heterogeneous data with overlapping categories, smoothing away subgroup-specific variation.
method Group-Aware Matrix Estimation (GAME) with overlapping nuclear-norm penalties.
result GAME outperforms global low-rank estimators in structured missingness regimes.
New method identifies latent variables in cognitive models using neural networks.
problem Inference of latent variables in complex cognitive models is limited.
method Recurrent neural networks and simulation-based inference for latent variable sequences.
result Extends neural Bayes estimation to broader classes of cognitive models.
We consider learning a causal ordering of variables in a linear non-Gaussian acyclic model called LiNGAM. Several existing methods have been shown to consistently estimate a causal ordering assuming that all the model assumptions are correct. But, the estimation results could be distorted if some assumptions actually a…
New method uses latent variables to estimate treatment effects from single-arm trials.
problem Estimating treatment effects from single-arm trials due to lack of external control groups.
method Latent-variable modeling with amortized variational inference for patient matching and direct effect estimation.
result Improved performance in direct treatment effect estimation and effect estimation via patient matching compared to previous methods.
We show a connection between the Fourier spectrum of Boolean functions and the REINFORCE gradient estimator for binary latent variable models. We show that REINFORCE estimates (up to a factor) the degree-1 Fourier coefficients of a Boolean function. Using this connection we offer a new perspective on variance reduction…
Local learning method selects covariates for causal effect estimation in the presence of latent variables.
problem Estimating causal effects from nonexperimental data with latent variables.
method Local learning approach that identifies valid adjustment sets for causal relationships.
result Ensures soundness and completeness of causal effect estimation under standard assumptions.