How to model distribution of sequential data, including but not limited to speech and human motions, is an important ongoing research problem. It has been demonstrated that model capacity can be significantly enhanced by introducing stochastic latent variables in the hidden states of recurrent neural networks. Simultan…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,694 papers · 148 categories
Trend · papers per month
48 results for “Sequential latent-variable models”
We consider the problem of inferring a latent function in a probabilistic model of data. When dependencies of the latent function are specified by a Gaussian process and the data likelihood is complex, efficient computation often involve Markov chain Monte Carlo sampling with limited applicability to large data sets. W…
NAS-X improves inference and model learning for SLVMs.
problem Challenges in analytic inference and model learning for flexible SLVMs.
method NAS-X combines reweighted wake-sleep and smoothing sequential Monte Carlo.
result NAS-X provides low-bias and low-variance gradient estimates.
New method improves generative model performance by fully conditioning variational posteriors.
problem Inaccurate inference due to partial conditioning of variational posteriors in sequential LVMs.
method Introduces fully-conditioned approximate posteriors to improve generative model performance.
result Improves generative modelling and multi-step prediction performance.
A new method for efficient inference in sequential latent-variable models.
problem Computational challenges in integrating subject-specific random effects.
method Anchored variational inference framework to approximate posterior distributions.
result The method achieves accurate estimation with significant computational gains.
This work extends identifiability analysis to sequential latent variable models, focusing on Switching Dynamical Systems.
problem Identifying latent variables in sequential data models.
method Proved identifiability of Markov Switching Models and established conditions for Switching Dynamical Systems.
result Identifiability of latent variables and non-linear mappings in Switching Dynamical Systems up to affine transformations.
New algorithm improves latent variable model estimation.
problem Estimating parameters in latent variable models.
method Jarzynski-adjusted Langevin algorithm (JALA) for SMC methods.
result JALA-EM provides maximum marginal likelihood estimate.
We present a factorized hierarchical variational autoencoder, which learns disentangled and interpretable representations from sequential data without supervision. Specifically, we exploit the multi-scale nature of information in sequential data by formulating it explicitly within a factorized hierarchical graphical mo…
New method selects features for sequential decision making.
problem Dynamic feature selection for instance-wise decisions.
method Latent variable model trained in a supervised manner; reasoning across stochastic latent space.
result Outperforms existing methods on various datasets.
Many efforts have been devoted to training generative latent variable models with autoregressive decoders, such as recurrent neural networks (RNN). Stochastic recurrent models have been successful in capturing the variability observed in natural sequential data such as speech. We unify successful ideas from recently pr…
Many real-world engineering problems rely on human preferences to guide their design and optimization. We present PrefOpt, an open source package to simplify sequential optimization tasks that incorporate human preference feedback. Our approach extends an existing latent variable model for binary preferences to allow f…
Hidden Markov Models (HMMs) comprise a powerful generative approach for modeling sequential data and time-series in general. However, the commonly employed assumption of the dependence of the current time frame to a single or multiple immediately preceding frames is unrealistic; more complicated dynamics potentially ex…
We propose a general formalism of iterated random functions with semigroup property, under which exact and approximate Bayesian posterior updates can be viewed as specific instances. A convergence theory for iterated random functions is presented. As an application of the general theory we analyze convergence behaviors…
Online method for state estimation and parameter learning in SSMs.
problem State estimation and parameter learning in state-space models.
method Stochastic gradient optimization of variational lower bound, using backward decompositions and Bellman recursions.
result Ability to operate online without revisiting historic observations.
Optimal recommendation system using user and item clustering.
problem Maximizing recommendation accuracy with limited feedback.
method Latent variable model with user and item clustering, exploiting i.i.d. structure.
result Near-optimal algorithm that combines item and user structures.
Improved SVAE models enhance sequential data prediction.
problem Challenges in implementing and using structured variational autoencoders.
method Modern machine learning tools, hardware acceleration, parallelization, automatic differentiation, exploiting structure in the prior.
result SVAEs outperform general alternatives in accuracy and efficiency.
This paper improves level generation using VAEs for coherent, logically following segments.
problem Generating coherent levels of non-fixed length and blending levels from different games.
method Sequential segment-based level generation using VAEs with a classifier for logical placement.
result Generated levels are more coherent and capable of blending levels from different games.
A new model captures variability in time series data.
problem Capturing high variability in time series data.
method Temporal latent variables and dynamic weight modifications.
result Demonstrated efficacy on various sequential data.
Paper improves VAEs using Monte Carlo methods.
problem Improving the Evidence Lower Bound (ELBO) for VAEs.
method Uses Monte Carlo techniques to improve ELBO, specifically Sequential Importance Sampling (SIS) with carefully chosen kernels.
result Demonstrates improved performance on various applications.
We improve a graph generation model to accurately recover Barabási-Albert graph parameters.
problem Recover Barabási-Albert graph parameters from graph data.
method Use a disentanglement-focused deep autoencoding framework with a sequential LSTM decoder trained on graph data.
result Successfully recover Barabási-Albert graph parameters.
GFlowNet-EM learns complex latent variable models with discrete structures.
problem Challenges in modeling posteriors over discrete compositional latents with expectation-maximization.
method Uses GFlowNets to learn stochastic policies for sampling from complex posterior distributions.
result GFlowNet-EM enables training expressive LVMs with discrete compositional latents.
We introduce a new approach for amortizing inference in directed graphical models by learning heuristic approximations to stochastic inverses, designed specifically for use as proposal distributions in sequential Monte Carlo methods. We describe a procedure for constructing and learning a structured neural network whic…
Due to the phenomenon of "posterior collapse," current latent variable generative models pose a challenging design choice that either weakens the capacity of the decoder or requires augmenting the objective so it does not only maximize the likelihood of the data. In this paper, we propose an alternative that utilizes t…
Proposes LDIDPs for efficient sequential data generation from latent dynamical models.
problem Challenges in generating high-fidelity sequential samples from latent dynamical models.
method Utilizes implicit diffusion processes to sample from latent dynamical processes.
result Demonstrates accurate learning of dynamics and efficient generation of high-quality sequential data.
Paper proposes semi-supervised learning with triplet Markov chains.
problem Lack of labels in training data.
method Variational Bayesian inference for semi-supervised learning.
result Derives semi-supervised algorithms for various sequential models.
We propose the factorized action variational autoencoder (FAVAE), a state-of-the-art generative model for learning disentangled and interpretable representations from sequential data via the information bottleneck without supervision. The purpose of disentangled representation learning is to obtain interpretable and tr…
VQ-DRAW compresses images and generates realistic samples.
problem Learning compact discrete representations of images.
method Sequential discrete VAE with vector quantization.
result VQ-DRAW effectively compresses and generates images.
New method disentangles latent variables in nonstationary data.
problem Disentangling latent variables in nonstationary sequential data.
method NCTRL framework exploiting Markov assumption and temporal structure.
result Independent latent components can be recovered from nonlinear mixture without auxiliary variables.
We propose a penalized orthogonal-components regression (POCRE) for large p small n data. Orthogonal components are sequentially constructed to maximize, upon standardization, their correlation to the response residuals. A new penalization framework, implemented via empirical Bayes thresholding, is presented to effecti…
With latent variables, stochastic recurrent models have achieved state-of-the-art performance in modeling sound-wave sequence. However, opposite results are also observed in other domains, where standard recurrent networks often outperform stochastic models. To better understand this discrepancy, we re-examine the role…
This paper tackles sequential distribution shifts in representation learning.
problem Learning meaningful representations in a sequence of distribution shifts.
method Nonlinear Independent Component Analysis (ICA) framework for continual causal representation learning.
result The method achieves performance comparable to joint training on multiple offline distributions and shows no benefit from the incoming new distribution on all latent variables.
New method estimates treatment effects over time with unobserved confounders.
problem Estimating treatment effects from observational data with unobserved confounders.
method Sequential Deconfounder using Gaussian process latent variable model.
result Unbiased estimates of individualized treatment responses over time.
When used as a surrogate objective for maximum likelihood estimation in latent variable models, the evidence lower bound (ELBO) produces state-of-the-art results. Inspired by this, we consider the extension of the ELBO to a family of lower bounds defined by a particle filter's estimator of the marginal likelihood, the …
Improved variational inference for GPLVMs using AIS.
problem Challenges in generating effective proposal distributions for high-dimensional or complex data.
method Annealed Importance Sampling (AIS) combined with reparameterization.
result Our method achieves tighter variational bounds and higher log-likelihoods.
This paper discovers classification models from sequential data without prior knowledge.
problem Lack of prior knowledge in defining kernels for online classification.
method Adapts GP-based time-series structure discovery with SMC to learn new features from sequential data.
result Improves classification accuracy by 10% on real-world data.
Learning a model of dynamics from high-dimensional images can be a core ingredient for success in many applications across different domains, especially in sequential decision making. However, currently prevailing methods based on latent-variable models are limited to working with low resolution images only. In this wo…
Generative models of graph structure have applications in biology and social sciences. The state of the art is GraphRNN, which decomposes the graph generation process into a series of sequential steps. While effective for modest sizes, it loses its permutation invariance for larger graphs. Instead, we present a permuta…
Improves Bayesian optimisation for engineering design problems with many variables.
problem Efficiently searching for global minima in high-dimensional design spaces.
method Integrates input and output data to identify a reduced latent subspace using probabilistic partial least squares.
result Significant improvements in convergence to the global minimum compared to existing methods.
A new method for training generative models with sparse supervision.
problem Training deep generative models with sparse and varying supervision.
method Caffeinated Wake-Sleep (CWS) method, combining reweighted wake-sleep and teacher-forcing.
result The CWS method is robust to variable length supervision and performs well on various datasets.
Computing the marginal likelihood (ML) of a model requires marginalizing out all of the parameters and latent variables, a difficult high-dimensional summation or integration problem. To make matters worse, it is often hard to measure the accuracy of one's ML estimates. We present bidirectional Monte Carlo, a technique…
New framework TDRL identifies latent causal variables from sequential data.
problem Identify latent causal variables from sequential data.
method Proposes TDRL framework to recover time-delayed latent causal variables and identify their relations from measured sequential data.
result Identifies latent causal variables reliably from sequential data.
We present Sequential Attend, Infer, Repeat (SQAIR), an interpretable deep generative model for videos of moving objects. It can reliably discover and track objects throughout the sequence of frames, and can also generate future frames conditioning on the current frame, thereby simulating expected motion of objects. Th…
Diverse and accurate vision+language modeling is an important goal to retain creative freedom and maintain user engagement. However, adequately capturing the intricacies of diversity in language models is challenging. Recent works commonly resort to latent variable models augmented with more or less supervision from ob…
Generative latent-variable models are emerging as promising tools in robotics and reinforcement learning. Yet, even though tasks in these domains typically involve distinct objects, most state-of-the-art generative models do not explicitly capture the compositional nature of visual scenes. Two recent exceptions, MONet …
Introduces alternators for modeling sequences, outperforming baselines.
problem Modeling complex sequential data with stability and efficiency.
method Two neural networks (OTN and FTN) alternate between outputting samples in observation and feature spaces, learned via cross-entropy criterion.
result Alternators outperform strong baselines in various domains (Lorenz equations, Neuroscience, Climate Science).
Adaptive querying learns user psychometrics with AI personas.
problem Learning user psychometrics within query budgets.
method Persona-induced latent variable model with AI personas and large language model response distributions.
result Persona-based posteriors deliver accurate probabilistic predictions.
Many kinds of data can be represented as a network or graph. It is crucial to infer the latent structure underlying such a network and to predict unobserved links in the network. Mixed Membership Stochastic Blockmodel (MMSB) is a promising model for network data. Latent variables and unknown parameters in MMSB have bee…
LADD models improve discrete diffusion for faster language generation.
problem Practical discrete diffusion models ignore cross-token dependencies, degrading performance.
method Introduces a learnable auxiliary latent channel, diffusing over the joint (token, latent) space.
result LADD models yield improvements on unconditional generation metrics.