Paper develops a classification method using matrix-variate t-distributions.
problem Classifying matrix-valued observations with dependence structure.
method Develops an Expectation-Maximization algorithm for discriminant analysis.
result Method shows promise on various datasets.
We present a family of expectation-maximization (EM) algorithms for binary and negative-binomial logistic regression, drawing a sharp connection with the variational-Bayes algorithm of Jaakkola and Jordan (2000). Indeed, our results allow a version of this variational-Bayes approach to be re-interpreted as a true EM al…
A new EM-based algorithm improves deep generative model training.
problem Training deep generative models with maximum likelihood is challenging.
method The paper proposes reweighted expectation maximization (REM), a new algorithm that directly maximizes the log marginal likelihood of the data.
result REM learns better generative models than the IWAE, leading to significantly better performance in density estimation benchmarks.
In this paper, we provide an information-theoretic interpretation of the Vector Quantized-Variational Autoencoder (VQ-VAE). We show that the loss function of the original VQ-VAE can be derived from the variational deterministic information bottleneck (VDIB) principle. On the other hand, the VQ-VAE trained by the Expect…
This paper bridges statistical and machine learning approaches to variational inference.
problem Statisticians struggle to understand variational inference from a Frequentist perspective.
method Explains VI, VAEs, and DDMs from a Frequentist viewpoint, starting with EM.
result VI emerges as a scalable solution for intractable E-steps in VAEs and DDMs.
We derive a novel variational expectation maximization approach based on truncated posterior distributions. Truncated distributions are proportional to exact posteriors within subsets of a discrete state space and equal zero otherwise. The treatment of the distributions' subsets as variational parameters distinguishes …
Adaptive learning method identifies and corrects corrupted data.
problem Robust learning from corrupted training sets.
method Identifies corrupted and non-corrupted samples with latent Bernoulli variables, formulates as likelihood maximization with marginalized latent variables, solved via variational inference and Expectation-Maximization.
result Improves over state-of-the-art by automatically inferring corruption level with minimal overhead.
New EM algorithm improves deep generative network training.
problem Training deep generative networks with complex posterior and likelihood distributions.
method Derive analytical posterior and marginal distributions using CPA property, derive analytical EM algorithm.
result EM training yields higher likelihood than Variational Autoencoders (VAEs).
A new GAN model LDAGAN uses Latent Dirichlet Allocation to model multimodal images.
problem Ignoring the structure and multimodal characteristics of vision data in GANs.
method Introduced a Dirichlet prior for multimodal image generation leading to LDAGAN. LDAGAN defines generative modes for each sample and uses a VEM algorithm for adversarial training.
result Experimental results show LDAGAN outperforms other GANs on real-world datasets.
QEM uses parallel importance weighting for fast approximate Bayesian inference.
problem Bayesian inference challenges in large models with many observations and latent variables.
method Expectation Maximization (EM) with massively parallel importance weighting.
result QEM is faster and more scalable than RWS and VI.
GFlowNet-EM learns complex latent variable models with discrete structures.
problem Challenges in modeling posteriors over discrete compositional latents with expectation-maximization.
method Uses GFlowNets to learn stochastic policies for sampling from complex posterior distributions.
result GFlowNet-EM enables training expressive LVMs with discrete compositional latents.
A new Bayesian model improves dynamic texture segmentation.
problem Automatic selection of DTs in video sequences.
method Joint Dirichlet process mixture and GDTM approach with VBEM and RTSS.
result The proposed algorithm outperforms previous methods in efficiency and accuracy.
New neural method for inferring Markov jump processes.
problem Inference in Markov jump processes is challenging.
method Variational inference using neural ODEs and backpropagation.
result Trains neural representations of data to approximate process rates.
Integrates VAEs into EM for deep clustering and generation.
problem Clustering and generating new samples from complex distributions.
method Combines VAEs and EM, updating model parameters and refining cluster assignments.
result Superior clustering performance on MNIST and FashionMNIST.
A scalable topic model for large document collections using MapReduce.
problem Scalability issues in topic modeling for large document collections.
method Correlated Topic Model with variational Expectation-Maximization in MapReduce framework.
result Comparable topic coherences with LDA in MapReduce framework.
Speech enhancement improved using variational autoencoders and alpha-stable noise models.
problem Improving speech quality in noisy environments.
method Deep generative model using variational autoencoders with alpha-stable noise model.
result The proposed method outperforms conventional approaches in speech intelligibility and quality.
In this paper, we consider the problem of low-rank phase retrieval whose objective is to estimate a complex low-rank matrix from magnitude-only measurements. We propose a hierarchical prior model for low-rank phase retrieval, in which a Gaussian-Wishart hierarchical prior is placed on the underlying low-rank matrix to …
MSFA clusters high-dimensional spatial data using spline-based covariance structures.
problem Clustering high-dimensional spatial data with flexible covariance structures.
method Mixture of spatial factor analyzers with spline-based covariance and matrix variate factor analyzers for dimensionality reduction.
result Proposed models accurately infer and differentiate distinct spatial patterns in tensor-variate data.
In this paper we address speaker-independent multichannel speech enhancement in unknown noisy environments. Our work is based on a well-established multichannel local Gaussian modeling framework. We propose to use a neural network for modeling the speech spectro-temporal content. The parameters of this supervised model…
A new MFG framework for evolving clusters from Gaussian mixtures.
problem Evolutionary clustering of time-dependent Gaussian mixtures.
method Control-theoretic framework based on Mean Field Games (MFG) with coupled HJB and Fokker-Planck systems.
result MFG dynamics recover classical EM algorithm trajectories with mass conservation.
A new speech enhancement method using variational autoencoders.
problem Improving speech quality in noisy environments.
method Using a variational autoencoder as a speech model, trained with unsupervised noise modeling.
result The method outperforms existing techniques in speech enhancement.
Paper improves tree probability estimation using stochastic optimization and variance reduction.
problem Improving tree probability estimation in phylogenetic inference.
method Introduces computationally efficient methods for training SBNs and variance reduction for optimization.
result Methods outperform previous baseline methods in tree topology probability estimation and Bayesian phylogenetic inference.
In this paper, we firstly give a brief introduction of expectation maximization (EM) algorithm, and then discuss the initial value sensitivity of expectation maximization algorithm. Subsequently, we give a short proof of EM's convergence. Then, we implement experiments with the expectation maximization algorithm (We im…
New algorithm for continuous-time switching systems using variational inference.
problem Inference in time-series data with continuous-time switching systems.
method Developed a variational inference algorithm combining Gaussian process approximation and posterior inference for Markov jump processes.
result Bayesian latent state estimates and point estimates of unknown parameters for arbitrary points on the real axis.
This paper tackles unsupervised speech enhancement using RVAE and proposes efficient sampling methods.
problem Unsupervised speech enhancement with high computational complexity.
method Recurrent variational autoencoder (RVAE) combined with Langevin dynamics and Metropolis-Hasting sampling.
result Sampling-based algorithms outperform VEM and achieve robust generalization.
A new model analyzes document structure and customer shopping patterns.
problem Understanding document structure and customer shopping patterns.
method Variational EM algorithm for multilayer correlated topic modeling.
result MCTM successfully captures document structure and customer shopping patterns.
Develops a new variational estimator for node popularity in bipartite networks.
problem Estimating node popularity in bipartite networks with varying patterns.
method Variational Expectation-Maximization (VEM) framework for the Two-Way Node Popularity Model (TNPM).
result The proposed method achieves superior estimation accuracy across different types of networks.
Survey of factor analysis, PCA, variational inference, and VAE.
problem Dimensionality reduction and generative modeling of data.
method Variational inference, factor analysis, probabilistic PCA, and VAE.
result Derivation and explanation of ELBO, EM, and closed-form solutions.
This paper models time-series data with a mixture of Markov chains, automatically determining the number of components.
problem Tackles the inability of common Markov state modeling frameworks to discern heterogeneities in complex data.
method Uses a mixture of Markov chains and variational expectation-maximization algorithm for automatic component selection.
result Achieves performance consistent with theoretically optimal error scaling, identifying meaningful heterogeneities in various data sets.
ProJIVE integrates multiple data types to explain joint and individual variation.
problem Integrating multiple types of data on the same subjects.
method Probabilistic EM algorithm for JIVE framework.
result ProJIVE learns biologically meaningful courses of variation and improves accuracy.
We face network data from various sources, such as protein interactions and online social networks. A critical problem is to model network interactions and identify latent groups of network nodes. This problem is challenging due to many reasons. For example, the network nodes are interdependent instead of independent o…
Extracting common narratives from multi-author dynamic text corpora requires complex models, such as the Dynamic Author Persona (DAP) topic model. However, such models are complex and can struggle to scale to large corpora, often because of challenging non-conjugate terms. To overcome such challenges, in this paper we …
Efficiently tunes hyperparameters for knowledge graph embeddings.
problem Hyperparameter tuning is challenging and costly for knowledge graph embeddings.
method Augmented probabilistic framework with variational expectation-maximization.
result Achieved state-of-the-art performance in link prediction.
Paper introduces TSSDMN for modeling dynamic multilayer networks.
problem Capturing temporal and cross-layer dynamics in multilayer networks.
method Tensor State Space Model (TSSDMN) using symmetric Tucker decomposition.
result TSSDMN uniquely captures temporal dynamics within and across layers.
Hidden Markov Models (HMM) have been used for several years in many time series analysis or pattern recognitions tasks. HMM are often trained by means of the Baum-Welch algorithm which can be seen as a special variant of an expectation maximization (EM) algorithm. Second-order training techniques such as Variational Ba…
A new 1-iteration GMM learning algorithm improves robustness and accuracy.
problem Improving robustness and accuracy in Gaussian Mixture Model learning.
method GMM expansion idea, 1-iteration learning algorithm, theoretical proof of convergence.
result Guaranteed convergence of the new algorithm regardless initial parameters.
Quantum EM algorithm improves clustering for Gaussian mixtures.
problem Improving clustering efficiency for Gaussian mixture models.
method Quantum expectation-maximization algorithm for Gaussian mixture models.
result Quantum EM algorithm demonstrates robustness and speedup.
Paper proposes variational inference for piecewise-linear systems.
problem Intractability of switching dynamical systems.
method Variational approximation and expectation-maximization algorithms.
result Parameters can be estimated off-line, including the number of linear modes.
Analytical method approximates ELBO gradient in clutter problem.
problem Clutter problem in Bayesian networks with Gaussian likelihood.
method Reparameterization trick, local approximation of likelihood factors.
result Good accuracy and linear computational complexity compared to classical methods.
EDAs are shown to be a type of EM algorithm.
problem Understanding and improving Estimation of Distribution Algorithms.
method Connecting EDAs to Expectation-Maximization (EM) framework.
result EDAs can be understood as a special case of EM.
Proposes a Latent Block Model for analyzing missing data.
problem Missing data can lead to misleading conclusions if not properly accounted for.
method Co-clustering model based on Latent Block Model, with variational EM algorithm for inference and model selection criterion.
result The proposed model reveals meaningful groups and insights from non-voters in French Parliament voting records.
Improved model for grouping nodes in bipartite networks.
problem Challenges in grouping nodes in bipartite graphs.
method Introduced DC-LBM and developed variational EM algorithm.
result Significantly enhanced performance on real-world data.
HARMLESS meta-learning method models short event sequences with relational information.
problem Learning heterogeneous point process models from short event sequence data.
method Hierarchical Bayesian mixture Hawkes process model with stochastic variational meta expectation maximization.
result HARMLESS outperforms existing methods in predicting future events.
We present a noise-injected version of the Expectation-Maximization (EM) algorithm: the Noisy Expectation Maximization (NEM) algorithm. The NEM algorithm uses noise to speed up the convergence of the EM algorithm. The NEM theorem shows that injected noise speeds up the average convergence of the EM algorithm to a local…
Object tracking is an ubiquitous problem that appears in many applications such as remote sensing, audio processing, computer vision, human-machine interfaces, human-robot interaction, etc. Although thoroughly investigated in computer vision, tracking a time-varying number of persons remains a challenging open problem.…
Empirical study shows overparameterization benefits unsupervised learning of latent variable models.
problem Improving optimization landscape in unsupervised learning with overparameterization.
method Synthetic and semi-synthetic experiments with various models and training algorithms.
result Overparameterization significantly increases the number of ground truth latent variables recovered.
The detection of rare variants is important for understanding the genetic heterogeneity in mixed samples. Recently, next-generation sequencing (NGS) technologies have enabled the identification of single nucleotide variants (SNVs) in mixed samples with high resolution. Yet, the noise inherent in the biological processe…
Maximum likelihood estimation (MLE) is one of the most important methods in machine learning, and the expectation-maximization (EM) algorithm is often used to obtain maximum likelihood estimates. However, EM heavily depends on initial configurations and fails to find the global optimum. On the other hand, in the field …