Proposes a Varying-Coefficient MoE model for analyzing dynamic data.
problem Inadequate constant coefficients in MoE models for dynamic settings.
method Varying-Coefficient Mixture of Experts (VCMoE) model with varying coefficients in gating and expert models.
result Established identifiability and consistency of the VCMoE model.
Improved model for grouping nodes in bipartite networks.
problem Challenges in grouping nodes in bipartite graphs.
method Introduced DC-LBM and developed variational EM algorithm.
result Significantly enhanced performance on real-world data.
Efficiently creates label-consistent backdoor attacks without obvious mislabeling.
problem Vulnerability of deep neural networks to backdoor attacks that can be triggered by a backdoor trigger.
method Uses adversarial perturbations and generative models to inject inputs that are consistent with their labels.
result Demonstrates the feasibility of creating undetectable backdoor attacks by maintaining label-consistency.
LC-GNN improves GNNs for node classification by incorporating label consistency.
problem Limited performance of GNNs due to label consistency assumption not always holding.
method LC-GNN uses node pairs with the same label but unconnected to expand GNN's receptive field.
result LC-GNN outperforms traditional GNNs in semi-supervised node classification.
Deep Convolutional Neural Networks (CNN) enforces supervised information only at the output layer, and hidden layers are trained by back propagating the prediction error from the output layer without explicit supervision. We propose a supervised feature learning approach, Label Consistent Neural Network, which enforces…
Develops a new variational estimator for node popularity in bipartite networks.
problem Estimating node popularity in bipartite networks with varying patterns.
method Variational Expectation-Maximization (VEM) framework for the Two-Way Node Popularity Model (TNPM).
result The proposed method achieves superior estimation accuracy across different types of networks.
EXoN creates an explainable latent space for semi-supervised learning.
problem Creating an explainable latent space for semi-supervised learning.
method EXoN combines VAE with SCI (Soft-label Consistency Interpolation) to create an explainable latent space.
result EXoN reduces the cost of investigating representation patterns on the latent space.
TIMELY improves consistency in labeling blood cell images.
problem Inconsistent labeling of blood cells in microscopy images leads to unreliable diagnoses.
method TIMELY combines pseudotime inference and hidden Markov trees to correct labeling mistakes.
result TIMELY outperforms baseline methods in identifying and correcting inconsistent labels.
Bayesian networks (BN) are used in a big range of applications but they have one issue concerning parameter learning. In real application, training data are always incomplete or some nodes are hidden. To deal with this problem many learning parameter algorithms are suggested foreground EM, Gibbs sampling and RBE algori…
The EM algorithm is one of many important tools in the field of statistics. While often used for imputing missing data, its widespread applications include other common statistical tasks, such as clustering. In clustering, the EM algorithm assumes a parametric distribution for the clusters, whose parameters are estimat…
Proposes a new deep ordinal classification model enforcing label uni-modality.
problem Deep ordinal classification with label consistency issues.
method Non-parametric uni-modality constraints via inequality constraints.
result Improves scalability and performance in ordinal classification tasks.
Generalising the idea of the classical EM algorithm that is widely used for computing maximum likelihood estimates, we propose an EM-Control (EM-C) algorithm for solving multi-period finite time horizon stochastic control problems. The new algorithm sequentially updates the control policies in each time period using Mo…
A new EM gradient algorithm for mixture models with skewed components.
problem Fitting mixture models with skewed components derived from the Manly transformation.
method Proposes an alternative EM gradient algorithm using Newton's method for better parameter updates.
result Shows improved convergence and parameter estimation compared to the Nelder-Mead optimization.
Paper proposes a faster SPIDER-EM variant for large-scale nonconvex optimization.
problem High computational cost of EM algorithm in large-scale learning.
method Extension of SPIDER-EM for nonconvex finite-sum optimization problems.
result Achieves state-of-the-art complexity bounds and linear convergence under certain conditions.
MLE and CVE are equivalent under exponential families, leading to faster and more stable EM algorithms.
problem Finding maximum likelihood estimators (MLE) efficiently and stably.
method Proved equivalence between MLE and CVE under exponential families, leading to an EM algorithm.
result EM algorithm achieves the same asymptotic variance as MLE and is faster and more stable.
sEM uses optimal transport to improve EM algorithm for better convergence and avoiding local optima.
problem Improving the EM algorithm for better convergence and avoiding local optima.
method sEM uses entropic optimal transport to compute responsibilities in the expectation step, leading to better global convergence guarantees and avoiding local optima.
result sEM learns cell labels significantly better than other approaches, improving convergence and avoiding local optima.
The speed of convergence of the Expectation Maximization (EM) algorithm for Gaussian mixture model fitting is known to be dependent on the amount of overlap among the mixture components. In this paper, we study the impact of mixing coefficients on the convergence of EM. We show that when the mixture components exhibit …
Paper refutes EM convergence theory and introduces a new EM algorithm.
problem The convergence theory of the EM algorithm is incorrect and affects its performance.
method Proposes a new EM algorithm called the Channel Matching (CM) EM algorithm and provides an initialization map.
result The locally maximal Q can affect the convergent speed but not the global convergence.
New framework improves EM algorithm convergence under log-Sobolev inequality.
problem Improving convergence of the EM algorithm.
method Extending gradient flow techniques to EM algorithm, using free energy representation.
result Exponential convergence of EM algorithm under log-Sobolev inequality.
Paper shows DMS as an EM algorithm with improved convergence.
problem Improving the convergence of DMS algorithm.
method Shows DMS as a generalized EM algorithm and provides new proofs.
result Demonstrates global convergence and linear convergence of DMS.
This paper analyzes and improves EM algorithms for large datasets.
problem Scaling EM algorithm to large datasets.
method Incremental and stochastic EM algorithms, variance reduction.
result Established non-asymptotic convergence bounds for global convergence.
EM algorithm converges in KL divergence for exponential families via mirror descent.
problem Lack of understanding of EM's non-asymptotic convergence properties.
method Viewing EM as a mirror descent algorithm, showing convergence rates in KL divergence.
result KL divergence rates for EM in exponential families, invariant to parametrization.
Introduces geometric formulation of EM algorithm for robust inference and various applications.
problem Statistical inference with missing data or unobservables.
method Information geometric formulation of EM algorithm and its extensions.
result Outlier-robust inference algorithm and various applications in deep learning.
The EM algorithm is a novel numerical method to obtain maximum likelihood estimates and is often used for practical calculations. However, many of maximum likelihood estimation problems are nonconvex, and it is known that the EM algorithm fails to give the optimal estimate by being trapped by local optima. In order to …
A fast Modal EM algorithm for Gaussian mixtures.
problem Clustering with Gaussian mixtures.
method Modal EM algorithm for Gaussian mixtures.
result High flexibility in various clustering contexts.
This paper develops a federated EM algorithm for unsupervised learning of mixture models.
problem Theoretical foundations of unsupervised federated learning are lacking.
method Introduces a federated gradient EM algorithm (FedGrEM) for unsupervised learning of mixture models.
result Theoretical analysis shows FedGrEM outperforms local single-task learning.
Regularized EM algorithm improves clustering performance with small sample sizes.
problem Performance reduction in EM algorithm due to small sample size and poorly conditioned covariance matrices.
method Regularized EM algorithm that uses prior knowledge to ensure positive definiteness of covariance matrices.
result The regularized EM algorithm outperforms standard EM in clustering tasks with small sample sizes.
New algorithm for estimating MLR parameters with non-Gaussian noise.
problem Estimating MLR parameters with non-Gaussian noise.
method Combining ADMM with EM algorithm idea.
result Our method outperforms EM algorithm in non-Gaussian noise case.
Improves EM algorithm for better local optima in mixture models.
problem EM algorithm's sensitivity to initialization and bad local optima.
method Big Learning principle applied to upgrade EM algorithm.
result BigLearn-EM delivers optimal solution with high probability.
Regularized EM algorithm improves GMM clustering in low sample settings.
problem Numerical instability and convergence issues in EM-GMM for low sample support.
method Regularized EM algorithm that maximizes penalized GMM likelihood, ensuring positive definiteness and structured covariance matrices.
result The regularized EM algorithm leads to better performing EM for structured covariance matrix models or low sample settings.
In this paper, we propose a dynamical systems perspective of the Expectation-Maximization (EM) algorithm. More precisely, we can analyze the EM algorithm as a nonlinear state-space dynamical system. The EM algorithm is widely adopted for data clustering and density estimation in statistics, control systems, and machine…
We develop a general framework for proving rigorous guarantees on the performance of the EM algorithm and a variant known as gradient EM. Our analysis is divided into two parts: a treatment of these algorithms at the population level (in the limit of infinite data), followed by results that apply to updates based on a …
EM algorithm achieves optimal sample complexity for learning two-component mixed linear regression.
problem Learning two-component mixed linear regression under varying signal-to-noise ratios.
method Analysis of EM algorithm convergence rates under different SNR regimes.
result EM algorithm achieves minimax optimal sample complexity in all SNR regimes.
EM algorithm converges to global max in latent Gaussian tree models.
problem Optimizing log-likelihood in latent Gaussian tree models.
method Analyzed the optimization landscape and convergence of EM algorithm.
result EM algorithm converges to global maximum in latent Gaussian tree models.
Paper analyzes EM algorithm's trajectory in 2MLR, revealing cycloid behavior.
problem Understanding the convergence and trajectory of EM algorithm in 2MLR.
method Explicit closed-form expressions for EM updates, recurrence relation derivation at population level.
result EM iterations lie on a cycloid trajectory, leading to theoretical estimate of convergence exponent.
We present the particle stochastic approximation EM (PSAEM) algorithm for learning of dynamical systems. The method builds on the EM algorithm, an iterative procedure for maximum likelihood inference in latent variable models. By combining stochastic approximation EM and particle Gibbs with ancestor sampling (PGAS), PS…
The iterative nature of the expectation maximization (EM) algorithm presents a challenge for privacy-preserving estimation, as each iteration increases the amount of noise needed. We propose a practical private EM algorithm that overcomes this challenge using two innovations: (1) a novel moment perturbation formulation…
Proposes an EM algorithm for high-dimensional Markov-switching VAR models.
problem Estimating regime shifts in high-dimensional time series data.
method Approximate EM algorithm for Markov-switching VAR models.
result Established consistency of the proposed EM algorithm in high dimensions.
Strongly polynomial algorithm for approximate Forster transforms and halfspace learning.
problem Computing approximate Forster transforms and halfspace learning.
method Strongly polynomial time algorithm for approximate Forster transforms and halfspace learning.
result First strongly polynomial time algorithm for distribution-free PAC learning of halfspaces.
EM algorithm converges linearly and achieves sharp rate in estimating mixtures of pairwise differences.
problem Estimating mixtures of pairwise differences from noisy data.
method Sharp analysis of the EM algorithm locally around the ground truth.
result The EM sequence converges linearly with an ℓ∞-norm guarantee on the estimation error and achieves the sharp rate of estimation in the ℓ2-norm. EDML is a recently proposed algorithm for learning MAP parameters in Bayesian networks. In this paper, we present a number of new advances and insights on the EDML algorithm. First, we provide the multivalued extension of EDML, originally proposed for Bayesian networks over binary variables. Next, we identify a simplif…
In this paper, we firstly give a brief introduction of expectation maximization (EM) algorithm, and then discuss the initial value sensitivity of expectation maximization algorithm. Subsequently, we give a short proof of EM's convergence. Then, we implement experiments with the expectation maximization algorithm (We im…
The present work proposes hybridization of Expectation-Maximization (EM) and K-Means techniques as an attempt to speed-up the clustering process. Though both K-Means and EM techniques look into different areas, K-means can be viewed as an approximate way to obtain maximum likelihood estimates for the means. Along with …
Develops a high-dimensional differentially-private EM algorithm with near-optimal statistical guarantees.
problem Designing differentially-private EM algorithms for high-dimensional latent variable models.
method Noisy iterative hard-thresholding, statistical guarantees, near-optimal convergence rates.
result Near-optimal statistical guarantees and minimax rate optimality in high-dimensional settings.
The Laplace approximation calls for the computation of second derivatives at the likelihood maximum. When the maximum is found by the EM-algorithm, there is a convenient way to compute these derivatives. The likelihood gradient can be obtained from the EM-auxiliary, while the Hessian can be obtained from this gradient …
Expectation Maximization (EM) is among the most popular algorithms for estimating parameters of statistical models. However, EM, which is an iterative algorithm based on the maximum likelihood principle, is generally only guaranteed to find stationary points of the likelihood objective, and these points may be far from…
This paper uses dynamical systems to analyze and ensure convergence of the Bayesian EM algorithm.
problem Ensuring convergence of the Bayesian EM algorithm in incomplete-data scenarios.
method Applying Lyapunov stability theory to discrete-time dynamical systems.
result Conditions for convergence and potential for fast convergence of MAP-EM are established.
Paper introduces deterministic EM approximations for non-convex likelihood functions.
problem Deterministic approximations for the E-step of EM algorithm are lacking.
method Developed a theoretical framework for deterministic approximations, analyzed Riemann sums and tempered EM.
result Proved convergence guarantees for deterministic approximations and new non-trivial temperature profiles.