A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Gradient EM converges globally for over-parameterized Gaussian mixtures.
A new EM gradient algorithm for mixture models with skewed components.
Gradient EM converges globally for over-parameterized Gaussian mixtures.
We develop a general framework for proving rigorous guarantees on the performance of the EM algorithm and a variant known as gradient EM. Our analysis is divided into two parts: a treatment of these algorithms at the population level (in the limit of infinite data), followed by results that apply to updates based on a …
Gradient descent on LSE objectives implicitly performs EM, leading to collapse without volume control.
The Laplace approximation calls for the computation of second derivatives at the likelihood maximum. When the maximum is found by the EM-algorithm, there is a convenient way to compute these derivatives. The likelihood gradient can be obtained from the EM-auxiliary, while the Hessian can be obtained from this gradient …
Gradient EM converges exponentially to optimal solution in agnostic mixtures.
EM algorithm converges in KL divergence for exponential families via mirror descent.
Gradient-EM Bayesian meta-learning accelerates adaptation with reduced computation and improved robustness.
New framework improves EM algorithm convergence under log-Sobolev inequality.
New DP EM algorithm with statistical guarantees for mixture models.
Enhances large language models' reasoning through simpler off-policy reinforcement learning.
We consider maximum likelihood estimation for Gaussian Mixture Models (Gmms). This task is almost invariably solved (in theory and practice) via the Expectation Maximization (EM) algorithm. EM owes its success to various factors, of which is its ability to fulfill positive definiteness constraints in closed form is of …
We study stochastic gradient descent {\em without replacement} (\sgdwor) for smooth convex functions. \sgdwor is widely observed to converge faster than true \sgd where each sample is drawn independently {\em with replacement} \cite{bottou2009curiously} and hence, is more popular in practice. But it's convergence prope…
Various bias-correction methods such as EXTRA, gradient tracking methods, and exact diffusion have been proposed recently to solve distributed {\em deterministic} optimization problems. These methods employ constant step-sizes and converge linearly to the {\em exact} solution under proper conditions. However, their per…
The speed of convergence of the Expectation Maximization (EM) algorithm for Gaussian mixture model fitting is known to be dependent on the amount of overlap among the mixture components. In this paper, we study the impact of mixing coefficients on the convergence of EM. We show that when the mixture components exhibit …
The expectation-maximization (EM) algorithm has been widely used in minimizing the negative log likelihood (also known as cross entropy) of mixture models. However, little is understood about the goodness of the fixed points it converges to. In this paper, we study the regions where one component is missing in two-comp…
Gradient-based meta-learning methods leverage gradient descent to learn the commonalities among various tasks. While previous such methods have been successful in meta-learning tasks, they resort to simple gradient descent during meta-testing. Our primary contribution is the {\em MT-net}, which enables the meta-learner…
While training a machine learning model using multiple workers, each of which collects data from their own data sources, it would be most useful when the data collected from different workers can be {\em unique} and {\em different}. Ironically, recent analysis of decentralized parallel stochastic gradient descent (D-PS…
We propose an expectation-maximization-like(EMlike) method to train Boltzmann machine with unconstrained connectivity. It adopts Monte Carlo approximation in the E-step, and replaces the intractable likelihood objective with efficiently computed objectives or directly approximates the gradient of likelihood objective i…
This paper develops a federated EM algorithm for unsupervised learning of mixture models.
Accelerates MMLE using SVGD with Nesterov acceleration.
Cryo-EM reconstruction is reformulated as a stochastic inverse problem to handle structural heterogeneity.
The paper analyzes EM for Mixtures of Experts and shows its equivalence to projected Mirror Descent.
New algorithm improves on EM for streaming data, outperforming existing methods.
Differentiable EM for Gaussian Mixture Models improves model integration.
AdaDEM decouples EM into two parts to improve class overlap and uncertainty.
Study EM and GD for clustering with penalties for misspecification and high dimensions.
Wide hypernetworks don't guarantee convergence under gradient descent.
This paper re-examines the problem of parameter estimation in Bayesian networks with missing values and hidden variables from the perspective of recent work in on-line learning [Kivinen & Warmuth, 1994]. We provide a unified framework for parameter estimation that encompasses both on-line learning, where the model is c…
Gradient descent on DDPM objective learns Gaussian mixtures efficiently.
New algorithm resists Byzantine attacks in distributed SGD for heterogeneous data.
This paper uses dynamical systems to analyze and ensure convergence of the Bayesian EM algorithm.
We study generalized complex Monge-Ampère type equations on closed Hermitian manifolds. We derive {\em a priori} estimates and then prove the existence of admissible solutions. Moreover, the gradient estimate is improved.
New EM algorithm improves deep generative network training.
Discriminative latent-variable models are typically learned using EM or gradient-based optimization, which suffer from local optima. In this paper, we develop a new computationally efficient and provably consistent estimator for a mixture of linear regressions, a simple instance of a discriminative latent-variable mode…
The EM algorithm is one of the most popular algorithm for inference in latent data models. The original formulation of the EM algorithm does not scale to large data set, because the whole data set is required at each iteration of the algorithm. To alleviate this problem, Neal and Hinton have proposed an incremental ver…
Two algorithms improve federated learning efficiency and resilience.
We present a family of expectation-maximization (EM) algorithms for binary and negative-binomial logistic regression, drawing a sharp connection with the variational-Bayes algorithm of Jaakkola and Jordan (2000). Indeed, our results allow a version of this variational-Bayes approach to be re-interpreted as a true EM al…
Quantum Earth Mover's distance improves stability and efficiency in quantum learning.
Derives Mirror Descent from gradient flow on a Riemannian manifold.
Over the past decade there has been considerable interest in spectral algorithms for learning Predictive State Representations (PSRs). Spectral algorithms have appealing theoretical guarantees; however, the resulting models do not always perform well on inference tasks in practice. One reason for this behavior is the m…
We derive gradient and second order {\em a priori} estimates for solutions of the Neumann problem for a general class of fully nonlinear elliptic equations on compact Riemannian manifolds with boundary. These estimates yield regularity and existence results.
Paper uses machine learning in EM framework for better nowcasting.
A neural network method for topic modeling from few documents.
Deep learning generalizes well despite being overparameterized.
The Extreme Deconvolution method fits a probability density to a dataset where each observation has Gaussian noise added with a known sample-specific covariance, originally intended for use with astronomical datasets. The existing fitting method is batch EM, which would not normally be applied to large datasets such as…
Determining the 3D structures of biological molecules is a key problem for both biology and medicine. Electron Cryomicroscopy (Cryo-EM) is a promising technique for structure estimation which relies heavily on computational methods to reconstruct 3D structures from 2D images. This paper introduces the challenging Cryo-…