Paper proposes a faster SPIDER-EM variant for large-scale nonconvex optimization.
problem High computational cost of EM algorithm in large-scale learning.
method Extension of SPIDER-EM for nonconvex finite-sum optimization problems.
result Achieves state-of-the-art complexity bounds and linear convergence under certain conditions.
Stochastic EM for shuffled linear regression improves parameter accuracy and robustness.
problem Inference in linear regression with unknown feature-label order.
method Stochastic EM approach treating permutation as latent variable.
result Stochastic EM outperforms hard EM on partially shuffled datasets.
PSAEM combines EM and particle methods for efficient dynamical system learning.
problem Learning dynamical systems with stochastic approximation and particle methods.
method Particle stochastic approximation EM (PSAEM) algorithm combining stochastic approximation EM and particle Gibbs with ancestor sampling (PGAS).
result PSAEM achieves superior computational performance and convergence compared to existing methods.
Generalising the idea of the classical EM algorithm that is widely used for computing maximum likelihood estimates, we propose an EM-Control (EM-C) algorithm for solving multi-period finite time horizon stochastic control problems. The new algorithm sequentially updates the control policies in each time period using Mo…
New algorithm improves on EM for streaming data, outperforming existing methods.
problem Processing high-volume, streaming data efficiently.
method Incremental stochastic Majorization-Minimization (MM) algorithm.
result The algorithm converges to a stationary point with vanishing gradient.
This paper analyzes and improves EM algorithms for large datasets.
problem Scaling EM algorithm to large datasets.
method Incremental and stochastic EM algorithms, variance reduction.
result Established non-asymptotic convergence bounds for global convergence.
We consider maximum likelihood estimation for Gaussian Mixture Models (Gmms). This task is almost invariably solved (in theory and practice) via the Expectation Maximization (EM) algorithm. EM owes its success to various factors, of which is its ability to fulfill positive definiteness constraints in closed form is of …
Two-Timescale EM Methods improve EM for nonconvex models.
problem Nonconvex latent variable models are challenging for EM.
method Two-stage stochastic updates to handle nonconvex optimization.
result Global convergence for nonconvex objective functions.
This work examines bias-correction methods for distributed stochastic optimization.
problem Performance of bias-correction methods in stochastic and adaptive settings.
method Exact diffusion method is studied under stochastic and adaptive settings.
result Exact diffusion outperforms traditional methods in sparsely-connected network topologies.
Cryo-EM reconstruction is reformulated as a stochastic inverse problem to handle structural heterogeneity.
problem Handling structural heterogeneity in cryo-EM 3D reconstruction.
method Formulated as a stochastic inverse problem over probability measures, using variational discrepancy and Wasserstein gradient flow.
result Validated approach using synthetic examples, demonstrating recovery of continuous structural distributions.
Study derives algorithmic detectability threshold for stochastic block model with EM and BP.
problem Detectability of stochastic block model in practice when parameters are unknown.
method Used expectation-maximization (EM) algorithm with belief propagation (BP).
result Algorithmic detectability threshold differs from Nishimori condition.
New numerical method for non-linear asset price model with CEV volatility.
problem Describing stochastic volatility in asset price dynamics.
method Proposes a mean-reverting theta-rho model with CEV volatility, constructs a truncated EM method.
result Truncated EM solutions can evaluate path-dependent financial products.
Determining the 3D structures of biological molecules is a key problem for both biology and medicine. Electron Cryomicroscopy (Cryo-EM) is a promising technique for structure estimation which relies heavily on computational methods to reconstruct 3D structures from 2D images. This paper introduces the challenging Cryo-…
Mini-batch EM algorithm speeds up convergence for large datasets.
problem Efficiently processing large datasets in latent variable models.
method Proposes mini-batch version of Stochastic Approximation EM algorithm for exponential models.
result Converges under classical conditions with mini-batch sampling.
A new EM algorithm improves inference from large datasets.
problem Efficient inference in latent variable models with large datasets.
method Introduces SPIDER-EM, a novel EM algorithm using SPIDER estimator.
result Finite-time complexity bounds for smooth non-convex likelihood.
Stochastic EM with biased MCMC improves inference stability.
problem Intractable E-step in EM algorithm.
method Stochastic approximation with biased MCMC.
result ULA is more stable and sometimes faster than MALA.
D 2 ^2 2 improves convergence in decentralized SGD for varied data.
problem Decentralized training's sensitivity to data variance among workers.
method D 2 ^2 2 is a variance-aware extension of D-PSGD. result D 2 ^2 2 achieves faster convergence and robustness to data variance. FIEM accelerates EM for large datasets with nonasymptotic convergence bounds.
problem Efficiently optimizing large datasets using EM framework.
method FIEM recasts EM in Stochastic Approximation framework and provides nonasymptotic convergence bounds.
result Nonasymptotic bounds for convergence in expectation as a function of n n n and $\kmax$ . The paper develops a method to learn SDE drift functions from sparse, noisy data.
problem Learning SDE drift functions from sparse and noisy data without strong structural assumptions.
method Data-driven approach using a penalized negative log-likelihood functional over RKHS, with an EM algorithm employing SMC for approximations.
result The method enables accurate estimation of SDE drift functions in low-data regimes.
GFlowNet-EM learns complex latent variable models with discrete structures.
problem Challenges in modeling posteriors over discrete compositional latents with expectation-maximization.
method Uses GFlowNets to learn stochastic policies for sampling from complex posterior distributions.
result GFlowNet-EM enables training expressive LVMs with discrete compositional latents.
Method infers dynamics from incomplete time series data.
problem Challenges in inferring stochastic dynamics from time series with missing data.
method Expectation Maximization (EM) algorithm that iterates between E-step and M-step.
result The EM algorithm effectively recovers missing data points and infers underlying network models from real neuronal activities.
In this paper we continue our descriptions of stock markets in terms of some non abelian operators which are used to describe the portfolio of the various traders and other {\em observable} quantities. After a first prototype model with only two traders, we discuss a more realistic model of market with an arbitrary num…
Paper introduces deterministic EM approximations for non-convex likelihood functions.
problem Deterministic approximations for the E-step of EM algorithm are lacking.
method Developed a theoretical framework for deterministic approximations, analyzed Riemann sums and tempered EM.
result Proved convergence guarantees for deterministic approximations and new non-trivial temperature profiles.
In this paper we provide a new analysis of the SEM algorithm. Unlike previous work, we focus on the analysis of a single run of the algorithm. First, we discuss the algorithm for general mixture distributions. Second, we consider Gaussian mixture models and show that with high probability the update equations of the EM…
Gaussian process state-space models (GP-SSMs) are a very flexible family of models of nonlinear dynamical systems. They comprise a Bayesian nonparametric representation of the dynamics of the system and additional (hyper-)parameters governing the properties of this nonparametric representation. The Bayesian formalism e…
Adaptive Bayesian sampling technique simplifies mass matrix learning.
problem Complexity in learning mass matrices for adaptive samplers.
method Monte Carlo EM framework with online learning of mass matrices.
result Comparable sampling accuracy to Riemannian samplers but faster.
New method uses joint stochastic approximation to improve learning of discrete latent models.
problem Challenges in learning discrete latent variable models, especially with inference model gradients and log-likelihood optimization.
method Proposes a new method based on stochastic approximation theory that directly maximizes the target log-likelihood and minimizes the posterior-inference model divergence.
result Consistently outperforms recent competitive algorithms in generative modeling and structured prediction tasks.
SGD without replacement converges faster for smooth convex functions.
problem Improving convergence rate of SGD for smooth convex functions.
method Using method of exchangeable pairs to bound Wasserstein distance, we provide non-asymptotic results for SGD without replacement.
result SGD without replacement converges at a rate of O(1/K^2) for general smooth, strongly-convex functions.
New algorithm tames non-linear growth in stochastic optimization.
problem Computational challenges in E-step of EM framework.
method Employing interacting particle systems and taming techniques to create tIPLA.
result Non-asymptotic convergence error estimates in Wasserstein-2 distance for tIPLA.
This work proves a strong convergence result for a geometric EM scheme on Riemannian manifolds.
problem Convergence of numerical schemes for manifold-valued SDEs.
method Geometric Euler-Maruyama scheme for Riemannian manifolds.
result Strong convergence of order 1/2 for the geometric EM scheme on Riemannian manifolds.
The paper develops algorithms for computing in stochastic acceptors.
problem Computing input marginals and acceptance probabilities in stochastic acceptors.
method Dynamic programming for input marginals, EM for parameter estimation.
result Efficient algorithms for computing in stochastic acceptors.
Paper explores combining compression and decentralization for robust distributed learning.
problem Balancing computation and communication in distributed learning systems.
method Developed a framework for compressed, decentralized training with extrapolation compression and difference compression strategies.
result Proposed algorithms converge at O ( 1 / n T ) O(1/\sqrt{nT}) O ( 1/ n T ) rate, matching centralized training, and outperform both decentralized and quantized methods. New integration method improves BSDE-based PDE solvers.
problem Discretization bias in standard BSDE-based solvers.
method Proposed Stratonovich-based BSDE formulation with stochastic Heun integration.
result Eliminates bias issues and outperforms EM-based variants.
A new method clusters data from multiple sources using a mixture of multilayer SBMs.
problem Aggregating multiple clustering results from different data sources.
method Uses a mixture of multilayer Stochastic Block Models (SBM) to group co-membership matrices.
result Identifies and clusters observations based on their specificities within components.
New framework tackles stochastic latent subgroup heterogeneity in online decision-making.
problem Stochastic latent heterogeneity in online decision-making where individual responses vary with unobserved subgroups.
method Latent heterogeneous bandit framework using EM-greedy algorithm to learn subgroup probabilities and reward parameters.
result Achieves optimal estimation and classification guarantees, revealing a fundamental stochastic barrier in online decision-making.
A new algorithm using sliced Wasserstein distance improves GMM parameter estimation.
problem Inefficiency of EM algorithm in finding optimal GMM parameters.
method Proposes a new algorithm using sliced Wasserstein distance to minimize the distance between the mixture model and data distribution.
result The new algorithm yields more robust and accurate GMM parameter estimates.
We present a family of expectation-maximization (EM) algorithms for binary and negative-binomial logistic regression, drawing a sharp connection with the variational-Bayes algorithm of Jaakkola and Jordan (2000). Indeed, our results allow a version of this variational-Bayes approach to be re-interpreted as a true EM al…
SOLVAR efficiently analyzes cryo-EM data's structural variability.
problem Analyzing continuous heterogeneity in cryo-EM data.
method Low-rank assumption on covariance matrix for tractable estimation.
result Accurately captures dominant components of structural variability.
A neural network method for topic modeling from few documents.
problem Training topic models requires many documents, limiting their applicability.
method Neural networks learn topic model priors from a few documents, optimizing likelihood using EM algorithm.
result The method achieves better perplexity than existing methods on real-world text document sets.
This paper analyzes EM algorithm for softmax mixture models in high dimensions.
problem Modeling heterogeneous populations choosing from multiple attributes.
method Comprehensive analysis of the EM algorithm for softmax mixture models (SMMs), proving identifiability and convergence.
result EM algorithm recovers mixture atoms at near-parametric rate under suitable initialization.
New model predicts stochastic dynamics with hidden variables.
problem Predicting state transitions in stochastic dynamical systems.
method Hierarchical Bayesian linear regression with local features and variational EM algorithm.
result Parsimonious model structures and fast, accurate predictions.
Deep learning generalizes well despite being overparameterized.
problem Why deep networks generalize well despite fitting training data perfectly.
method Empirical study of training methods and derivation of data-dependent generalization bounds.
result Uniform convergence alone is insufficient for explaining generalization in overparameterized settings.
New method reduces distributed non-convex optimization rounds and bits.
problem Efficiently optimizing non-convex models in distributed systems.
method Introduces permutation compressors to reduce communication complexity.
result PermK compressors lead to significant communication complexity improvements.
The study of higher-order homology embeddings for manifold topology.
problem Understanding the structure of higher-order homology embeddings to disclose geometric or topological information.
method Analysis of the null space of the k k k -th order Laplacian and proposing an algorithm to factorize the homology embedding. result The proposed spectral loop detection algorithm is more efficient and effective on various data types.
New algorithm resists Byzantine attacks in distributed SGD for heterogeneous data.
problem Byzantine attacks in distributed SGD for heterogeneous data.
method Polynomial-time outlier-filtering for robust mean estimation, new matrix concentration result.
result Tolerates up to 25% Byzantine workers and achieves optimal convergence rates.
Study proves convergence of interest rate model approximations.
problem Investigating convergence of stochastic interest rate models.
method Developed analytical tools for true and truncated EM solutions, proving convergence in probability.
result True solution converges in probability to truncated EM solution as step size approaches zero.
The paper analyzes how knowing the market price of risk impacts optimal investment strategies.
problem Optimal allocation in financial markets with uncertain market price of risk.
method Solves optimal allocation problem in continuous time for CRRA utility investors in two scenarios: full and partial information.
result Investors are willing to pay for more accurate information on market price of risk, affecting optimal strategies.
Efficiently trains GMMs for streaming data with SGD, addressing local optima and numerical instabilities.
problem Local optima and numerical instabilities in training GMMs for high-dimensional streaming data.
method Stochastic Gradient Descent (SGD) with adaptive annealing and exponential-free approximation.
result SGD approach trains GMMs without k-means initialization and outperforms sEM for high-dimensional data.