arXiv research

A new EM gradient algorithm for mixture models with skewed components.

We consider maximum likelihood estimation for Gaussian Mixture Models (Gmms). This task is almost invariably solved (in theory and practice) via the Expectation Maximization (EM) algorithm. EM owes its success to various factors, of which is its ability to fulfill positive definiteness constraints in closed form is of …

Mixed linear regression involves the recovery of two (or more) unknown vectors from unlabeled linear measurements; that is, where each sample comes from exactly one of the vectors, but we do not know which one. It is a classic problem, and the natural and empirically most popular approach to its solution has been the E…

New algorithm for estimating MLR parameters with non-Gaussian noise.

New framework improves EM algorithm convergence under log-Sobolev inequality.

It is well known that the braid index of a link equals the minimum number of Seifert circles among all link diagrams representing it. For a link with a reduced alternating diagram $D$, $s(D)$, the number of Seifert circles in $D$, equals the braid index $\textbf{b}(D)$ of $D$ if $D$ contains no {\em lone crossings} (a …

We propose an expectation-maximization-like(EMlike) method to train Boltzmann machine with unconstrained connectivity. It adopts Monte Carlo approximation in the E-step, and replaces the intractable likelihood objective with efficiently computed objectives or directly approximates the gradient of likelihood objective i…

DM framework improves robustness and efficiency in latent-mixture models.

Semi-supervised EM improves convergence rate with labeled samples.

Paper analyzes agnostic learning of mixed linear regression without generative models.

Latent class model (LCM), which is a finite mixture of different categorical distributions, is one of the most widely used models in statistics and machine learning fields. Because of its non-continuous nature and the flexibility in shape, researchers in practice areas such as marketing and social sciences also frequen…

Integrates VAEs into EM for deep clustering and generation.

Expectation-Maximization (EM) is a prominent approach for parameter estimation of hidden (aka latent) variable models. Given the full batch of data, EM forms an upper-bound of the negative log-likelihood of the model at each iteration and updates to the minimizer of this upper-bound. We first provide a "model level" in…

A rational link may be represented by any of the (infinitely) many link diagrams corresponding to various continued fraction expansions of the same rational number. The continued fraction expansion of the rational number in which all signs are the same is called a {\em nonalternating form} and the diagram corresponding…

Marginal MAP problems are notoriously difficult tasks for graphical models. We derive a general variational framework for solving marginal MAP problems, in which we apply analogues of the Bethe, tree-reweighted, and mean field approximations. We then derive a "mixed" message passing algorithm and a convergent alternati…

Develops FSC for maxima nominated samples, improving classification in rare-event data.

Call {\em i-hedrite} any 4-valent n-vertex plane graph, whose faces are 2-, 3- and 4-gons only and $p_2+p_3=i$. The edges of an i-hedrite, as of any Eulerian plane graph, are partitioned by its {\em central circuits}, i.e. those, which are obtained by starting with an edge and continuing at each vertex by the edge oppo…

An evolutionary algorithm (EA) is developed as an alternative to the EM algorithm for parameter estimation in model-based clustering. This EA facilitates a different search of the fitness landscape, i.e., the likelihood surface, utilizing both crossover and mutation. Furthermore, this EA represents an efficient approac…

In this paper we propose new techniques to sample arbitrary third-order tensors, with an objective of speeding up tensor algorithms that have recently gained popularity in machine learning. Our main contribution is a new way to select, in a biased random way, only $O(n^{1.5}/ε^2)$ of the possible $n^3$ elements while s…

Paper tackles MLR prediction error without assuming realizable models.

Enhances large language models' reasoning through simpler off-policy reinforcement learning.

A method improves Cryo-EM 3D map refinement by regularizing rotation estimation.

Capsule Networks (CapsNets) have been proposed as an alternative to Convolutional Neural Networks (CNNs). This paper showcases how CapsNets are more capable than CNNs for autonomous agent exploration of realistic scenarios. In real world navigation, rewards external to agents may be rare. In turn, reinforcement learnin…

Conventional approaches of sampling signals follow the celebrated theorem of Nyquist and Shannon. Compressive sampling, introduced by Donoho, Romberg and Tao, is a new paradigm that goes against the conventional methods in data acquisition and provides a way of recovering signals using fewer samples than the traditiona…

Random utility theory models an agent's preferences on alternatives by drawing a real-valued score on each alternative (typically independently) from a parameterized distribution, and then ranking the alternatives according to scores. A special case that has received significant attention is the Plackett-Luce model, fo…

Paper improves BN structure learning from incomplete data.

This paper models time-series data with a mixture of Markov chains, automatically determining the number of components.

We describe some regular techniques of calculating finite degree invariants of triple points free smooth plane curves $S^1 \to R^2$. They are a direct analog of similar techniques for knot invariants and are based on the calculus of {\em triangular diagrams} and {\em connected hypergraphs} in the same way as the calcul…

In this paper, we formulate the problem of inferring a Finite Mixture Model from discrete data as an optimal transport problem with entropic regularization of parameter $λ\geq 0$. Our method unifies hard and soft clustering, the Expectation-Maximization (EM) algorithm being exactly recovered for $λ=1$. The family of cl…

Method infers dynamics from incomplete time series data.

We propose a nonparametric procedure to achieve fast inference in generative graphical models when the number of latent states is very large. The approach is based on iterative latent variable preselection, where we alternate between learning a 'selection function' to reveal the relevant latent variables, and use this …

Gaussian mixture models (GMM) are powerful parametric tools with many applications in machine learning and computer vision. Expectation maximization (EM) is the most popular algorithm for estimating the GMM parameters. However, EM guarantees only convergence to a stationary point of the log-likelihood function, which c…

When sufficient labeled data are available, classical criteria based on Receiver Operating Characteristic (ROC) or Precision-Recall (PR) curves can be used to compare the performance of un-supervised anomaly detection algorithms. However , in many situations, few or no data are labeled. This calls for alternative crite…

A new KF handles outliers without MSE loss.

This paper proposes a method of segmenting temporal data into ordered classes. It is based on mixture models and a discrete latent process, which enables to successively activates the classes. The classification can be performed by maximizing the likelihood via the EM algorithm or by simultaneously optimizing the model…

Question-answering (QA) data often encodes essential information in many facets. This paper studies a natural question: Can we get supervision from QA data for other tasks (typically, non-QA ones)? For example, {\em can we use QAMR (Michael et al., 2017) to improve named entity recognition?} We suggest that simply furt…

Mixture models are a fundamental tool in applied statistics and machine learning for treating data taken from multiple subpopulations. The current practice for estimating the parameters of such models relies on local search heuristics (e.g., the EM algorithm) which are prone to failure, and existing consistent methods …

Graphical Gaussian models have proven to be useful tools for exploring network structures based on multivariate data. Applications to studies of gene expression have generated substantial interest in these models, and resulting recent progress includes the development of fitting methodology involving penalization of th…

New approach improves AI's handling of incomplete data.

We present a new mixture model-based discriminant analysis approach for functional data using a specific hidden process regression model. The approach allows for fitting flexible curve-models to each class of complex-shaped curves presenting regime changes. The model parameters are learned by maximizing the observed-da…

Mixture model-based clustering has become an increasingly popular data analysis technique since its introduction over fifty years ago, and is now commonly utilized within a family setting. Families of mixture models arise when the component parameters, usually the component covariance (or scale) matrices, are decompose…

For a knot K in S^3, let T(K) be the characteristic toric sub-orbifold of the orbifold (S^3,K) as defined by Bonahon and Siebenmann. If K has unknotting number one, we show that an unknotting arc for K can always be found which is disjoint from T(K), unless either K is an EM-knot (of Eudave-Munoz) or (S^3,K) contains a…

Consider a movie recommendation system where apart from the ratings information, side information such as user's age or movie's genre is also available. Unlike standard matrix completion, in this setting one should be able to predict inductively on new users/movies. In this paper, we study the problem of inductive matr…

The paper analyzes meta-learning in a Gaussian setting, providing bounds and matching algorithms.

We consider the problem of solving mixed random linear equations with $k$ components. This is the noiseless setting of mixed linear regression. The goal is to estimate multiple linear models from mixed samples in the case where the labels (which sample corresponds to which model) are not observed. We give a tractable a…

Deep neural networks with discrete latent variables offer the promise of better symbolic reasoning, and learning abstractions that are more useful to new tasks. There has been a surge in interest in discrete latent variable models, however, despite several recent improvements, the training of discrete latent variable m…

We derive both {\em local} and {\em global} generalized {\em Bianchi identities} for classical Lagrangian field theories on gauge-natural bundles. We show that globally defined generalized Bianchi identities can be found without the {\em a priori} introduction of a connection. The proof is based on a {\em global} decom…

Paper proposes a faster SPIDER-EM variant for large-scale nonconvex optimization.