New algorithm learns mixtures of subcubes efficiently, with applications to decision trees.
problem Learning mixtures of subcubes over binary space.
method Higher-order multilinear moments for nO(logk)-time learning. result Polynomial dependence on 1/ε for decision trees with stochastic transitions. Transforms uniform learners to work under arbitrary distributions efficiently.
problem Learning under arbitrary distributions from uniform learners.
method Black-box transformation using decision tree decomposition.
result Efficient transformation with runtime scaling with distribution complexity.
We create a framework for odd Khovanov homology in the spirit of Bar-Natan's construction for the ordinary Khovanov homology. Namely, we express the cube of resolutions of a link diagram as a diagram in a certain 2-category of chronological cobordisms and show that it is 2-commutative: the composition of 2-morphisms al…
A new anomaly detection method using partial identification.
problem Detecting anomalies in large datasets.
method Partial Identification framework and PIDScore geometric anomaly measure.
result PIDForest outperforms other methods in anomaly detection.
We show that the map on components from the space of classical long knots to the n-th stage of its Goodwillie-Weiss embedding calculus tower is a map of monoids whose target is an abelian group and which is invariant under clasper surgery. We deduce that this map on components is a finite type-(n-1) knot invariant. We …
New sampling bounds improve uniform coverage verification in machine learning.
problem Conservative bounds in classical coverage analyses at small failure probabilities.
method Variance-based analysis of uniform random sampling on a d-dimensional unit hypercube. result Sample complexity bound with logarithmic dependence on failure probability.
New proof of Heegaard Floer surgery formulas using Fukaya category of the torus.
problem Heegaard Floer surgery formulas
method New exact triangle in Fukaya category of the torus
result Extensions of link surgery formula to arbitrary links
Study calculates tail risk for various mixture distributions.
problem Estimating tail risk for complex distribution mixtures.
method Analyzes tail conditional expectation for location-scale mixtures of elliptical distributions.
result Developed methods for calculating tail risk in various distributions.
Enhances mixture models with classifier-defined weights.
problem Density evaluation and sampling in mixture models.
method Introduces Classifier Weighted Mixtures (CWM) with functional weights.
result Improves expressivity in variational estimation without increasing complexity.
Two approaches improve parameter learning in various mixture models.
problem Parameter learning in mixture models.
method Complex-analytic and algebraic-combinatorial methods.
result Improved sample sufficiency for parameter estimation in specific mixture models.
We consider unsupervised estimation of mixtures of discrete graphical models, where the class variable corresponding to the mixture components is hidden and each mixture component over the observed variables can have a potentially different Markov graph structure and parameters. We propose a novel approach for estimati…
Paper introduces Normalized Wasserstein measure for better handling of imbalanced mixture distributions.
problem Wasserstein distance fails for mixture distributions with imbalanced proportions.
method Introduce mixture proportions as optimization variables to normalize Wasserstein formulation.
result Normalized Wasserstein measure leads to significant performance gains for mixture distributions.
Improved VB algorithm for NIG mixtures outperforms Gaussian mixtures for non-Gaussian data.
problem Clustering non-Gaussian data, especially heavy-tailed and asymmetric.
method Proposed an improved VB algorithm for NIG mixture models and extended Dirichlet process mixture models.
result Outperforms Gaussian mixtures and existing NIG mixture models, especially for highly non-normative data.
When estimating finite mixture models, it is common to make assumptions on the mixture components, such as parametric assumptions. In this work, we make no distributional assumptions on the mixture components and instead assume that observations from the mixture model are grouped, such that observations in the same gro…
The two most extended density-based approaches to clustering are surely mixture model clustering and modal clustering. In the mixture model approach, the density is represented as a mixture and clusters are associated to the different mixture components. In modal clustering, clusters are understood as regions of high d…
Optimal mixtures of generative models outperform individual models on image datasets.
problem Selecting the best single model from a group of trained generative models.
method Formulated a quadratic optimization problem and proposed the Mixture-UCB algorithm for efficient selection.
result Mixture of generative models achieves better evaluation scores than individual models on benchmark datasets.
New bounds on sample size for identifying mixture models with grouped samples.
problem Identifying mixture models with minimal sample size.
method Generalized identifiability bounds for mixture models with grouped samples.
result Identifiability with (2m−1)/(k−1) samples per group, with no improvement possible. Bayesian approach learns nonparametric mixture components from heterogeneous data.
problem Realistic modeling of heterogeneous data populations with nonparametric mixture components.
method Bayesian nonparametric modeling using Dirichlet process mixture priors.
result Posterior contraction rates for component densities are nearly polynomial, improving over deconvolution methods.
DGMM uses deep layers of Gaussian mixtures for flexible data modeling.
problem Efficiently modeling complex data relationships.
method Deep Gaussian Mixture Models (DGMM) with nested mixtures of linear models and factor models.
result DGMM provides a flexible nonlinear model for data description.
Robust learning mixtures of linear regressions improve robustness.
problem Improving robustness in learning mixtures of linear regressions.
method Connecting mixtures of linear regressions and mixtures of Gaussians with thresholding for a quasi-polynomial time algorithm.
result The algorithm has significantly better robustness than previous results.
Spatially constrained Gaussian mixture models reduce covariance complexity.
problem High dimensionality in finite mixture models for spatial data.
method Spatial covariance constraint with only four free parameters.
result Improves clustering of multi-way spatial data and inference of spatial patterns.
This paper compares EM and GD in two-component mixture models, finding EM escapes bad local optima more reliably.
problem Understanding the convergence of EM and GD in mixture models, especially in regions where one component is missing.
method Analyzing regions called one-cluster regions in two-component mixture models of Gaussians and Bernoullis, comparing the propensity of EM and GD to converge to these regions.
result EM escapes one-cluster regions exponentially fast, while GD escapes them linearly fast, indicating EM is less likely to converge to bad local optima.
A new method detects outliers using ensembles of Dirichlet process mixtures.
problem Challenges in unsupervised outlier detection using Dirichlet process mixtures.
method Ensembles of Dirichlet process Gaussian mixtures with random subspace and subsampling.
result Empirically outperforms existing approaches in unsupervised outlier detection.
The parsimonious Gaussian mixture models, which exploit an eigenvalue decomposition of the group covariance matrices of the Gaussian mixture, have shown their success in particular in cluster analysis. Their estimation is in general performed by maximum likelihood estimation and has also been considered from a parametr…
Information-theoretic measures such as the entropy, cross-entropy and the Kullback-Leibler divergence between two mixture models is a core primitive in many signal processing tasks. Since the Kullback-Leibler divergence of mixtures provably does not admit a closed-form formula, it is in practice either estimated using …
Paper establishes universal lower bounds and optimal rates for clustering sub-exponential mixture models.
problem Achieving optimal error rates in clustering sub-exponential mixture models.
method Establishes universal lower bounds and demonstrates iterative algorithms' optimality in sub-exponential mixture models.
result Iterative algorithms achieve the universal lower bound in sub-exponential mixture models.
New method for summarizing Bayesian mixture models using sliced Wasserstein distances.
problem Estimating the mixing measure in nonparametric Bayesian mixture models.
method Decision-theoretic approach using sliced Wasserstein distances for Gaussian mixtures.
result Effective estimation of the mixing measure and mixture density.
The paper tackles learning mixtures of two multinomial logits, showing identifiability and presenting an algorithm.
problem Learning an arbitrary mixture of two multinomial logits.
method Reduction to solving a system of univariate quartic equations, followed by an algorithm using polynomial and linear samples.
result Identifiability of the mixture models may only fail on an algebraic variety of negligible measure.
Algorithm estimates nonparametric mixtures from grouped data.
problem Estimating identifiable nonparametric mixture models from grouped observations.
method Oracle inequality for weighted kernel density estimators and general consistency result.
result Consistent estimation of mixture components from grouped observations.
A new method for fast Bayesian mixture model estimation.
problem Estimating Bayesian mixture models is computationally challenging.
method Amortized Bayesian Inference (ABI) framework for mixture models.
result The method provides fast inference for mixture models.
New algorithm recovers mixture means even with many outliers.
problem Estimating mixture means when outliers overwhelm small groups.
method Proposes an algorithm for robust mixture learning with minimal list-size overhead.
result Order-optimal error guarantees for each mixture mean.
Finite mixture models are statistical models which appear in many problems in statistics and machine learning. In such models it is assumed that data are drawn from random probability measures, called mixture components, which are themselves drawn from a probability measure P over probability measures. When estimating …
Study on Dirichlet process mixtures for clustering consistency.
problem Consistency of clustering with Dirichlet process mixtures.
method Analysis of posterior distribution as sample size increases, focusing on consistency for the number of clusters.
result Consistency for the number of clusters can be achieved with a properly adapted concentration parameter in a Bayesian setting.
The paper uses Gaussian mixture models for Bayesian networks and proposes an optimization algorithm.
problem Modeling nodes in Bayesian networks with complex distributions.
method Gaussian mixture models combined with double iteration algorithm.
result The double iteration algorithm optimizes Gaussian mixture models effectively.
Paper uses Gaussian mixture models and Wasserstein distance for schema matching.
problem Schema matching between different datasets.
method Gaussian mixture models and Wasserstein distance for comparison.
result Derives an approximation for Wasserstein distance between Gaussian mixture models.
Improves scalability and efficiency of mixture models in black-box variational inference.
problem Scaling mixture models in black-box variational inference leads to high parameter and time costs.
method Introduces MISVAE for amortized mixture parameter space and new ELBO estimators.
result Achieves superior estimation performance with fewer parameters and shorter inference time.
SMT trains generative models by estimating mixture scores, outperforming existing methods.
problem Training one-step generative models efficiently and effectively.
method Score-of-Mixture Training (SMT) estimates the score of mixture distributions between real and fake samples.
result SMT/SMD outperform existing methods on CIFAR-10 and ImageNet 64x64 datasets.
Causal Inference over Mixtures models cyclic, evolving causal processes using a mixture of DAGs.
problem Cycles, time evolution, and population differences in causal processes are challenging for traditional graphical models.
method Causal Inference over Mixtures uses a mixture of directed cyclic graphs (DAGs) to model longitudinal data and infer causal relations.
result Improved performance compared to prior approaches in inferring causal relations from a mixture of DAGs.
New method estimates mixture model components efficiently.
problem Estimating the number of components in finite mixture models.
method Group-Sort-Fuse (GSF) procedure for simultaneous estimation of order and mixing measure.
result GSF achieves consistent estimation of true mixture order and n−1/2 convergence rate. Discriminative classifier for compositional data using hierarchical mixture of Generalized Dirichlet models.
problem Classifying compositional data, especially in spam detection and color space identification.
method Hierarchical mixture of discriminative Generalized Dirichlet classifiers, using variational approximation for parameter learning.
result First time a variational upper-bound for Generalized Dirichlet mixture is proposed in literature.
New method classifies mixtures without labels, recovering latent classes.
problem Classification without reliable instance-level labels.
method Posterior simplex geometry for multiclass learning.
result Classifier trained on mixture identities recovers latent classes and their proportions.
Mixture modeling is a general technique for making any simple model more expressive through weighted combination. This generality and simplicity in part explains the success of the Expectation Maximization (EM) algorithm, in which updates are easy to derive for a wide class of mixture models. However, the likelihood of…
MixTS uses a mixture prior to analyze Thompson Sampling in multi-task learning.
problem Analyzing Thompson Sampling in environments with uncertain and multi-class problems.
method Developed MixTS by incorporating a mixture prior into Thompson Sampling and using a novel proof technique for mixture distributions.
result Proved Bayes regret bounds for MixTS in linear bandits and finite-horizon reinforcement learning.
Study uniform rates for estimating Gaussian mixtures without separation assumption.
problem Estimating parameters in two-component Gaussian mixtures without separation.
method Uniform convergence rates derived using minimax lower bounds and careful analysis of polynomial equalities.
result Phase transition in optimal estimation rate based on mixture balance.
The study bounds the stability of Gaussian mixtures under small perturbations.
problem Stability of Gaussian mixtures under small changes in distribution.
method Deriving an explicit bound on parameter stability of spherical Gaussian Mixture Models (sGMM) in a pre-defined model class.
result Upper bound on parameter distance of close sGMMs to the original sGMM, dependent only on the original model.
A method connects KDE to sparse mixture models with adaptive regularization.
problem Estimating Gaussian mixture models from sparse data.
method Generalized expectation-maximization method with adaptive regularization.
result Sparse mixture models retain details from adaptive KDE.
New concept of mixture complexity helps detect gradual clustering changes.
problem Determining the number of clusters in mixture models with overlaps and weight biases.
method Introducing mixture complexity (MC) as a new measure of cluster size, defined from information theory.
result MC can detect gradual clustering changes, allowing earlier detection and finer distinction.
The paper examines risk aggregation under mixtures of marginals, finding that more homogeneous distributions lead to larger uncertainty.
problem Investigating the impact of mixing on risk aggregation uncertainty.
method Analyzes ordering relations and inequalities for aggregation sets under distribution and quantile mixtures.
result More homogeneous marginals result in larger aggregation sets, indicating greater model uncertainty.