Mixture Density RNNs learn to model predictions as multiple Gaussian distributions.
problem Understanding predictions made by MD-RNNs in complex environments.
method Analyzed predictions from trained MD-RNNs, focusing on their Gaussian components.
result MD-RNNs' Gaussian components separately model stochastic events and scenarios governed by different rules.
CMDRNN predicts user location using WiFi fingerprints with deep learning.
problem Predicting user activity with WiFi fingerprints is challenging due to high dimensionality.
method Combines CNN, RNN, and MDN to model high-dimensional time-series data.
result CMDRNN effectively predicts user location using WiFi fingerprints.
Although Recurrent Neural Network (RNN) has been a powerful tool for modeling sequential data, its performance is inadequate when processing sequences with multiple patterns. In this paper, we address this challenge by introducing a novel mixture layer and constructing an adaptive RNN. The mixture layer augmented RNN (…
Study compares MoE and RNN models for stock price prediction across volatility profiles.
problem Improving stock price prediction accuracy across different volatility levels.
method Dynamic Mixture of Experts model combining RNN and linear models, adjusting weights through a gating network.
result MoE model outperforms individual models in reducing prediction errors.
Paper uses RNNs to design LDPC codes for binary erasure channels.
problem Designing capacity-approaching LDPC codes for binary erasure channels.
method Model Density Evolution using RNNs to determine LDPC code coefficients and structure.
result NDE improves LDPC design performance and complexity compared to differential evolution.
The two most extended density-based approaches to clustering are surely mixture model clustering and modal clustering. In the mixture model approach, the density is represented as a mixture and clusters are associated to the different mixture components. In modal clustering, clusters are understood as regions of high d…
Density-based clustering is the task of discovering high-density regions of entities (clusters) that are separated from each other by contiguous regions of low-density. DBSCAN is, arguably, the most popular density-based clustering algorithm. However, its cluster recovery capabilities depend on the combination of the t…
The paper proposes a method for interpretable mixture density estimation using a tree structure.
problem Complex probability distributions in machine learning models.
method Interpretable tree structure for mixture density estimation with fast inference.
result The method achieves both high speed and interpretability for mixture density estimation.
Graph Mixture Density Networks model multimodal data on graphs.
problem Challenging conditional density estimation problems with structured data.
method Combining mixture models and graph representation learning.
result Significant improvement in likelihood of epidemic outcomes.
Estimates Gaussian mixtures from weighted samples efficiently.
problem Estimating Gaussian mixtures from weighted samples with correct weight treatment.
method Density interpretation and expectation-maximization method considering weights.
result Correctly estimates Gaussian mixtures with weighted samples.
Improves sequence modeling with a flow-based recurrent mixture density network.
problem Sequence modeling and sequence-to-sequence mapping applications.
method Generalized recurrent mixture density networks using normalized flow transformations.
result Significantly improved fit to image sequences measured by log-likelihood.
A fast Modal EM algorithm for Gaussian mixtures.
problem Clustering with Gaussian mixtures.
method Modal EM algorithm for Gaussian mixtures.
result High flexibility in various clustering contexts.
Enhances mixture models with classifier-defined weights.
problem Density evaluation and sampling in mixture models.
method Introduces Classifier Weighted Mixtures (CWM) with functional weights.
result Improves expressivity in variational estimation without increasing complexity.
The study applies spatial density models to mobile node movements using Möbius distributions.
problem Modeling the steady-state density of mobile nodes on a 2D terrain.
method Used mixture density networks with Möbius distributions to describe node density over a disk.
result Möbius distributions are more suitable for capturing radial changes in node density compared to Gaussian distributions.
Regularized mixtures improve inflation and interest rate forecasts, especially correcting overconfidence.
problem Improving density forecasts of Eurozone inflation and real interest rates.
method Construct regularized mixtures of density forecasts with various objectives and penalties.
result Regularized mixtures outperform individual forecasters, especially correcting overconfidence.
The paper analyzes how a known density function can be deviated by a mixture distribution as more data is collected.
problem Modeling the deviation of a known density function when more data is collected.
method A novel distinguishability notion is used to establish rates of convergence for maximum likelihood estimates of the deviated proportion and latent mixing measure.
result Rates of convergence for the maximum likelihood estimates of the deviated proportion and latent mixing measure are established under the Wasserstein metric.
Paper proposes new costs for learning multiple centers in MDNs.
problem Learning multiple centers for density approximation in MDNs.
method Combines MDNs with contrastive costs using four types of kernelized matrix costs.
result New costs improve data density approximation in MDNs.
A new method for density estimation using mixture discrepancy and moments.
problem Generalizing histogram statistics to higher dimensions.
method Density estimation via mixture discrepancy and moments (DSP-mix and MSP).
result DSP-mix and MSP are computationally tractable and maintain accuracy with increased speed.
GGMPs improve non-Gaussian conditional density estimation.
problem Multimodality, heteroscedasticity, and strong non-Gaussianity in conditional density estimation.
method GGMP combines local Gaussian mixture fitting, cross-input component alignment, and per-component heteroscedastic GP training.
result GGMPs improve distributional approximation on synthetic and real-world datasets.
The paper introduces a new method for multivariate density estimation using deep neural mixture models.
problem Multivariate density estimation is a fundamental but underexplored task in machine learning.
method The paper extends Neural Mixture Densities (NMMs) to multivariate Deep Neural Mixture Models (DNMMs) using maximum-likelihood algorithm.
result The DNMMs can model any probability density function to any degree of precision and outperform traditional statistical estimation techniques.
Paper uses GMM and MAF for probabilistic classification, outperforming simpler models.
problem Classifying data with complex distributions.
method Density estimation using Gaussian Mixture Model and Masked Autoregressive Flow.
result Proposed classifiers outperform simpler models like linear discriminant analysis.
In the present paper, given an evolving mixture of probability densities, we define a candidate diffusion process whose marginal law follows the same evolution. We derive as a particular case a stochastic differential equation (SDE) admitting a unique strong solution and whose density evolves as a mixture of Gaussian d…
This paper studies sparse density estimation via ℓ1 penalization (SPADES). We focus on estimation in high-dimensional mixture models and nonparametric adaptive density estimation. We show, respectively, that SPADES can recover, with high probability, the unknown components of a mixture of probability densities an…
A new method for density estimation using nearest neighbor Dirichlet mixtures.
problem Slow and unstable Bayesian density estimation methods.
method Nearest neighbor grouping, local Bayesian parametric models, Dirichlet prior, Monte Carlo sampling.
result Effective density estimation with improved computational efficiency.
EnEMF uses Epanechnikov kernel for high-dimensional filtering, improving accuracy and robustness.
problem Suboptimal Gaussian mixture kernel density estimates in high-dimensional settings.
method Ensemble Epanechnikov mixture filter (EnEMF) using optimal Epanechnikov kernel.
result EnEMF reduces error per particle on high-dimensional systems like Lorenz '96.
Method estimates joint probability density from samples using low-rank decomposition and random projections.
problem Estimating joint probability density from limited samples.
method Low-rank tensor decomposition, dictionaries, and Radon transforms.
result Algorithm outperforms previous methods in estimating synthetic probability densities.
Simulates sepsis treatment decisions using a world model approach.
problem Predicting optimal sepsis treatment actions based on noisy EHR data.
method Uses a Variational Auto-Encoder and Mixture Density Network (MDN-RNN) to model sepsis patient trajectories.
result Simulator learns from MIMIC dataset to predict patient states.
Mixture models are powerful statistical models used in many applications ranging from density estimation to clustering and classification. When dealing with mixture models, there are many issues that the experimenter should be aware of and needs to solve. The MixEst toolbox is a powerful and user-friendly package for M…
Normalizing flows improve density estimation from noisy data.
problem Estimating underlying density from noisy samples.
method Use normalizing flows for density estimation with arbitrary noise distributions, using amortized variational inference.
result Normalizing flows can outperform Gaussian mixtures for density deconvolution.
The Normalizing Flow (NF) models a general probability density by estimating an invertible transformation applied on samples drawn from a known distribution. We introduce a new type of NF, called Deep Diffeomorphic Normalizing Flow (DDNF). A diffeomorphic flow is an invertible function where both the function and its i…
Autoregressive generative models consistently achieve the best results in density estimation tasks involving high dimensional data, such as images or audio. They pose density estimation as a sequence modeling task, where a recurrent neural network (RNN) models the conditional distribution over the next element conditio…
Bayesian approach learns nonparametric mixture components from heterogeneous data.
problem Realistic modeling of heterogeneous data populations with nonparametric mixture components.
method Bayesian nonparametric modeling using Dirichlet process mixture priors.
result Posterior contraction rates for component densities are nearly polynomial, improving over deconvolution methods.
Algorithm estimates nonparametric mixtures from grouped data.
problem Estimating identifiable nonparametric mixture models from grouped observations.
method Oracle inequality for weighted kernel density estimators and general consistency result.
result Consistent estimation of mixture components from grouped observations.
New method for private density estimation of high-dimensional Gaussian mixtures.
problem Private density estimation for mixtures of unrestricted high-dimensional Gaussians.
method Exploits list global stability to prove upper bound on sample complexity.
result First upper bound on sample complexity for agnostic private density estimation.
Paper develops a consistent estimator for discrete mixture models.
problem Estimating mixing probability density in discrete mixture models.
method Develops an L1 consistent estimator under specific conditions. result Consistent estimation of mixing probability density for every density f. We introduce a balloon estimator in a generalized expectation-maximization method for estimating all parameters of a Gaussian mixture model given one data sample per mixture component. Instead of limiting explicitly the model size, this regularization strategy yields low-complexity sparse models where the number of eff…
We address two shortcomings in online travel time estimation methods for congested urban traffic. The first shortcoming is related to the determination of the number of mixture modes, which can change dynamically, within day and from day to day. The second shortcoming is the wide-spread use of Gaussian probability dens…
Survival MDN uses invertible functions to speed up survival analysis models.
problem Training neural ODEs for survival analysis is computationally expensive.
method Survival MDN applies an invertible positive function to MDN outputs.
result Survival MDN outperforms or matches other models on concordance, Brier score, and log-likelihood.
We consider nonparametric estimation of the state price density encapsulated in option prices. Unlike usual density estimation problems, we only observe option prices and their corresponding strike prices rather than samples from the state price density. We propose to model the state price density directly with a nonpa…
Generative model prices basket options efficiently.
problem Real-time pricing of basket options with varying market inputs.
method Truncated path signatures and Mixture Density Networks (MDN) for learning the terminal density.
result The model produces small pricing errors and matches Monte Carlo simulations closely.
We introduce RNADE, a new model for joint density estimation of real-valued vectors. Our model calculates the density of a datapoint as the product of one-dimensional conditionals modeled using mixture density networks with shared parameters. RNADE learns a distributed representation of the data, while having a tractab…
We show that the visible sector probability density function of the Riemann-Theta Boltzmann machine corresponds to a gaussian mixture model consisting of an infinite number of component multi-variate gaussians. The weights of the mixture are given by a discrete multi-variate gaussian over the hidden state space. This a…
This work explores CS-RBFs as novel probability density functions.
problem Underexplored use of CS-RBFs in statistics and probability modeling.
method Derivation of statistical properties of CS-RBFs as univariate and conditional densities, analysis of mixture models, and introduction of an incremental learning algorithm.
result CS-RBF densities provide competitive results in likelihood and model complexity compared to Gaussian mixture models.
New method improves Gaussian Mixture Model fitting speed.
problem Slow convergence of EM algorithm for complex data.
method Riemannian Newton Trust-Region method for Gaussian Mixture Models.
result Outperforms existing methods in runtime and iterations.
In this paper, we introduce and evaluate a data-driven staged mixture modeling technique for building density, regression, and classification models. Our basic approach is to sequentially add components to a finite mixture model using the structural expectation maximization (SEM) algorithm. We show that our technique i…
Least Squares EM converges globally for log-concave mixtures.
problem Location estimation in mixtures of two log-concave densities.
method Least Squares EM algorithm applied to log-concave mixtures.
result Least Squares EM converges globally to the true location parameter.
New mixture models for clustering and density estimation of unknown distributions.
problem Clustering and density estimation of data with unknown distributions.
method Two fitting methods: EM algorithm and Bayesian non-parametric method using Gibbs sampler.
result Effective clustering and density estimation of data with unknown distributions.
Generative Adversarial Networks have surprising ability for generating sharp and realistic images, though they are known to suffer from the so-called mode collapse problem. In this paper, we propose a new GAN variant called Mixture Density GAN that while being capable of generating high-quality images, overcomes this p…