Improved PixelCNN model with faster training and better performance.
problem Generating high-quality images with tractable likelihood.
method Discretized logistic mixture likelihood, pixel conditioning, downsampling, short-cut connections, dropout regularization.
result State-of-the-art log likelihood results on CIFAR-10.
Proposes logistic-beta process for modeling dependent probabilities with beta marginals.
problem Limited work on flexible and computationally convenient stochastic process extensions for dependent random probabilities.
method Introduces logistic-beta process with logistic transformation and beta marginals, capable of modeling dependence in discrete and continuous domains.
result Logistic-beta processes enable effective posterior inference and design of computationally tractable dependent Bayesian nonparametric models.
Detecting and recovering labels in binomial logistic mixtures is challenging due to an information gap.
problem Detecting and recovering labels in binomial logistic mixtures
method Propose two feasibility-aware inference procedures
result Avoid misleading component selections and improve label probability calibration
Develops a semi-supervised learning method using exponential tilt mixture models.
problem Improves classification accuracy with labeled and unlabeled data.
method Extends logistic regression to exponential tilt modeling, derives maximum likelihood estimation, and proposes regularized estimation.
result Demonstrates improved prediction accuracy compared to existing methods.
The paper shows how variable discretization and cost-sensitive logistic regression improve credit scoring models on imbalanced data.
problem Bias in classification models on imbalanced datasets.
method Variable discretization and cost-sensitive logistic regression.
result Improves model performance on imbalanced credit scoring data and other domains.
ETM models improve efficiency in semi-supervised logistic regression.
problem Improving efficiency in logistic regression with limited labeled data.
method Developed exponential tilt mixture (ETM) models for semi-supervised estimation.
result ETM-based estimation demonstrates improved efficiency over supervised logistic regression.
Researchers show mixtures of ranking models are generally identifiable.
problem Understanding when and how parameters of mixtures of ranking models can be uniquely determined.
method Algebraic geometry framework applied to verify the number of solutions in polynomial systems.
result Popular mixtures of ranking models with two components are generically identifiable.
Study calculates tail risk for various mixture distributions.
problem Estimating tail risk for complex distribution mixtures.
method Analyzes tail conditional expectation for location-scale mixtures of elliptical distributions.
result Developed methods for calculating tail risk in various distributions.
Improved multi-speaker TTS using GANs and waveform loss.
problem Training acoustic models for neural vocoders in multi-speaker TTS systems.
method Proposed frameworks incorporating Wasserstein GAN with gradient penalty (WGAN-GP) and discretized mixture logistic loss (DML) into acoustic models trained with WaveNet.
result Acoustic models trained with WGAN-GP and DML loss achieve highest subjective evaluation scores in multi-speaker TTS.
We improve MoE models for classification with rigorous guarantees and practical methods.
problem Limited guarantees for stable maximum-likelihood training and model selection in softmax-gated MoE models.
method Derived a batch MM algorithm with closed-form updates, proved finite-sample rates, and developed a dendrogram selector.
result Achieved near-parametric optimal rates for parameter recovery and improved accuracy over baselines.
PbP strategy improves logistic model prediction with missing values.
problem Predicting with missing inputs in logistic models.
method Pattern-by-Pattern (PbP) strategy for logistic models with missing values.
result PbP accurately approximates Bayes probabilities under GPMM across various missing data scenarios.
VBphenoR uses variational Bayes for EHR-based patient phenotyping.
problem Phenotyping patients from EHR data for targeted treatments.
method Variational Bayes Gaussian Mixture Model (GMM) and logistic regression.
result Closed-form inference for efficient patient phenotype determination.
Combines linear acyclic model with logistic regression for causal structure estimation from mixed data.
problem Estimating causal structure from mixed continuous and discrete data.
method Defines a hybrid causal model combining linear acyclic model for continuous variables and logistic regression for discrete variables. Derives BIC scoring function for model selection. Employs a new discovery algorithm to learn causal structures without discretization.
result Empirically demonstrates the power of the method through simulations.
Sparse logistic regression recovers any discrete pairwise graph model.
problem Recovering the Markov graph of discrete pairwise graphical models.
method Maximum conditional log-likelihood with convex optimization.
result The algorithm can recover any arbitrary discrete pairwise graphical model.
HiSS sampling overcomes local mode traps in rugged discrete spaces.
problem Sampling multimodal discrete distributions with gradient-based methods.
method Integrates Metropolis-within-Gibbs framework with logistic convolution.
result HiSS outperforms alternatives on various tasks, including Ising models and binary neural networks.
The paper establishes convergence rates for MoE models in classification problems.
problem Understanding the behavior of MoE models in classification settings.
method Established convergence rates for density and parameter estimation in softmax gating multinomial logistic MoE models.
result Parameter estimation rates are significantly improved with a novel modified softmax gating function.
Refined analysis of Mitra's algorithm for discrete mixtures.
problem Classifying general discrete mixture distribution models.
method Spectral clustering tailored to bipartite stochastic block models.
result Improved separation conditions for probability distributions.
Improved regret bound for MNL MDPs with variance-aware approach.
problem Optimal reinforcement learning for MNL MDPs with structured variance.
method Introducing a problem-dependent constant measuring average variance, proposing an algorithm with improved regret bound.
result Minimax optimal regret bound of O ( d H 2 σ ˉ T T ) O(dH^2\barσ_T\sqrt{T}) O ( d H 2 σ ˉ T T ) for structured MDPs. Proposes efficient Bayesian logistic regression for large sparse datasets.
problem Infeasibility of theoretical Bayesian methods for large sparse feature sets.
method Low complexity analytical approximations for sparse online logistic and probit regressions.
result Empirical results show superior performance compared to more complex methods.
Feature selection improves learning from mixtures of discrete variables.
problem Learning mixtures of discrete random variables, especially in unreliable crowdsourcing.
method Algorithm based on mutual information to rank workers and induce a low-order statistical model.
result Improvement in real data sets can be substantial.
A new method uncovers discrete and continuous factors in gene expression data.
problem Jointly identifying discrete and continuous factors of variability without supervision.
method cpl-mixVAE framework using multiple interacting networks.
result The method successfully uncovers discrete and continuous factors in gene expression data.
Flexible nonparametric model for discrete choice analysis.
problem Modeling heterogeneity in discrete choice data without fixed component limits.
method Dirichlet process mixture model with expectation maximisation algorithm.
result Proposed model outperforms latent class MNL and mixed MNL models in both fit and predictive ability.
Paper optimizes clustering for multi-layer networks and discrete mixtures.
problem Optimizing clustering in multi-layer networks and discrete mixtures.
method Two-stage method: tensor-based initialization and likelihood-based refinement.
result Achieves minimax optimal error rate for multi-layer networks and discrete mixtures.
We show how to use a variational approximation to the logistic function to perform approximate inference in Bayesian networks containing discrete nodes with continuous parents. Essentially, we convert the logistic function to a Gaussian, which facilitates exact inference, and then iteratively adjust the variational par…
The paper tackles learning mixtures of two multinomial logits, showing identifiability and presenting an algorithm.
problem Learning an arbitrary mixture of two multinomial logits.
method Reduction to solving a system of univariate quartic equations, followed by an algorithm using polynomial and linear samples.
result Identifiability of the mixture models may only fail on an algebraic variety of negligible measure.
The Riemann-Theta Boltzmann machine's visible sector is sampled using a discrete multi-variate Gaussian.
problem Sampling the visible sector of the Riemann-Theta Boltzmann machine.
method Discrete multi-variate Gaussian over the hidden state space.
result The visible sector probability density function is an infinite mixture of multi-variate Gaussians.
Machine learning improves joint default assessment by capturing non-linear dependencies.
problem Capturing non-linear dependencies among covariates for accurate joint default assessment.
method Application of machine learning techniques to credit card dataset, comparing with logistic regression.
result Machine learning outperforms logistic regression in assessing portfolio riskiness.
Early stopping improves gradient descent's performance in Gaussian mixture classification.
problem Gradient descent's implicit bias can be suboptimal in overparameterised classification.
method Early stopping combined with sharp upper and matching lower bounds.
result Early-stopped GD achieves minimax-optimal risk for Gaussian mixture models.
The paper introduces flat-topped PDFs for better fitting machine learning models.
problem Improving goodness of fit in machine learning models.
method Developed a new PDF based on the Fermi-Dirac or logistic function for adaptability.
result Flat-topped PDFs enhance model simplicity and fit quality.
Two EM algorithms estimate prior distributions in mixture of linear regressions.
problem Estimating prior distributions in mixture of linear regressions.
method Two EM algorithms: one for continuous priors, one for discrete priors.
result Both algorithms accurately estimate prior distributions and the number of clusters.
Paper develops a consistent estimator for discrete mixture models.
problem Estimating mixing probability density in discrete mixture models.
method Develops an L 1 L_1 L 1 consistent estimator under specific conditions. result Consistent estimation of mixing probability density for every density f f f . Improved logistic MoE with sigmoid gate shows better sample efficiency.
problem Improving sample efficiency in logistic MoE models.
method Comprehensive analysis of multinomial logistic MoE with modified sigmoid gate, incorporating temperature parameter and using Euclidean score.
result The sigmoid gate leads to lower sample complexity than softmax gate for both parameter and expert estimation.
A new approach for signal parametrization, which consists of a specific regression model incorporating a discrete hidden logistic process, is proposed. The model parameters are estimated by the maximum likelihood method performed by a dedicated Expectation Maximization (EM) algorithm. The parameters of the hidden logis…
Mixture model-based clustering, usually applied to multidimensional data, has become a popular approach in many data analysis problems, both for its good statistical properties and for the simplicity of implementation of the Expectation-Maximization (EM) algorithm. Within the context of a railway application, this pape…
In this paper we propose a novel framework for the construction of sparsity-inducing priors. In particular, we define such priors as a mixture of exponential power distributions with a generalized inverse Gaussian density (EP-GIG). EP-GIG is a variant of generalized hyperbolic distributions, and the special cases inclu…
New model clusters discrete time series data.
problem Handling discreteness and time series properties in data.
method Finite mixture model with INAR type models.
result Demonstrated clustering on real data.
Estimates customer segments from continuous marketing data streams.
problem Analyzing large, continuously updated marketing data streams.
method oFMLR: online estimation of finite mixture of logistic regression models.
result oFMLR provides interpretable customer segment clustering.
The study characterizes learning Gaussian mixtures using GLMs in high dimensions.
problem Learning Gaussian mixtures with generalised linear models in high-dimensional settings.
method Empirical risk minimization with convex loss and regularisation.
result Exact asymptotics of the ERM estimator for Gaussian mixtures in high dimensions.
A new approach for feature extraction from time series is proposed in this paper. This approach consists of a specific regression model incorporating a discrete hidden logistic process. The model parameters are estimated by the maximum likelihood method performed by a dedicated Expectation Maximization (EM) algorithm. …
Develops a functional mix-of-experts model for multiclass classification.
problem Multiclass classification with univariate functional predictors.
method Functional mix-of-experts model with regularization and sparsity constraints.
result Regularized maximum likelihood estimation yields interpretable coefficient functions.
A new method for categorical variational inference using discrete normalizing flows.
problem Challenges in optimizing variational approximations for discrete latent variables.
method Differentiable reparameterization using a mixture of discrete normalizing flows.
result Improves optimization of evidence lower bound and reduces sensitivity to hyperparameters.
A new sampling method accelerates inference in discrete probabilistic models.
problem Slow convergence of Markov chain Monte Carlo algorithms in discrete probabilistic models.
method Proposes a mixture of product distributions using semigradient information to accelerate convergence.
result Combining the new sampler with existing ones improves inference in various models.
Plug-and-play L-GM-AMP improves CS recovery for any i.i.d. source prior.
problem Efficiently recovering signals from compressed measurements with unknown priors.
method Deep learning with Gaussian-mixture model to approximate source prior, combined with learned denoising.
result L-GM-AMP achieves state-of-the-art performance without prior knowledge of source distribution.
Generative model predicts vehicle faults up to 1000 hours in advance.
problem Forecasting vehicle faults for predictive maintenance.
method Generative model trained on US Army data, incorporating real-world factors.
result Highly accurate predictions of time to first fault.
We study the problem of learning a distribution from samples, when the underlying distribution is a mixture of product distributions over discrete domains. This problem is motivated by several practical applications such as crowd-sourcing, recommendation systems, and learning Boolean functions. The existing solutions e…
Study shows how over-parameterized classifiers can still perform well on noisy data.
problem Understanding how maximum margin classifiers perform in over-parameterized settings with noisy data.
method Analyzes maximum margin classifiers on sub-Gaussian mixtures, providing risk bounds.
result Characterizes conditions for 'benign overfitting' in linear classification problems.
Regularization helps improve classification of noisy high-dimensional data.
problem Classifying high-dimensional noisy Gaussian mixture with limited oracle knowledge.
method Analysis of regularized convex classifiers including ridge, hinge, and logistic regression.
result Regularization can reach Bayes-optimal performance under certain conditions.
We present the discrete infinite logistic normal distribution (DILN), a Bayesian nonparametric prior for mixed membership models. DILN is a generalization of the hierarchical Dirichlet process (HDP) that models correlation structure between the weights of the atoms at the group level. We derive a representation of DILN…