Mixed likelihood GPs improve model performance in human-in-the-loop experiments.
problem Lack of auxiliary information in traditional GPs for human responses.
method Propose mixed likelihood variational GPs to leverage auxiliary information.
result Modeling performance improvements across diverse applications.
A universal framework for constructing confidence sets using sequential likelihood mixing.
problem Constructing reliable confidence sets for realizable likelihood functions.
method Sequential likelihood mixing, integrating Bayesian inference and regret inequalities.
result Establishes fundamental connections and provable coverage guarantees for various inference techniques.
We present the Mixed Likelihood Gaussian process latent variable model (GP-LVM), capable of modeling data with attributes of different types. The standard formulation of GP-LVM assumes that each observation is drawn from a Gaussian distribution, which makes the model unsuited for data with e.g. categorical or nominal a…
The likelihood for the parameters of a generalized linear mixed model involves an integral which may be of very high dimension. Because of this intractability, many approximations to the likelihood have been proposed, but all can fail when the model is sparse, in that there is only a small amount of information availab…
Inference is typically intractable in high-treewidth undirected graphical models, making maximum likelihood learning a challenge. One way to overcome this is to restrict parameters to a tractable set, most typically the set of tree-structured parameters. This paper explores an alternative notion of a tractable set, nam…
New GP kernel handles mixed-categorical data, improving model accuracy.
problem Improving Gaussian process models for mixed-categorical data.
method Extends continuous exponential kernels to handle mixed-categorical variables.
result The proposed GP model gives higher likelihood and smaller residual error.
New ICA method for sources with mixed spectra.
problem Inaccurate separation of sources with temporal autocorrelations and mixed spectra.
method Estimates spectral density functions and line spectra using cubic splines and indicator functions, then maximizes the Whittle likelihood function.
result Outperforms existing ICA methods in simulations and EEG data applications.
New MMM captures hierarchical marketing effects and sign restrictions.
problem Measuring effectiveness of marketing activities with hierarchical structure and sign constraints.
method Proposes a constrained maximum likelihood approach using Hamiltonian Monte Carlo algorithm.
result Demonstrates superior performance on real datasets compared to multi-stage methods.
Expectation Maximization (EM) is among the most popular algorithms for maximum likelihood estimation, but it is generally only guaranteed to find its stationary points of the log-likelihood objective. The goal of this article is to present theoretical and empirical evidence that over-parameterization can help EM avoid …
Method completes mixed matrix from complex surveys with heterogeneous missingness.
problem Recovering a mixed dataframe matrix from complex survey sampling with different missingness patterns.
method Two-stage procedure: logistic regression for missingness modeling, and weighted log-likelihood maximization with low-rank constraint.
result The proposed method achieves sublinear convergence and shows superior performance compared to existing methods.
This work analyzes Gibbs samplers for Bayesian hierarchical models without dimensionality constraints.
problem Analyzing convergence properties of Gibbs samplers for Bayesian hierarchical models.
method Using Bayesian asymptotics and total variation mixing times, the study provides dimension-free convergence results.
result Dimension-free convergence results for Gibbs samplers targeting hierarchical models under random data-generating assumptions.
Neural models improve GLMMs for complex data.
problem Nonlinear relationships in grouped data.
method Replaced linear function with neural networks.
result Improved performance on synthetic and real-world data.
A scalable Bayesian inference method for mixed-effects models in systems biology.
problem Scalable Bayesian inference for complex hierarchical mixed-effects models in systems biology.
method Constructing amortized approximations of likelihood and posterior distributions, refined for each individual dataset.
result Our method is both fast and competitive in statistical accuracy compared to exact pseudomarginal Bayesian inference.
New method balances multivariate model fitting for mixed likelihoods.
problem Multivariate models often fit only a subset of observed variables.
method Lipschitz standardization for data preprocessing.
result Lipschitz standardization leads to more accurate multivariate models.
Paper develops a novel approach for classifying high-dimensional mixed data.
problem Handling datasets with both categorical and continuous variables of high dimensions.
method Location model with Gaussian conditional distributions, kernel smoothing for bandwidth choice, penalized likelihood estimation.
result Competitive performance of the proposed classifier demonstrated through simulations and real data.
New algorithm for estimating MLR parameters with non-Gaussian noise.
problem Estimating MLR parameters with non-Gaussian noise.
method Combining ADMM with EM algorithm idea.
result Our method outperforms EM algorithm in non-Gaussian noise case.
This paper is concerned with an important issue in finite mixture modelling, the selection of the number of mixing components. We propose a new penalized likelihood method for model selection of finite multivariate Gaussian mixture models. The proposed method is shown to be statistically consistent in determining of th…
Three RFF-based methods for nonlinear causal discovery in mixed data.
problem Nonlinear causal discovery in mixed data with computational constraints.
method FFML, TRFF, and FFCI methods for score-based, constraint-based, and hybrid causal discovery.
result FFML and TRFF methods provide complementary performance in causal discovery.
Paper extends LME models to allow sign constraints on coefficients with SDTN random effects.
problem Inference with sign constraints on random effects in LME models.
method Proposes SDTN distribution for random effects and develops likelihood-based approaches for estimation.
result Proposed constrained model improves real-world interpretations and achieves satisfactory performance.
New Riemannian optimization improves variance estimation in mixed models.
problem Challenges in estimating variance parameters in linear mixed models due to constraints.
method Formulated as an optimization problem on a Riemannian manifold, using Riemannian gradient and Hessian.
result Yields higher quality variance parameter estimates compared to existing methods.
Paper proves EM algorithm convergence for mixtures of discrete and continuous parameters.
problem Nontrivial convergence analysis for EM algorithms with mixed-integer parameters.
method Introduces conditions for EM convergence in mixed-integer optimization.
result Proves convergence of EM-based sparse Bayesian learning algorithm.
Identifies interpretable generative model for multivariate data.
problem Black-box architectures of deep generative models are often unidentified and difficult to interpret.
method Introduces Deep Discrete Encoder (DDE) Copula, a hierarchical binary latent variable model inside a copula framework.
result Establishes conditions for identification of DDE copula parameters and proves posterior consistency.
Partition Tree estimates conditional densities for mixed continuous and categorical variables.
problem Estimating conditional densities for mixed data types.
method Tree-based framework modeling conditional distributions as piecewise-constant densities on adaptive partitions, minimizing conditional negative log-likelihood.
result Improved probabilistic prediction compared to CART-style trees and state-of-the-art methods.
GBMixed boosts mixed models for clustered data, estimating mean and variance flexibly.
problem Flexible estimation of mean and variance components in clustered data.
method Gradient Boosting framework for linear mixed models with likelihood-based gradients.
result GBMixed accurately recovers complex nonlinear fixed effects and covariances.
Unified framework for SGMoE resolves estimation and selection issues.
problem Non-identifiability, coupled differential relations, and tight coupling in softmax-Gated models.
method Unified statistical framework with Voronoi-type loss functions and dendrograms of mixing measures.
result Consistent selection of the number of experts without model sweeps, optimal parameter rates under overfitting.
Modeling stock returns and volatility using a bivariate gamma generalized Laplace law.
problem Analyzing stock returns and volatility using a new statistical model.
method Maximum likelihood estimation for a bivariate generalized Laplace distribution, simplifying to linear regression.
result Explicit estimators derived with nonstandard convergence rates for certain parameter configurations.
Enhances MMSB for complex graph structures with HL-MRF priors.
problem Limited modeling of correlated graph structures in MMSB.
method HL-MRF as structured prior for mixed membership distributions.
result Improves log-likelihood by 15% on average across datasets.
We consider the problem of learning parameters of latent variable models from mixed (continuous and ordinal) data with missing values. We propose a novel Bayesian Gaussian copula factor (BGCF) approach that is consistent under certain conditions and that is quite robust to the violations of these conditions. In simulat…
We introduce a mixture model for censored durations (C-mix), and develop maximum likelihood inference for the joint estimation of the time distributions and latent regression parameters of the model. We consider a high-dimensional setting, with datasets containing a large number of biomedical covariates. We therefore p…
NPMLE estimator automatically chooses the right model complexity for Gaussian mixtures.
problem Learning mixture models and empirical Bayes estimation with non-convex likelihoods.
method Nonparametric maximum likelihood estimator (NPMLE) using complex-analytic techniques.
result NPMLE solution has O(logn) atoms with high probability, improving model complexity. Proposes a guaranteed regularization method for maximum likelihood estimation using gauge symmetry in Kullback-Leibler divergence.
problem Overfitting in maximum likelihood estimation.
method Introduces a regularization approach based on gauge symmetry in Kullback-Leibler divergence.
result The method provides a theoretically guaranteed optimal model without frequent hyperparameter tuning.
R package for Bayesian empirical likelihood sampling using HMC.
problem Sampling from non-convex Bayesian empirical likelihood posteriors.
method Hamiltonian Monte Carlo (HMC) algorithm for numerical integration.
result Efficient HMC sampling from BayesEL posteriors.
New method for accurate Bayesian inference in GLMMs for large-scale data.
problem Challenges in scalable Bayesian inference for GLMMs due to intractable marginal likelihood and high computational cost.
method Stochastic gradient Markov Chain Monte Carlo (SGMCMC) algorithm using Fisher's identity for gradient estimation.
result Accurate posterior inference in large-sample settings with properly calibrated uncertainty.
Gradient Boosted Mixed Models estimate mean and variance components for clustered data.
problem Limited flexibility in linear mixed models for complex settings.
method Gradient Boosting extended to mixed models with likelihood-based gradients and flexible base learners.
result Accurate recovery of variance components and improved predictive accuracy.
Develops a functional mix-of-experts model for multiclass classification.
problem Multiclass classification with univariate functional predictors.
method Functional mix-of-experts model with regularization and sparsity constraints.
result Regularized maximum likelihood estimation yields interpretable coefficient functions.
"Mixed Data" comprising a large number of heterogeneous variables (e.g. count, binary, continuous, skewed continuous, among other data types) are prevalent in varied areas such as genomics and proteomics, imaging genetics, national security, social networking, and Internet advertising. There have been limited efforts a…
Paper develops federated GLMM algorithms for analyzing hierarchical data.
problem Analyzing hierarchical data with non-independent observations in a federated setting.
method Developed two federated GLMM algorithms using Laplace and Gaussian Hermite approximations.
result Federated GLMM can handle hierarchical data and achieve comparable or superior performance.
The Tweedie Compound Poisson-Gamma model is routinely used for modeling non-negative continuous data with a discrete probability mass at zero. Mixed models with random effects account for the covariance structure related to the grouping hierarchy in the data. An important application of Tweedie mixed models is pricing …
We consider a jump-type Cox--Ingersoll--Ross (CIR) process driven by a standard Wiener process and a subordinator, and we study asymptotic properties of the maximum likelihood estimator (MLE) for its growth rate. We distinguish three cases: subcritical, critical and supercritical. In the subcritical case we prove weak …
Linear Mixed Models (LMMs) are important tools in statistical genetics. When used for feature selection, they allow to find a sparse set of genetic traits that best predict a continuous phenotype of interest, while simultaneously correcting for various confounding factors such as age, ethnicity and population structure…
Novel network model estimates mixed-membership structure with covariate information.
problem Estimating latent mixed-membership structure in networks with covariate information.
method Proposes a novel network model that incorporates both community information and node covariate similarities.
result Achieves optimal estimation accuracy for similarity matrix and mixed-membership.
This paper considers the problem of networks reconstruction from heterogeneous data using a Gaussian Graphical Mixture Model (GGMM). It is well known that parameter estimation in this context is challenging due to large numbers of variables coupled with the degeneracy of the likelihood. We propose as a solution a penal…
Improves likelihood-free inference by using a new sampling approach to avoid biased data collection.
problem Efficient Bayesian inference without likelihood evaluation for real-world datasets.
method Introduces Neural Proposal (NP) to sample simulation inputs i.i.d. for unbiased posterior inference.
result Demonstrates improved performance, especially for multi-modal posteriors, through experiments.
We study asymptotic properties of maximum likelihood estimators of drift parameters for a jump-type Heston model based on continuous time observations, where the jump process can be any purely non-Gaussian Lévy process of not necessarily bounded variation with a Lévy measure concentrated on (−1,∞). We prove stro…
Injective flows for star-like manifolds improve variational inference efficiency.
problem Efficiently modeling densities on star-like manifolds with exact Jacobian computation.
method Proposed injective flows for star-like manifolds with exact Jacobian computation.
result Exact Jacobian computation for star-like manifolds reduces computational cost to NFs.
We consider a stable Cox--Ingersoll--Ross process driven by a standard Wiener process and a spectrally positive strictly stable Lévy process, and we study asymptotic properties of the maximum likelihood estimator (MLE) for its growth rate based on continuous time observations. We distinguish three cases: subcritical, c…
We focus on the problem of unsupervised cell outlier detection and repair in mixed-type tabular data. Traditional methods are concerned only with detecting which rows in the dataset are outliers. However, identifying which cells are corrupted in a specific row is an important problem in practice, and the very first ste…
New method learns disentangled signals without prior or model constraints.
problem Learning disentangled signals from data without prior or model constraints.
method Minimizes conditional KL divergence using a sequential algorithm to learn de-mixing flow models.
result Method learns self-sufficient signals that can reconstruct missing values.