A new method combines machine learning with mixed-effects models for better repeated measurement analysis.
problem Inference of linear coefficients in partially linear mixed-effects models with complex interactions and high-dimensional variables.
method Double machine learning approach to estimate nonparametrically nonlinear variables, then use standard linear mixed-effects techniques to estimate the linear coefficient.
result The estimated fixed effects coefficient converges at the parametric rate and is semiparametrically efficient.
GBMixed boosts mixed models for clustered data, estimating mean and variance flexibly.
problem Flexible estimation of mean and variance components in clustered data.
method Gradient Boosting framework for linear mixed models with likelihood-based gradients.
result GBMixed accurately recovers complex nonlinear fixed effects and covariances.
We develop an HMC algorithm to easily marginalize random effects in LMMs.
problem Bayesian inference in LMMs is challenging, especially marginalizing random effects.
method Developed an HMC algorithm to marginalize random effects in LMMs efficiently.
result Marginalization is always beneficial when applicable and improves various models, especially cognitive science models.
Many scientific and engineering challenges -- ranging from pharmacokinetic drug dosage allocation and personalized medicine to marketing mix (4Ps) recommendations -- require an understanding of the unobserved heterogeneity in order to develop the best decision making-processes. In this paper, we develop a hypothesis te…
Neural models improve GLMMs for complex data.
problem Nonlinear relationships in grouped data.
method Replaced linear function with neural networks.
result Improved performance on synthetic and real-world data.
New Riemannian optimization improves variance estimation in mixed models.
problem Challenges in estimating variance parameters in linear mixed models due to constraints.
method Formulated as an optimization problem on a Riemannian manifold, using Riemannian gradient and Hessian.
result Yields higher quality variance parameter estimates compared to existing methods.
A scalable model for high-dimensional longitudinal data.
problem Modeling high-dimensional, non-linear, time-varying longitudinal data.
method LMM-VAE, combining linear mixed models and amortized variational inference.
result Competitive performance across simulated and real-world datasets.
This paper concerns the development of an inferential framework for high-dimensional linear mixed effect models. These are suitable models, for instance, when we have n repeated measurements for M subjects. We consider a scenario where the number of fixed effects p is large (and may be larger than M), but the n…
Gradient Boosted Mixed Models estimate mean and variance components for clustered data.
problem Limited flexibility in linear mixed models for complex settings.
method Gradient Boosting extended to mixed models with likelihood-based gradients and flexible base learners.
result Accurate recovery of variance components and improved predictive accuracy.
Proposes an efficient algorithm for mHealth that makes real-time physical activity suggestions.
problem Training complex models efficiently in real-time mHealth applications.
method Streamlined empirical Bayes procedure for fitting linear mixed effects models.
result Improves accuracy and speed over state-of-the-art approaches in mHealth applications.
Paper extends LME models to allow sign constraints on coefficients with SDTN random effects.
problem Inference with sign constraints on random effects in LME models.
method Proposes SDTN distribution for random effects and develops likelihood-based approaches for estimation.
result Proposed constrained model improves real-world interpretations and achieves satisfactory performance.
Method improves regression models using unlabeled data.
problem Improving predictive performance of regression models with limited labeled data.
method Mixed semi-supervised generalized-linear-regression with different mixing mechanisms.
result Integrating unlabeled data consistently improves predictive performance.
The likelihood for the parameters of a generalized linear mixed model involves an integral which may be of very high dimension. Because of this intractability, many approximations to the likelihood have been proposed, but all can fail when the model is sparse, in that there is only a small amount of information availab…
GLMM trees identify subgroups with different growth patterns in longitudinal data.
problem Identifying subgroups with distinct growth trajectories in longitudinal studies.
method Extended GLMM trees for longitudinal data.
result Extended GLMM trees outperform other methods in accuracy and speed.
BayesBoost combines boosting and Bayesian methods for linear mixed models, improving uncertainty estimation and variable selection.
problem Lack of straightforward uncertainty estimation for parameters in high-dimensional linear mixed models.
method BayesBoost: Combines boosting and Bayesian inference for linear mixed models.
result Improves uncertainty estimation and variable selection in linear mixed models.
New method optimizes mixed integer optimization for hierarchical modeling of clustered and longitudinal data.
problem Optimizing subset selection in hierarchical models with clustered and longitudinal data.
method Distribution-free mixed-integer optimization approach for cluster-aware regression.
result The method efficiently solves problems within minutes and outperforms traditional models in generating sparse solutions with high predictive power.
Gradient EM converges exponentially to optimal solution in agnostic mixtures.
problem Fitting k parametric functions to given data points without a generative model. method Gradient EM algorithm for agnostic mixtures of arbitrary parametric functions.
result Gradient EM converges exponentially to population loss minimizers with high probability.
We solve structure learning for cyclic linear causal models using observational data.
problem Learning the structure of cyclic linear causal models from observational data.
method Assuming simple graphs, we use a criterion for distributional equivalence and implement a greedy search method.
result We show that simple cyclic models are of expected dimension and justify score-based methods for structure learning.
Paper develops federated GLMM algorithms for analyzing hierarchical data.
problem Analyzing hierarchical data with non-independent observations in a federated setting.
method Developed two federated GLMM algorithms using Laplace and Gaussian Hermite approximations.
result Federated GLMM can handle hierarchical data and achieve comparable or superior performance.
MOMENT selects and estimates mixed-effects models using moment identities.
problem Selecting and estimating random-effects covariance matrix and fixed-effects coefficients in multiresponse linear mixed-effects models.
method MOMENT is a stage-wise moment-based framework that reduces the random-effects selection problem to a smooth constrained convex optimization problem.
result MOMENT performs competitively and can outperform separate univariate analyses for correlated responses.
Study improves maize yield prediction using BNs with mixed-effects models.
problem Limited causal inference in agronomic data models.
method Integrates random effects into Bayesian networks, leveraging hierarchical data structure.
result Significantly reduces maize yield prediction error from 28% to 17%.
Causal inference concerns the identification of cause-effect relationships between variables. However, often only linear combinations of variables constitute meaningful causal variables. For example, recovering the signal of a cortical source from electroencephalography requires a well-tuned combination of signals reco…
New method selects sparse predictors in large LMMs.
problem Sparse learning for LMMs is computationally infeasible for large datasets.
method Developed an ℓ0 regularized method with coordinate descent and local search algorithms. result Method selects thousands of predictors in seconds to minutes.
Combines boosting with Gaussian process and mixed effects models.
problem Model misspecifications and independence assumptions in boosting.
method Relaxes zero or linearity assumption in Gaussian process and mixed effects models, and independence assumption in boosting.
result Increased prediction accuracy compared to existing approaches.
Mix-IRLS solves imbalanced mixed linear regression problems efficiently.
problem Imbalanced mixed linear regression problems.
method Sequential robust regression approach.
result Mix-IRLS outperforms other methods on imbalanced mixtures and real-world datasets.
Paper addresses online identification and clustering for mixed linear regression models.
problem Online identification and clustering of mixed linear regression models.
method Introduces two online identification algorithms based on the EM principle, proving global convergence without i.i.d. data assumptions.
result Global convergence of the proposed algorithms for mixed linear regression models.
Integrates neural encoders into GLMMs for multimodal data analysis.
problem Scalable Bayesian inference for GLMMs assumes low-dimensional tabular predictors and does not handle high-dimensional modalities.
method Jointly learns modality-specific neural encoders with GLMM objective, performs variance-corrected stochastic-gradient MCMC.
result Preserves interpretable fixed and random effects while scaling to large longitudinal datasets.
Paper analyzes agnostic learning of mixed linear regression without generative models.
problem Learning mixed linear regression without assuming stochastic generation.
method Expectation Maximization (EM) and Alternating Minimization (AM) algorithms.
result AM and EM algorithms converge to population loss minimizers under standard conditions.
The optimization of expensive to evaluate, black-box, mixed-variable functions, i.e. functions that have continuous and discrete inputs, is a difficult and yet pervasive problem in science and engineering. In Bayesian optimization (BO), special cases of this problem that consider fully continuous or fully discrete doma…
New method for Bayesian inference on large datasets.
problem Scalable sampling for Bayesian generalized linear mixed models on large datasets.
method Mirror Langevin dynamics with data subsampling, post-processing for variance estimation.
result Asymptotic, order-wise correct estimation of posterior variance.
We derive streamlined mean field variational Bayes algorithms for fitting linear mixed models with crossed random effects. In the most general situation, where the dimensions of the crossed groups are arbitrarily large, streamlining is hindered by lack of sparseness in the underlying least squares system. Because of th…
Study on elasticity with mixed boundary conditions, proving spectral asymptotics.
problem Analyzing spectral asymptotics for linear elasticity with mixed boundary conditions.
method Established two-term spectral asymptotics for linear elasticity on smooth compact manifolds.
result Verification of general formulae through explicit examples in 2D and 3D.
Develops RF-GLS for binary geospatial data.
problem Challenges in extending RF to binary geospatial data.
method Proposes RF-GLS for binary data, embedding it in generalized mixed effects models.
result Establishes consistency of RF-GP for mean function and covariate effect estimation.
NoLimits.jl: Flexible and Composable Nonlinear Mixed-Effects Modeling in Julia
problem Flexible and composable nonlinear mixed-effects modeling
method Macro-based modeling language and unified interface
result Substantially expand the range of nonlinear mixed-effects models
The study prevents model collapse in overparameterized linear regression by mixing real and synthetic labels.
problem Preventing model collapse in overparameterized linear regression.
method Iterative mixing of real and synthetic labels, deriving generalization error formulae.
result Optimal mixing ratio converges to the reciprocal of the golden ratio for isotropic features.
metabeta uses neural networks to speed up Bayesian mixed-effects regression.
problem Bayesian mixed-effects regression is computationally expensive.
method metabeta is a neural network model that pre-trains to estimate posterior distributions.
result metabeta achieves comparable performance to MCMC at a fraction of the time.
Spectral methods improve signal recovery in mixed GLMs with precise asymptotics.
problem Estimating multiple signals from unlabeled observations in mixed GLMs.
method Developed exact asymptotics for spectral methods in a proportional regime.
result Optimized spectral method combined with a linear estimator minimizes estimation error.
Study compares neural and statistical models for Parkinson's disease progression from voice data.
problem Difficult statistical analysis of longitudinal voice biomarkers due to subject correlation, small cohorts, and varied disease trajectories.
method Evaluated Neural Mixed Effects (NME), Generalized Neural Network Mixed Models (GNMMs), and semi-parametric Generalized Additive Mixed Models (GAMMs).
result GAMMs achieve stronger predictive performance and retain interpretable smooth effects and subject-level structure.
Novel GLMMNet model tackles high-cardinality categorical features in actuarial applications.
problem Inadequate encoding methods for high-cardinality categorical features in actuarial data.
method Generalised Linear Mixed Model Neural Network (GLMMNet) integrating a generalised linear mixed model in a deep learning framework.
result GLMMNet often outperforms or performs comparably with entity embedded neural networks, providing transparency.
We consider the problem of solving mixed random linear equations with k components. This is the noiseless setting of mixed linear regression. The goal is to estimate multiple linear models from mixed samples in the case where the labels (which sample corresponds to which model) are not observed. We give a tractable a…
The spectral asymptotics for linear elasticity with mixed boundary conditions are shown to be old results.
problem Analyzing the spectral asymptotics for linear elasticity with mixed boundary conditions.
method Demonstrating that the results are essentially old well-known results by other authors.
result The spectral asymptotics results for linear elasticity with mixed boundary conditions are shown to be old results by other authors.
New Krylov subspace methods speed up mixed-effects models with crossed random effects.
problem Slow computations for high-dimensional crossed random effects in mixed-effects models.
method Krylov subspace-based methods for generalized mixed-effects models with cross effects.
result Speedups by factors of up to 10,000 in computations for mixed-effects models.
Study integrates reliability constraints into generation planning models.
problem Challenges in integrating reliability constraints with generation planning models.
method Leverages a weighted oblique decision tree (WODT) technique to embed reliability verification constraints.
result Demonstrates effectiveness in achieving reliable and optimal planning solutions.
Improved spatial prediction for massive datasets using SME model.
problem Efficiently estimating parameters in massive spatial datasets.
method Spatial Mixed Effects (SME) model with AECM algorithm for flexibility.
result Improved estimation without sacrificing prediction accuracy.
SBMs learn manifold-like structures by mixing samples with a non-conservative field.
problem How SBMs learn data distributions on low-dimensional manifolds.
method Investigating linear approximations and subspaces of local feature vectors during diffusion.
result SBMs mix samples by a non-conservative field within the manifold, maintaining manifold-like structure.
Efficient Bayesian LMM framework for high-dimensional longitudinal data.
problem Scalability and dependence in high-dimensional longitudinal data.
method Partitioned empirical Bayes ECM algorithm for scalable MAP estimation.
result Identification of genes and clinical factors associated with a lupus biomarker.
Transformers improve Alzheimer's disease progression prediction by accounting for irregular biomarker histories.
problem Difficult prediction of medium-horizon Alzheimer's disease progression due to tied clinical scores and irregular biomarker observations.
method Developed a residual gap-aware transformer that combines statistical reference with transformer-based residual learning.
result The proposed model reduces mean error and improves prediction-observation correlation compared to baseline models.
Mixed effects (ME) models inform a vast array of problems in the physical and social sciences, and are pervasive in meta-analysis. We consider ME models where the random effects component is linear. We then develop an efficient approach for a broad problem class that allows nonlinear measurements, priors, and constrain…