Bayesian approach improves network lasso for multi-task learning.
problem Improving the determination of relational coefficients in network lasso.
method Proposes a Bayesian approach to solve multi-task learning problems using network lasso.
result Objective determination of relational coefficients through Bayesian estimation.
The Bayesian Lasso is constructed in the linear regression framework and applies the Gibbs sampling to estimate the regression parameters. This paper develops a new sparse learning model, named the Bayesian Lasso Sparse (BLS) model, that takes the hierarchical model formulation of the Bayesian Lasso. The main differenc…
Scalable Bayesian LASSO using variational inference for large p and n.
problem Large-scale regression with sparsity constraints.
method Variational Bayesian inference with scale mixtures of Normal distributions.
result The method achieves comparable performance to Bayesian LASSO but at a lower computational cost.
The fused lasso penalizes a loss function by the L1 norm for both the regression coefficients and their successive differences to encourage sparsity of both. In this paper, we propose a Bayesian generalized fused lasso modeling based on a normal-exponential-gamma (NEG) prior distribution. The NEG prior is assumed in…
High-dimensional feature selection arises in many areas of modern science. For example, in genomic research we want to find the genes that can be used to separate tissues of different classes (e.g. cancer and normal) from tens of thousands of genes that are active (expressed) in certain tissue cells. To this end, we wi…
In this article, we propose a new class of priors for Bayesian inference with multiple Gaussian graphical models. We introduce fully Bayesian treatments of two popular procedures, the group graphical lasso and the fused graphical lasso, and extend them to a continuous spike-and-slab framework to allow self-adaptive shr…
MicroRNAs (miRNAs) are small RNA molecules composed of 19-22 nt, which play important regulatory roles in post-transcriptional gene regulation by inhibiting the translation of the mRNA into proteins or otherwise cleaving the target mRNA. Inferring miRNA targets provides useful information for understanding the roles of…
PliableBVS extends Bayesian lasso for modeling interactions with modifying variables.
problem Modeling interactions between large and small sets of variables, especially in omics studies.
method Bayesian variable selection with spike-and-slab priors and hierarchical structure.
result PliableBVS outperforms pliable lasso in identifying active main and interaction effects.
Bayesian method improves portfolio management with limited data.
problem Estimating covariance or precision matrix for large portfolios is challenging.
method Bayesian graphical LASSO for precision matrix estimation.
result The Bayesian approach outperforms non-Bayesian methods in stability and precision matrix estimation.
New method reduces bias in sparse Bayesian learning.
problem High sparsity in statistical models leads to significant bias.
method Variable-coefficient ℓ1 penalty with hyperpriors. result Reduces bias in sparse Bayesian learning.
Complex network reconstruction is a hot topic in many fields. Currently, the most popular data-driven reconstruction framework is based on lasso. However, it is found that, in the presence of noise, lasso loses efficiency for weighted networks. This paper builds a new framework to cope with this problem. The key idea i…
Bayesian pliable lasso with horseshoe prior models interactions in GLMs with missing data.
problem Modeling interactions in sparse regression problems with missing responses.
method Bayesian pliable lasso with hierarchical horseshoe prior for sparsity and uncertainty quantification.
result Advantages over existing methods in recovering complex interaction patterns under incomplete data.
New method corrects selection bias in post-selective inference for Group LASSO.
problem Inference after Group LASSO selection is unreliable.
method Develops a consistent, post-selective Bayesian method to adjust for selection bias.
result Corrects bias in recovering effects of selected variables.
Bayesian GAMs improve predictive performance for high-dimensional data.
problem Sparse regularization in GAMs leads to excess shrinkage and difficulty in selecting nonlinear effects.
method Developed a novel spike-and-slab LASSO prior and scalable EM-Coordinate Descent algorithm.
result Improved predictive and computational performance compared to existing models.
Bayesian rLASSO improves model selection and prediction.
problem Improving model selection and prediction in linear regression.
method A Bayesian formulation of the rLASSO problem using inverse Laplace priors and scale mixture distributions.
result Bayesian rLASSO outperforms classical rLASSO in estimation, prediction, and variable selection.
Study evaluates scikit-learn regularization frameworks for machine learning models.
problem Choosing the right regularization framework for applied machine learning models.
method Empirical evaluation of four canonical frameworks (Ridge, Lasso, ElasticNet, Post-Lasso OLS) across 134,400 simulations.
result Lasso recall is highly fragile under multicollinearity; at high condition numbers (kappa) and low SNR, Lasso recall collapses to 0.18 while ElasticNet maintains 0.93.
We investigate the choice of tuning parameters for a Bayesian multi-level group lasso model developed for the joint analysis of neuroimaging and genetic data. The regression model we consider relates multivariate phenotypes consisting of brain summary measures (volumetric and cortical thickness values) to single nucleo…
Novel Bayesian method for high-dimensional count data prediction.
problem Count data in high-dimensional settings requires feature selection.
method Pseudo-Bayesian framework with scaled Student prior and exponential weights.
result Strong performance compared to Lasso in various settings.
We consider a joint processing of n independent sparse regression problems. Each is based on a sample (yi1,xi1)...,(yim,xim) of m \iid observations from $y_{i1}=x_{i1}\tβ_i+\eps_{i1}$, yi1∈R, xi1∈Rp, i=1,...,n, and $\eps_{i1}\dist N(0,\sig^2)$, say. p is large enough so that the…
We describe a simple, efficient, permutation based procedure for selecting the penalty parameter in the LASSO. The procedure, which is intended for applications where variable selection is the primary focus, can be applied in a variety of structural settings, including generalized linear models. We briefly discuss conn…
The thesis tackles structure learning and parameter estimation for PGMs using penalized maximum likelihood methods.
problem Recovering the true structure of probabilistic graphical models (PGMs) for decision-making and interpretation.
method Penalized maximum likelihood estimation with the LASSO penalty.
result The approach successfully recovers the true structure of PGMs, including Bayesian networks and continuous time Bayesian networks, for both complete and incomplete data.
Random forests and LASSO methods improve small area estimation using auxiliary data.
problem Estimating household consumption in small areas with limited sampled data.
method Model-based small area estimation using random forests and LASSO with auxiliary information.
result Bayesian shrinkage performed best in terms of bias, MSE, and prediction interval coverages.
Bayesian method for estimating functional graphical models from neuroimaging data.
problem Estimating dependence structures from functional data in neuroscience.
method Fully Bayesian regularization scheme, including direct Bayesian analog of functional graphical lasso and graphical horseshoe.
result Insight into brain compensation after traumatic brain injury.
In this paper, we consider modelling interaction between a set of variables in the context of time series and high dimension. We suggest two approaches. The first is similar to the neighborhood lasso when the lasso model is replaced by a support vector machine (SVMs). The second is a restricted Bayesian network adapted…
Bayesian Tobit model tackles high-dimensional censored data with Horseshoe prior.
problem High-dimensional censored data with unknown bounds.
method Horseshoe prior for shrinkage, data augmentation for Gibbs sampling.
result Established posterior consistency and concentration rates for Bayesian Tobit models.
Bayesian method discovers PDEs with variable coefficients robustly.
problem Discovering PDEs from noisy data is challenging.
method Bayesian sparse learning with tBGL-SS and Gibbs sampler.
result Method enhances robustness and model selection criteria.
New method recovers relative rates in spatial compositional data from IMS.
problem Challenges in analyzing spatial data from IMS due to competitive sampling.
method Hierarchical Variational Graph Fused Lasso using heavy-tailed graphical lasso prior and automatic differentiation variational inference.
result Our method outperforms state-of-the-practice point estimate methodologies in IMS and has superior posterior coverage.
Paper proposes new Bayesian neural network models for efficient learning.
problem Efficient learning and model compression in deep neural networks.
method Proposes Spike-and-Slab Group Lasso (SS-GL) and Spike-and-Slab Group Horseshoe (SS-GHS) priors for structured sparsity in Bayesian neural networks.
result Establishes competitive performance in prediction accuracy, model compression, and inference latency compared to baseline models.
Statistical models with constrained probability distributions are abundant in machine learning. Some examples include regression models with norm constraints (e.g., Lasso), probit, many copula models, and latent Dirichlet allocation (LDA). Bayesian inference involving probability distributions confined to constrained d…
Discovering interaction effects on a response of interest is a fundamental problem faced in biology, medicine, economics, and many other scientific disciplines. In theory, Bayesian methods for discovering pairwise interactions enjoy many benefits such as coherent uncertainty quantification, the ability to incorporate b…
Bayesian transfer learning improves predictive performance with limited source data.
problem Improving statistical procedures with limited target and source datasets.
method Total risk prior for joint parameter distribution, Bayesian Lasso, model averaging, Gibbs sampling.
result Superior predictive performance compared to frequentist baseline, especially with limited source data.
Develops a model for RNA-seq data clustering.
problem Challenges in clustering RNA-seq count data with variable selection.
method Sparse negative binomial mixture model with lasso or fused lasso regularization.
result Superior performance in clustering accuracy, feature selection, and biological interpretation.
A new method learns DAG structures from data without false edges.
problem Learning DAG structures from observational data is hard due to combinatorial search space and non-sparse solutions.
method Developed NOTEARS-AL using adaptive Lasso to ensure sparsity and rule out false edges.
result NOTEARS-AL outperforms NOTEARS in synthetic and real-world datasets.
Sparse convex clustering is to cluster observations and conduct variable selection simultaneously in the framework of convex clustering. Although a weighted L1 norm is usually employed for the regularization term in sparse convex clustering, its use increases the dependence on the data and reduces the estimation acc…
HALO learns to prune neural networks by adaptively shrinking weights.
problem Sparsity and model size in deep neural networks.
method Bayesian hierarchical models and trainable parameters for adaptive sparsification.
result HALO learns to create highly sparse networks with significant performance gains.
For large scale on-line inference problems the update strategy is critical for performance. We derive an adaptive scan Gibbs sampler that optimizes the update frequency by selecting an optimum mini-batch size. We demonstrate performance of our adaptive batch-size Gibbs sampler by comparing it against the collapsed Gibb…
A new framework detects changepoints in complex data.
problem Detecting structural changes in data with various patterns and trends.
method Iteratively Reweighted Fused Lasso (IRFL) for L0 model selection.
result IRFL achieves accurate changepoint detection across various challenging scenarios.
One of the fundamental tasks of science is to find explainable relationships between observed phenomena. One approach to this task that has received attention in recent years is based on probabilistic graphical modelling with sparsity constraints on model structures. In this paper, we describe two new approaches to Bay…
A common strategy for sparse linear regression is to introduce regularization, which eliminates irrelevant features by letting the corresponding weights be zeros. However, regularization often shrinks the estimator for relevant features, which leads to incorrect feature selection. Motivated by the above-mentioned issue…
We analyze mixing times of three DA algorithms for regression models.
problem Analyzing mixing times of data augmentation algorithms for regression models.
method Modified conductance-based method to study mixing times of Gibbs samplers.
result Prove non-asymptotic polynomial upper bounds on mixing times for DA algorithms.
ADVI speeds up Bayesian inference for bridge regression models.
problem Slow MCMC for large datasets in bridge regression.
method Automatic Differentiation Variational Inference (ADVI) for Bayesian inference.
result ADVI implementation speeds up inference for large datasets.
The efficacy of family-based approaches to mixture model-based clustering and classification depends on the selection of parsimonious models. Current wisdom suggests the Bayesian information criterion (BIC) for mixture model selection. However, the BIC has well-known limitations, including a tendency to overestimate th…
New bounds for Lasso and Group Lasso in high dimensions derived.
problem Estimation error bounds for Lasso and Group Lasso in high-dimensional settings.
method Recent advances in high-dimensional statistics to derive new L2 estimation upper bounds.
result Bounds match optimal minimax rate for Lasso and improve over existing results for Group Lasso.
PLS-Lasso integrates dimension reduction into regression for financial index tracking.
problem Dimension reduction and regression are traditionally treated separately in multivariate data analysis.
method PLS-Lasso integrates dimension reduction directly into the regression process, presenting two formulations: PLS-Lasso-v1 and PLS-Lasso-v2.
result PLS-Lasso-v1 and PLS-Lasso-v2 outperform Lasso in financial index tracking.
The paper examines Adaptive Lasso and Transfer Lasso, highlighting their differences and proposing a new method.
problem Comparing and contrasting Adaptive Lasso and Transfer Lasso.
method Theoretical analysis of asymptotic properties and introduction of a new method.
result The Transfer Lasso method reduces non-asymptotic estimation errors compared to Adaptive Lasso.
Bayesian autoencoders discover physics from noisy data.
problem Challenges in identifying governing equations and coordinates from noisy, low-data real-world data.
method Bayesian SINDy autoencoders with hierarchical Bayesian sparsifying prior and adaptive empirical Bayesian method.
result Better physics discovery with lower data and fewer training epochs, along with valid uncertainty quantification.
Bayesian model improves cure fraction estimation in survival analysis.
problem Estimating cure fraction in survival analysis.
method Hierarchical Bayesian framework for semiparametric mixture cure model.
result Improved accuracy in capturing unobserved heterogeneity.
DFR reduces the computational cost of sparse-group lasso and adaptive sparse-group lasso.
problem Sparse-group lasso's computational expense and need for tuning.
method Dual Feature Reduction (DFR) using strong screening rules and dual norms.
result DFR drastically reduces computational cost without affecting solution optimality.