Group model selection is the problem of determining a small subset of groups of predictors (e.g., the expression data of genes) that are responsible for majority of the variation in a response variable (e.g., the malignancy of a tumor). This paper focuses on group model selection in high-dimensional linear models, in w…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “grouped predictors”
Proposes a two-stage method for selecting correlated predictors in high-dimensional data.
problem Selecting correlated predictors in high-dimensional data with unknown group structures.
method Two-stage approach: variable clustering followed by group selection.
result The two-stage method improves prediction accuracy and active predictor selection.
The article compares predictor importance in classification problems with categorical outcomes.
problem Comparing predictor importance in classification problems with categorical response variables.
method The approach is based on the categorical Gini correlation (CGC) and tests differences in CGCs across predictor groups.
result The proposed methodology accommodates predictors of arbitrary and unequal dimensions and allows for dependence between predictor groups.
Proposes a method to learn fair predictors for multiple subgroups with limited data.
problem Fairness and accuracy issues in learning from multiple subgroups with limited data.
method Formulates a bilevel objective to learn subgroup-specific predictors and a fair predictor that is close to all of them.
result The method effectively controls group sufficiency and generalization error, improving fairness and accuracy.
High-dimensional data pose challenges in statistical learning and modeling. Sometimes the predictors can be naturally grouped where pursuing the between-group sparsity is desired. Collinearity may occur in real-world high-dimensional applications where the popular technique suffers from both selection inconsisten…
Loss minimization leads to multicalibration for neural networks.
problem Ensuring fairness in predictions across multiple protected groups.
method Minimizing squared loss over neural networks of size n.
result Minimizing loss over neural nets of size n implies multicalibration for most values of n.
Various measures can be used to estimate bias or unfairness in a predictor. Previous work has already established that some of these measures are incompatible with each other. Here we show that, when groups differ in prevalence of the predicted event, several intuitive, reasonable measures of fairness (probability of p…
GRASP simplifies Bayesian regression with grouped predictors using an adaptive NBP prior.
problem Regression with grouped predictors and adaptive shrinkage.
method Normal Beta Prime (NBP) prior with tunable hyperparameters for flexible sparsity control.
result Empirical validation of robust and versatile GRASP across various sparsity and signal-to-noise ratios.
Recursive Group lassostat.ME
We introduce a recursive adaptive group lasso algorithm for real-time penalized least squares prediction that produces a time sequence of optimal sparse predictor coefficient vectors. At each time index the proposed algorithm computes an exact update of the optimal -penalized recursive least squares (R…
This paper presents Sparse Partitioning, a Bayesian method for identifying predictors that either individually or in combination with others affect a response variable. The method is designed for regression problems involving binary or tertiary predictors and allows the number of predictors to exceed the size of the sa…
Lasso is a widely used regression technique to find sparse representations. When the dimension of the feature space and the number of samples are extremely large, solving the Lasso problem remains challenging. To improve the efficiency of solving large-scale Lasso problems, El Ghaoui and his colleagues have proposed th…
Study shows competition feedback can make ML predictors biased towards specific user groups.
problem How competition affects machine learning predictors and user prediction quality.
method Flexible model of competing ML predictors, empirical and mathematical analysis.
result Competition causes predictors to specialize for specific sub-populations at the cost of general performance.
WeakNAS uses a set of weaker predictors to find top architectures with fewer samples.
problem Finding the best neural architecture with heavy computation costs.
method Proposes a paradigm shift from fitting the whole architecture space to progressively fitting a search path through a set of weaker predictors.
result WeakNAS produces coarse-to-fine iteration to gradually refine the ranking of sampling space, requiring fewer samples to find top-performance architectures.
As data collections become larger, exploratory regression analysis becomes more important but more challenging. When observations are hierarchically clustered the problem is even more challenging because model selection with mixed effect models can produce misleading results when nonlinear effects are not included into…
We resolve the open problem of optimal sample complexity for multicalibration and deterministic predictors.
problem Optimal sample complexity for multicalibration and deterministic predictors
method Minimax-optimal multicalibration algorithm and generalization to OI predictors
result Minimax-optimal multicalibration algorithm and deterministic predictors with optimal sample complexity
A new approach to group fairness treats it as a bargaining problem.
problem Fairness in deploying predictors across subpopulations.
method Interpreting fairness as a bargaining problem and proposing relative improvement.
result Relative improvement provides axiomatic justification and finite-sample convergence guarantees.
We propose a penalized orthogonal-components regression (POCRE) for large p small n data. Orthogonal components are sequentially constructed to maximize, upon standardization, their correlation to the response residuals. A new penalization framework, implemented via empirical Bayes thresholding, is presented to effecti…
Debiased Machine Learning of Conditional Average Treatment Effects and Other Causal Functionsstat.ME
This paper provides estimation and inference methods for the best linear predictor (approximation) of a structural function, such as conditional average structural and treatment effects, and structural derivatives, based on modern machine learning (ML) tools. We represent this structural function as a conditional expec…
We present two sets of theoretical results on the grouped lasso with overlap of Jacob, Obozinski and Vert (2009) in the linear regression setting. This method allows for joint selection of predictors in sparse regression, allowing for complex structured sparsity over the predictors encoded as a set of groups. This flex…
DynForest R package predicts outcomes with time-dependent predictors.
problem Handling time-dependent predictors in random forest models.
method Random forests with time-dependent predictors summarized using flexible linear mixed models.
result DynForest can predict continuous, categorical, and survival outcomes.
Identifying homogeneous subgroups of variables can be challenging in high dimensional data analysis with highly correlated predictors. We propose a new method called Hexagonal Operator for Regression with Shrinkage and Equality Selection, HORSES for short, that simultaneously selects positively correlated variables and…
This work shows how disentangled and sparse representations improve multi-task learning.
problem Improving generalization in multi-task learning with disentangled and sparse representations.
method Proved a new identifiability result and proposed a practical approach using sparsity-promoting bi-level optimization.
result Maximally sparse base-predictors yield disentangled representations under certain conditions.
Develops methods for selecting and estimating smooth functional coefficients in high-dimensional multivariate functional data.
problem Functional predictor selection and estimation of smooth functional coefficients in high-dimensional multivariate functional data.
method Functional group-sparse regression methods in a generic Hilbert space of infinite dimension.
result Consistency of estimation and selection (oracle property) under infinite-dimensional Hilbert spaces.
We study the interplay between sequential decision making and avoiding discrimination against protected groups, when examples arrive online and do not follow distributional assumptions. We consider the most basic extension of classical online learning: "Given a class of predictors that are individually non-discriminato…
A new method for sparse regression models using graph structure.
problem Sparse regression models for high-dimensional data.
method Decomposes coefficient vector into latent variables, performs regularization on latent variables, uses proximal projection.
result Stable performance compared to other models, especially for high-dimensional data.
Metrizing Fairnesscs.LG
The paper explores fair predictors in supervised learning using IPMs and Kolmogorov distance.
problem Achieving fairness in supervised learning with significant demographic effects.
method Identifying conditions for SP-fair predictors and using IPMs to measure unfairness.
result Fair predictors can improve accuracy and are computationally efficient.
The paper offers simple, near-optimal algorithms for multi-group learning.
problem Learning predictors within subgroups of a population, addressing fairness and hidden stratification.
method Studies the structure of solutions and provides simple, near-optimal algorithms.
result Simple and near-optimal algorithms for multi-group learning.
New estimators improve sparse semiparametric additive modeling.
problem Sparse semiparametric additive modeling with structured sparsity.
method Combines group subset selection with shrinkage for nonconvex optimization.
result New estimators outperform alternatives in synthetic and real-world data.
A non-trivial predictor satisfies demographic parity and equalizes group risks in regression.
problem Achieving fairness in regression models while maintaining equal risks across groups.
method Provided an explicit example of a non-constant predictor satisfying Demographic Parity and Equal Group-Wise Risks.
result First explicit construction of a non-constant predictor satisfying both fairness notions.
flexBART improves BART for categorical predictors by creating flexible tree partitions.
problem Limitation of BART in handling categorical predictors with one-hot encoding.
method flexBART re-implements BART with regression trees that can assign multiple levels to both branches of a decision tree node, and proposes a new decision rule prior for spatial data.
result flexBART often yields improved predictive performance and scales better to larger datasets than existing BART implementations.
We relax demographic parity in regression by enforcing parity at quantile levels and score thresholds.
problem Enforcing full distributional fairness in regression can lead to substantial accuracy loss.
method Introduce (, Z)-fair predictor, derive closed-form solutions, and develop post-processing algorithm.
result The risk gap to the continuous optimum vanishes as the grid is refined, and we enable targeted fairness corrections.
Kernel Cox partially linear regression: building predictive models for cancer patients' survivalstat.ML
Paper develops a method to predict cancer patient survival using molecular profiles.
problem Accurately predicting cancer patient survival with complex survival-molecular profile relationships.
method Kernel Cox partially linear regression with a novel regularized garrotized kernel machine (RegGKM) method.
result The proposed method outperforms other methods in predicting survival accuracy.
The problem of forecasting conditional probabilities of the next event given the past is considered in a general probabilistic setting. Given an arbitrary (large, uncountable) set C of predictors, we would like to construct a single predictor that performs asymptotically as well as the best predictor in C, on any data.…
We study a norm for structured sparsity which leads to sparse linear predictors whose supports are unions of prede ned overlapping groups of variables. We call the obtained formulation latent group Lasso, since it is based on applying the usual group Lasso penalty on a set of latent variables. A detailed analysis of th…
The study simplifies assessing overlap in logistic regression models using empirical likelihood.
problem Assessing overlap in multidimensional logistic regression models.
method Translation of Silvapulle's condition to empirical likelihood maximization, mechanized with R code.
result Minimal overlapping structures are cataloged in dimensions less than four, providing rules for higher dimensions.
The paper connects counterfactual fairness to robust prediction and group fairness using causal context.
problem The challenge of ensuring fairness in AI systems when counterfactuals cannot be directly observed.
method Using causal context to bridge counterfactual fairness, robust prediction, and group fairness.
result Counterfactual fairness is equivalent to group fairness metrics in specific contexts.
Unified multitask learning framework for mixed-type outcomes.
problem Difficulty in formulating a unified objective for tasks with different outcomes.
method Multitask transformation framework with shared sparsity, using deep neural networks and rank-based optimization.
result Improved prediction and variable selection across continuous, binary, and mixed outcomes.
SOCP uses SOM to find groups and local calibration buffers for better regional coverage.
problem Heterogeneous regional coverage gaps in conformal prediction.
method Self-Organizing Map (SOM) for group discovery; local calibration buffers at BMU or fixed grid.
result Reduces regional coverage gaps on 7/8 benchmarks by 7.1%.
A method for fair representation learning through bi-level optimization and implicit differentiation.
problem Ensuring fair predictors invariant across sub-groups.
method Bi-level optimization with inner-loop for invariant predictors, implicit path alignment for efficiency.
result Consistently better trade-off in prediction performance and fairness measurement.
Study on Transfer Elastic Net error bounds and grouping effect.
problem Estimation error and grouping effect in Transfer Elastic Net.
method Derives non-asymptotic error bound and examines grouping effect scenarios.
result Effective error bounds and grouping effect observed in Transfer Elastic Net.
New bounds explain deterministic non-smooth deep nets without large Lipschitz constants.
problem Challenges in explaining generalization of deterministic non-smooth deep nets.
method De-randomized PAC-Bayes margin bounds for deterministic non-convex and non-smooth predictors.
result New bounds avoid large Lipschitz constants, providing generalization guarantees.
Penalized regression is an attractive framework for variable selection problems. Often, variables possess a grouping structure, and the relevant selection problem is that of selecting groups, not individual variables. The group lasso has been proposed as a way of extending the ideas of the lasso to the problem of group…
This paper proposes a method to reduce complexity in GLMs with categorical predictors.
problem Wasteful, hard-to-interpret, and prone to overfitting of traditional one-hot encoding for high-cardinality categorical predictors.
method Clustering categories of categorical predictors through a numerical method that preserves or improves accuracy while reducing the number of coefficients.
result Clustering categories of categorical predictors reduces complexity substantially without harming accuracy.
A Brief Introduction to the Temporal Group LASSO and its Potential Applications in Healthcarestat.ML
The Temporal Group LASSO is an example of a multi-task, regularized regression approach for the prediction of response variables that vary over time. The aim of this work is to introduce the reader to the concepts behind the Temporal Group LASSO and its related methods, as well as to the type of potential applications …
Paper proposes a sparse synthetic control method to select important predictors.
problem Choosing and weighting predictors affects synthetic control estimator performance.
method Sparse synthetic control procedure that penalizes predictors, derived in a linear factor model.
result Sparse synthetic control achieves lower bias and better post-treatment performance.
In this paper we combine two important extensions of ordinary least squares regression: regularization and optimal scaling. Optimal scaling (sometimes also called optimal scoring) has originally been developed for categorical data, and the process finds quantifications for the categories that are optimal for the regres…
Proposes a method to create fair, robust predictors that remain consistent across different scenarios.
problem Creating fair and robust machine learning models that behave consistently across different scenarios.
method Graphical criteria and a model-agnostic framework called CIP based on HSCIC.
result Demonstrates the effectiveness of CIP in enforcing counterfactual invariance across various datasets.
This paper continues study, both theoretical and empirical, of the method of Venn prediction, concentrating on binary prediction problems. Venn predictors produce probability-type predictions for the labels of test objects which are guaranteed to be well calibrated under the standard assumption that the observations ar…