Proposes a group-splicing algorithm for efficient BSGS in high-dimensional settings.
problem Efficiently selecting a small part of non-overlapping groups for best interpretability in high-dimensional settings.
method Iteratively detects relevant groups and excludes irrelevant ones using a novel group information criterion.
result Certifiable polynomial-time algorithm for identifying the optimal subset of groups with high probability.
Bayesian method selects subsets for LMMs with structured dependence.
problem Subset selection challenge in LMMs with structured dependence.
method Bayesian decision analysis with Mahalanobis loss function.
result Optimal linear coefficients for subsets and cardinality constraints.
abess efficiently solves various machine learning problems quickly.
problem Efficiently solving diverse machine learning problems.
method Unified framework of best-subset selection, polynomial time optimal solution.
result Certifiably gets the optimal solution within polynomial times with high probability.
Paper tackles fair low-rank approximation and column subset selection.
problem Minimize loss over sub-populations in machine learning.
method Developed algorithms for fair low-rank approximation and fair column subset selection.
result Achieved polynomial time algorithms for fair low-rank approximation.
Feature selection aims to select the smallest feature subset that yields the minimum generalization error. In the rich literature in feature selection, information theory-based approaches seek a subset of features such that the mutual information between the selected features and the class labels is maximized. Despite …
In today world of enormous amounts of data, it is very important to extract useful knowledge from it. This can be accomplished by feature subset selection. Feature subset selection is a method of selecting a minimum number of features with the help of which our machine can learn and predict which class a particular dat…
New estimators improve sparse semiparametric additive modeling.
problem Sparse semiparametric additive modeling with structured sparsity.
method Combines group subset selection with shrinkage for nonconvex optimization.
result New estimators outperform alternatives in synthetic and real-world data.
New method proves neural networks can select features consistently.
problem Feature selection for deep neural networks is challenging.
method Adaptive Group Lasso selection procedure with Group Lasso as the base estimator.
result Adaptive Group Lasso is selection-consistent for a wide class of neural networks.
Online selection of dynamic features has attracted intensive interest in recent years. However, existing online feature selection methods evaluate features individually and ignore the underlying structure of feature stream. For instance, in image analysis, features are generated in groups which represent color, texture…
This paper improves volatility forecasting using dynamic subset selection in genetic programming.
problem Improving accuracy of implied volatility forecasting.
method Dynamic training-subset selection methods applied to genetic programming.
result Dynamic subset selection improves predictive accuracy of genetic programming models.
We consider the empirical risk minimization problem for linear supervised learning, with regularization by structured sparsity-inducing norms. These are defined as sums of Euclidean norms on certain subsets of variables, extending the usual ℓ1-norm and the group ℓ1-norm by allowing the subsets to overlap. T…
One-pass algorithm finds small subset for ℓp subspace approximation with additive error.
problem Finding a small subset of data points for ℓp subspace approximation. method One-pass subset selection with additive approximation guarantee for p∈[1,∞). result First one-pass algorithm with additive error for ℓp subspace approximation. Bayesian approach selects subsets of variables for interpretable prediction and identifies key factors in educational outcomes.
problem Challenges in subset selection for stability, regularization, and inference.
method Bayesian perspective on subset selection, deriving optimal subsets and variable importance metrics.
result Better prediction, interval estimation, and variable selection compared to competing methods.
BWS selects best window subsets for efficient data pruning.
problem Challenges in selecting subsets of large datasets for neural network training.
method Best Window Selection (BWS) by choosing optimal window intervals from ordered sample scores.
result BWS outperforms other methods across various selection ratios and datasets.
New MCMC algorithm reduces subset selection passes to 2 for optimal k-dimensional subspace approximation.
problem Subset selection for k-dimensional subspace approximation with ε-approximation. method MCMC sampling algorithm reducing passes to 2 for p=2 case, poly(k/ε) size subset. result Subset selection of nearly optimal size in 2 passes, (1+ε) approximation. Optimizes kernel discrepancies by selecting subsets efficiently.
problem Improving kernel discrepancies for QMC methods.
method Introduces a novel subset selection algorithm for kernel discrepancies.
result Efficiently generates low-discrepancy samples from various distributions.
IWeS selects examples by entropy-based importance sampling for subset selection.
problem Efficiently selecting examples for model training in batch settings.
method IWeS uses importance sampling based on model entropy to select examples.
result IWeS outperforms other subset selection algorithms on seven datasets.
New algorithm finds best subset in high-dimensional data models.
problem Finding the best subset of predictors in high-dimensional data models.
method Proposes a scalable algorithm using a generalized information criterion.
result Directly proves consistency and oracle property for the best-subset selection.
Efficiently selects predictors in sparse regression without approximations.
problem High computational cost in subset selection for sparse regression.
method Conditional uncorrelation formula and efficient non-approximate method.
result Significant reduction in computational complexity for subset selection.
Algorithm improves recommendation subset selection in the presence of biases.
problem Maximizing submodular functions for recommendation in the presence of social biases.
method Algorithm for submodular maximization with fairness constraints.
result Algorithm provably outputs subsets with near-optimal utility and proportional representation.
New suboptimal algorithm for best subset selection in high-dimensional data.
problem Nonconvex and computationally challenging best subset selection in linear regression.
method Introducing a new suboptimal algorithm and comparing it with other popular methods.
result The new procedure is a competitive suboptimal algorithm for high-dimensional data.
A fast algorithm selects best subsets in high-dimensional models.
problem Identifying sparse models in high-dimensional generalized linear models.
method Splicing technique for fast and consistent best subset selection.
result Our algorithm achieves high certainty in selecting best subsets with polynomial computational complexity.
Subset selection improves weak supervision performance.
problem Optimizing the use of weakly-labeled data.
method Combining pretrained data representations with the cut statistic for subset selection.
result Subset selection improves weak supervision performance by up to 19%.
Proposes a neural framework to select subsets efficiently across different models.
problem Lack of generalizability in subset selection methods for unseen architectures.
method Introduces a trainable subset selection framework, SubSelNet, that uses attention-based neural gadgets and subset samplers.
result SubSelNet generalizes across architectures and outperforms existing methods.
Optimizes subset selection in sparse learning problems.
problem Sparse learning problems, particularly best subset selection.
method Developed an efficient primal-dual algorithm leveraging dual range estimation and incremental strategy.
result Improves solutions of best subset selection with reduced redundant computation.
Group model selection is the problem of determining a small subset of groups of predictors (e.g., the expression data of genes) that are responsible for majority of the variation in a response variable (e.g., the malignancy of a tumor). This paper focuses on group model selection in high-dimensional linear models, in w…
Efficient algorithm solves best subset selection problem.
problem Sparse learning problems, especially best subset selection.
method Primal-dual method based on dual forms of ℓ0-regularized problems. result Improves solutions of best subset selection with reduced redundant computation.
Paper develops a new algorithm to improve screening processes.
problem Improving the efficiency and accuracy of screening processes.
method Developed a distribution-free screening algorithm called Calibrated Subset Selection (CSS).
result CSS provides near-optimal shortlists with provable diversity guarantees.
This paper considers extractive summarisation in a comparative setting: given two or more document groups (e.g., separated by publication time), the goal is to select a small number of documents that are representative of each group, and also maximally distinguishable from other groups. We formulate a set of new object…
The paper develops an algorithm to select a subset of training data for efficient regression models.
problem Designing an efficient algorithm for selecting a subset of training data to train regression models quickly without sacrificing accuracy.
method The paper tackles this problem by formulating it as a minimization of training loss with respect to both trainable parameters and subset of training data, subject to error bounds on the validation set. They use a novel problem formulation and represent it with simplified constraints using the dual of the original training problem. They then develop SELCON, an efficient majorization-minimization algorithm for data subset selection, which admits an approximation guarantee.
result The experiments show that SELCON trades off accuracy and efficiency more effectively than the current state-of-the-art.
We consider the problem of matrix column subset selection, which selects a subset of columns from an input matrix such that the input can be well approximated by the span of the selected columns. Column subset selection has been applied to numerous real-world data applications such as population genetics summarization,…
Supervised machine learning based state-of-the-art computer vision techniques are in general data hungry and pose the challenges of not having adequate computing resources and of high costs involved in human labeling efforts. Training data subset selection and active learning techniques have been proposed as possible s…
Algorithm selects variables and bandwidths for geographically weighted regression.
problem Estimating variable subsets and bandwidths for geographically weighted regression.
method Mathematical programming-based approach integrating variable selection and bandwidth estimation.
result Proposed algorithm provides stable spatially varying patterns with competitive explanatory power.
High-dimensional data in many areas such as computer vision and machine learning tasks brings in computational and analytical difficulty. Feature selection which selects a subset from observed features is a widely used approach for improving performance and effectiveness of machine learning models with high-dimensional…
For massive data sets, efficient computation commonly relies on distributed algorithms that store and process subsets of the data on different machines, minimizing communication costs. Our focus is on regression and classification problems involving many features. A variety of distributed algorithms have been proposed …
Classification with a sparsity constraint on the solution plays a central role in many high dimensional machine learning applications. In some cases, the features can be grouped together so that entire subsets of features can be selected or not selected. In many applications, however, this can be too restrictive. In th…
Proposes a neural network framework for feature selection in high-dimensional settings.
problem Challenges in feature selection and non-linear function estimation in high-dimensional settings.
method Sparse-input neural networks using group concave regularization.
result Establishes finite-sample guarantees for variable selection consistency and prediction accuracy.
Composite likelihood inference of fractional Gaussian processes with sequentially optimal subset selection
problem Estimating parameters in time series
method Composite likelihood method
result The method reduces computational cost
Structured sparsity has recently emerged in statistics, machine learning and signal processing as a promising paradigm for learning in high-dimensional settings. All existing methods for learning under the assumption of structured sparsity rely on prior knowledge on how to weight (or how to penalize) individual subsets…
Cost-efficient feature selection for multi-label classification in medicine.
problem Feature selection in multi-label classification with cost constraints.
method Sequential feature selection maximizing conditional mutual information, followed by cost-free feature selection using shadow features.
result The method effectively reduces prediction costs in medical applications.
Multitask learning can be effective when features useful in one task are also useful for other tasks, and the group lasso is a standard method for selecting a common subset of features. In this paper, we are interested in a less restrictive form of multitask learning, wherein (1) the available features can be organized…
A new framework selects information sources to test hypotheses robustly, even with misclassifications.
problem Robust hypothesis testing with misclassification penalties.
method Introduces a misclassification penalty framework and an efficient greedy algorithm.
result Proposes a submodular surrogate metric for better selection.
Dimensionality reduction is a first step of many machine learning pipelines. Two popular approaches are principal component analysis, which projects onto a small number of well chosen but non-interpretable directions, and feature selection, which selects a small number of the original features. Feature selection can be…
A/B testing improves marketing decisions by selecting effective stratification variables.
problem Improving the sensitivity of A/B testing through stratified sampling.
method Designing an algorithm to select a subset of stratification variables for variance reduction.
result The subset selection method outperforms other variance reduction techniques in A/B testing.
New framework combines semi-supervised data programming with subset selection for improved text classification.
problem Sub-optimal performance in data programming with noisy labelling functions.
method Introduces a framework \model for semi-supervised data programming that uses joint models and subset selection.
result Significantly outperforms state-of-the-art on seven datasets.
Conventional mutual information (MI) based feature selection (FS) methods are unable to handle heterogeneous feature subset selection properly because of data format differences or estimation methods of MI between feature subset and class label. A way to solve this problem is feature transformation (FT). In this study,…
A system of nested dichotomies is a method of decomposing a multi-class problem into a collection of binary problems. Such a system recursively splits the set of classes into two subsets, and trains a binary classifier to distinguish between each subset. Even though ensembles of nested dichotomies with random structure…
Researchers expand on best subset selection theory, identifying key complexities.
problem Understanding model selection performance in high-dimensional sparse linear regression.
method Analyzing residualized signals, orthogonality, and spurious projections to establish margin conditions.
result Established necessary and sufficient margin conditions for BSS model consistency.