Improves performance in various machine learning tasks by reparameterizing subset sampling.
problem Stochastic optimization involving subset sampling is not reparameterizable.
method Continuous relaxation of subset sampling to provide reparameterization gradients.
result Improves performance in instance-wise feature selection, deep stochastic k-nearest neighbors, and parametric t-SNE.
A learner selects subsets of choices for a user who then picks from them, aiming to minimize regret.
problem Optimizing subset selection for user choices in a stochastic setting.
method Introduces a new problem and defines regret, then proposes algorithms with matching upper and lower bounds.
result Upper and lower bounds on expected regret match up to a logarithmic term, demonstrating algorithm efficiency.
A dynamic keyword selection model for topic modeling of tweets.
problem Adjusting keywords dynamically to mimic past topics with novelty.
method Generative process selects keywords and documents, trained with variational lower bound and stochastic gradient optimization.
result Keyword-based topic model outperforms a sophisticated baseline model by 67%.
BSF algorithm reduces neural network size and selects features efficiently.
problem Neural network size and feature selection optimization.
method Binary Stochastic Filtering (BSF) layer that penalizes information, stochastically passes or drops features.
result Multifold decrease in neural network size and optimal feature selection.
Unified framework for gradient estimation in combinatorial spaces.
problem Scaling relaxed gradient estimators to large combinatorial distributions.
method Introducing stochastic softmax tricks within the perturbation model framework.
result Stochastic softmax tricks improve model performance and discover more latent structure.
A new method for feature selection in high-dimensional data reduces search cost while maintaining performance.
problem Feature selection in very-high-dimensional datasets is computationally expensive and challenging.
method Stochastic Sequential Search (SSS) using temperature-controlled softmax sampling and dependency-aware statistics.
result The method significantly reduces search cost while maintaining or improving performance.
Optimizes worker allocation for crowdsourcing to improve accuracy.
problem Maximizing confidence in crowdsourced answers with limited budget.
method Submodular stochastic subset selection with dynamic task allocation.
result Dynamic task allocation leads to higher accuracy with fewer labels.
CRAIG selects a coreset for faster machine learning training.
problem Efficiently training machine learning models with limited data.
method Develops CRAIG, a method to select a weighted subset (coreset) of training data.
result CRAIG achieves a speedup that is inversely proportional to the size of the subset.
A new method for sparse linear bandits reduces exploration-exploitation tradeoff.
problem Sparse linear bandits in high-dimensional settings with finite actions.
method Best subset selection for parameter estimation and doubly growing epochs for regret minimization.
result Achieves nearly dimension-independent regret of i l d e O ( s T ) ilde{\mathcal{O}}(s\sqrt{T}) i l d e O ( s T ) with high probability. Bayesian method improves adaptive testing item selection, ensuring full item exposure.
problem Adaptive testing selects items to estimate ability, but must also ensure diverse item exposure.
method Formulated as Bayesian model averaging, deriving optimal item sampling probabilities.
result Stochastic method achieves full item bank exposure without sacrificing accuracy.
Improved Bayesian neural network inference by selectively removing redundant modes.
problem Redundant modes in Bayesian neural network posteriors complicate approximate inference.
method Structured partial stochasticity and deterministic subset selection of weights.
result Improved performance of approximate inference schemes with simplified posterior distribution.
This paper improves volatility forecasting using dynamic subset selection in genetic programming.
problem Improving accuracy of implied volatility forecasting.
method Dynamic training-subset selection methods applied to genetic programming.
result Dynamic subset selection improves predictive accuracy of genetic programming models.
A new algorithm THV-UCB reduces regret in multi-objective bandit problems.
problem Maintaining a small set of actions that jointly approximate the Pareto frontier in multi-objective slate selection.
method THV-UCB, an optimistic algorithm that selects arms based on optimistic estimates of their marginal hypervolume contributions.
result The algorithm achieves a gap-free regret bound of i l d e O ( d n k T ) ilde{O}(d\sqrt{nkT}) i l d e O ( d nk T ) and a gap-dependent bound of i l d e O ( n k 2.5 / Δ min ) ilde{O}(nk^{2.5}/Δ_{\min}) i l d e O ( n k 2.5 / Δ m i n ) . In many classification problems unlabelled data is abundant and a subset can be chosen for labelling. This defines the context of active learning (AL), where methods systematically select that subset, to improve a classifier by retraining. Given a classification problem, and a classifier trained on a small number of la…
One-pass algorithm finds small subset for ℓ p \ell_p ℓ p subspace approximation with additive error.
problem Finding a small subset of data points for ℓ p \ell_p ℓ p subspace approximation. method One-pass subset selection with additive approximation guarantee for p ∈ [ 1 , ∞ ) p \in [1, \infty) p ∈ [ 1 , ∞ ) . result First one-pass algorithm with additive error for ℓ p \ell_p ℓ p subspace approximation. Bayesian approach selects subsets of variables for interpretable prediction and identifies key factors in educational outcomes.
problem Challenges in subset selection for stability, regularization, and inference.
method Bayesian perspective on subset selection, deriving optimal subsets and variable importance metrics.
result Better prediction, interval estimation, and variable selection compared to competing methods.
ADASS selects adaptive subsets for SGD training acceleration.
problem Fixed sample size in SGD limits training efficiency.
method ADASS selects adaptive subsets based on Lipschitz constants.
result ADASS achieves comparable accuracy with full training set.
New algorithm for combinatorial bandit problems reduces regret.
problem Optimal selection of sets of arms in bandit problems.
method SGB algorithm with optimized exploration of unselected arms.
result Achieves ( 1 − 1 / e ) (1-1/e) ( 1 − 1/ e ) -regret bound of O ( n 1 3 k 2 3 T 2 3 log ( T ) 2 3 ) \mathcal{O}(n^{\frac{1}{3}} k^{\frac{2}{3}} T^{\frac{2}{3}} \log(T)^{\frac{2}{3}}) O ( n 3 1 k 3 2 T 3 2 log ( T ) 3 2 ) . BWS selects best window subsets for efficient data pruning.
problem Challenges in selecting subsets of large datasets for neural network training.
method Best Window Selection (BWS) by choosing optimal window intervals from ordered sample scores.
result BWS outperforms other methods across various selection ratios and datasets.
New batch selection strategy improves deep learning model performance.
problem Training deep neural networks efficiently.
method Submodular function maximization for mini-batch selection.
result Deep models trained with proposed batch selection outperform SGD and baseline.
New MCMC algorithm reduces subset selection passes to 2 for optimal k k k -dimensional subspace approximation.
problem Subset selection for k k k -dimensional subspace approximation with ε ε ε -approximation. method MCMC sampling algorithm reducing passes to 2 for p = 2 p=2 p = 2 case, poly(k/ε) size subset. result Subset selection of nearly optimal size in 2 passes, ( 1 + ε ) (1+ε) ( 1 + ε ) approximation. Optimizes kernel discrepancies by selecting subsets efficiently.
problem Improving kernel discrepancies for QMC methods.
method Introduces a novel subset selection algorithm for kernel discrepancies.
result Efficiently generates low-discrepancy samples from various distributions.
We consider learning problems over training sets in which both, the number of training examples and the dimension of the feature vectors, are large. To solve these problems we propose the random parallel stochastic algorithm (RAPSA). We call the algorithm random parallel because it utilizes multiple parallel processors…
Two diversity models improve subset selection for image classification tasks.
problem Data scarcity and high costs in human labeling for supervised learning.
method Facility-Location and Disparity-Min models for training data subset selection and active learning.
result Subset selection improves accuracy by 2-3% with less training data.
IWeS selects examples by entropy-based importance sampling for subset selection.
problem Efficiently selecting examples for model training in batch settings.
method IWeS uses importance sampling based on model entropy to select examples.
result IWeS outperforms other subset selection algorithms on seven datasets.
Bayesian method selects subsets for LMMs with structured dependence.
problem Subset selection challenge in LMMs with structured dependence.
method Bayesian decision analysis with Mahalanobis loss function.
result Optimal linear coefficients for subsets and cardinality constraints.
New algorithm finds best subset in high-dimensional data models.
problem Finding the best subset of predictors in high-dimensional data models.
method Proposes a scalable algorithm using a generalized information criterion.
result Directly proves consistency and oracle property for the best-subset selection.
Efficiently selects predictors in sparse regression without approximations.
problem High computational cost in subset selection for sparse regression.
method Conditional uncorrelation formula and efficient non-approximate method.
result Significant reduction in computational complexity for subset selection.
A new algorithm selects web links to maximize revenue while ensuring user attractiveness.
problem Selecting web links to maximize revenue from multi-level feedbacks.
method Constrained Upper Confidence Bound (Con-UCB) algorithm for stochastic multi-armed bandit.
result Proves O ( T ln T ) O(\sqrt{T\ln T}) O ( T ln T ) bounds on regret and attractiveness constraint violation. New suboptimal algorithm for best subset selection in high-dimensional data.
problem Nonconvex and computationally challenging best subset selection in linear regression.
method Introducing a new suboptimal algorithm and comparing it with other popular methods.
result The new procedure is a competitive suboptimal algorithm for high-dimensional data.
OLPA optimizes online user-centric selection with probing, achieving near-optimal regret bounds.
problem Sequential decision-making with unknown resources and rewards.
method Probing-augmented user-centric selection (PUCS) framework, greedy probing algorithm, OLPA algorithm.
result OLPA achieves a near-optimal regret bound of O ( T + ln 2 T ) \mathcal{O}(\sqrt{T} + \ln^{2} T) O ( T + ln 2 T ) for online settings. A fast algorithm selects best subsets in high-dimensional models.
problem Identifying sparse models in high-dimensional generalized linear models.
method Splicing technique for fast and consistent best subset selection.
result Our algorithm achieves high certainty in selecting best subsets with polynomial computational complexity.
Subset selection improves weak supervision performance.
problem Optimizing the use of weakly-labeled data.
method Combining pretrained data representations with the cut statistic for subset selection.
result Subset selection improves weak supervision performance by up to 19%.
Proposes a neural framework to select subsets efficiently across different models.
problem Lack of generalizability in subset selection methods for unseen architectures.
method Introduces a trainable subset selection framework, SubSelNet, that uses attention-based neural gadgets and subset samplers.
result SubSelNet generalizes across architectures and outperforms existing methods.
abess efficiently solves various machine learning problems quickly.
problem Efficiently solving diverse machine learning problems.
method Unified framework of best-subset selection, polynomial time optimal solution.
result Certifiably gets the optimal solution within polynomial times with high probability.
Optimizes subset selection in sparse learning problems.
problem Sparse learning problems, particularly best subset selection.
method Developed an efficient primal-dual algorithm leveraging dual range estimation and incremental strategy.
result Improves solutions of best subset selection with reduced redundant computation.
Efficient algorithm solves best subset selection problem.
problem Sparse learning problems, especially best subset selection.
method Primal-dual method based on dual forms of ℓ 0 \ell_0 ℓ 0 -regularized problems. result Improves solutions of best subset selection with reduced redundant computation.
The paper develops an algorithm to select a subset of training data for efficient regression models.
problem Designing an efficient algorithm for selecting a subset of training data to train regression models quickly without sacrificing accuracy.
method The paper tackles this problem by formulating it as a minimization of training loss with respect to both trainable parameters and subset of training data, subject to error bounds on the validation set. They use a novel problem formulation and represent it with simplified constraints using the dual of the original training problem. They then develop SELCON, an efficient majorization-minimization algorithm for data subset selection, which admits an approximation guarantee.
result The experiments show that SELCON trades off accuracy and efficiency more effectively than the current state-of-the-art.
Subset selection struggles in high noise; new method improves performance.
problem Subset selection's poor performance in high noise levels.
method Regularized version of least-squares criterion.
result Proposed estimators outperform best subset selection in high noise.
We consider the problem of matrix column subset selection, which selects a subset of columns from an input matrix such that the input can be well approximated by the span of the selected columns. Column subset selection has been applied to numerous real-world data applications such as population genetics summarization,…
We propose and analyze a new parallel coordinate descent method---`NSync---in which at each iteration a random subset of coordinates is updated, in parallel, allowing for the subsets to be chosen non-uniformly. We derive convergence rates under a strong convexity assumption, and comment on how to assign probabilities t…
Paper tackles fair low-rank approximation and column subset selection.
problem Minimize loss over sub-populations in machine learning.
method Developed algorithms for fair low-rank approximation and fair column subset selection.
result Achieved polynomial time algorithms for fair low-rank approximation.
Selective forgetting method cleans deep network weights of forgotten data.
problem Selective forgetting of specific data subsets in deep neural networks.
method A method to scrub weights clean of forgotten data without retraining.
result The method ensures indistinguishability of probing functions from a non-forgotten network.
Algorithm selects variables and bandwidths for geographically weighted regression.
problem Estimating variable subsets and bandwidths for geographically weighted regression.
method Mathematical programming-based approach integrating variable selection and bandwidth estimation.
result Proposed algorithm provides stable spatially varying patterns with competitive explanatory power.
Apricot selects subsets from large data sets efficiently using submodular optimization.
problem Efficiently selecting representative subsets from large data sets.
method Submodular optimization with efficient greedy algorithm.
result Strong theoretical guarantees on the quality of selected subsets.
For massive data sets, efficient computation commonly relies on distributed algorithms that store and process subsets of the data on different machines, minimizing communication costs. Our focus is on regression and classification problems involving many features. A variety of distributed algorithms have been proposed …
Automated model selects best subset of variables for regression.
problem Finding a subset of variables that minimizes errors and meets regression assumptions.
method Integrates model building and validation using mathematical programming.
result Proposes a model that minimizes mean squared errors while satisfying most regression assumptions.
Optimizes subset selection in multiple linear regression models.
problem Choosing a subset of variables for regression models to balance fit and complexity.
method Developed mathematical programming models and algorithms for subset selection, tested with branch-and-bound and iterative heuristic approaches.
result Proposed models and algorithms efficiently find optimal or near-optimal solutions.