SOAK assesses data subset similarity for better model training.
problem Estimating similarity between data subsets for accurate predictions.
method Same/Other/All K-fold cross-validation method.
result SOAK estimates similarity of learnable/predictable patterns in data subsets.
The paper develops an algorithm to select a subset of training data for efficient regression models.
problem Designing an efficient algorithm for selecting a subset of training data to train regression models quickly without sacrificing accuracy.
method The paper tackles this problem by formulating it as a minimization of training loss with respect to both trainable parameters and subset of training data, subject to error bounds on the validation set. They use a novel problem formulation and represent it with simplified constraints using the dual of the original training problem. They then develop SELCON, an efficient majorization-minimization algorithm for data subset selection, which admits an approximation guarantee.
result The experiments show that SELCON trades off accuracy and efficiency more effectively than the current state-of-the-art.
This paper improves volatility forecasting using dynamic subset selection in genetic programming.
problem Improving accuracy of implied volatility forecasting.
method Dynamic training-subset selection methods applied to genetic programming.
result Dynamic subset selection improves predictive accuracy of genetic programming models.
Supervised machine learning based state-of-the-art computer vision techniques are in general data hungry and pose the challenges of not having adequate computing resources and of high costs involved in human labeling efforts. Training data subset selection and active learning techniques have been proposed as possible s…
Simplified DGPs training by fixing inducing inputs to subset of data.
problem Challenging training of deep Gaussian processes.
method Fixed subset of data for inducing inputs, variational sampling.
result Significant reduction in trainable parameters and computation cost without performance degradation.
Subset selection improves weak supervision performance.
problem Optimizing the use of weakly-labeled data.
method Combining pretrained data representations with the cut statistic for subset selection.
result Subset selection improves weak supervision performance by up to 19%.
BWS selects best window subsets for efficient data pruning.
problem Challenges in selecting subsets of large datasets for neural network training.
method Best Window Selection (BWS) by choosing optimal window intervals from ordered sample scores.
result BWS outperforms other methods across various selection ratios and datasets.
Defense against backdoor attacks by clustering incompatible data subsets.
problem Backdoor poisoning attacks on deep neural networks.
method Incompatibility clustering of data subsets during training.
result Successfully reduces attack success rate to below 1%.
Proposes faster neural network learning by using subsets of training data.
problem Manual design and validation of network topologies is time-consuming.
method Exploits subsets of training data at each incremental training step and performs online hyperparameter selection.
result Significantly reduces overall training time while maintaining performance.
Improved defense against data poisoning attacks by aggregating smaller subsets.
problem Mitigating the impact of poisoned data on model robustness.
method Finite Aggregation method that combines duplicates of smaller disjoint subsets for training.
result Consistent improvement in certified robustness bounds, up to 4.77% on GTSRB.
The paper proposes a new algorithm to select subsets of training data for better accuracy and explainability.
problem Tackles the challenge of balancing accuracy and explainability in pattern recognition.
method Identifies multiple subsets with simple local patterns by clustering similar instances.
result The sub-setting algorithm outperformed traditional decision trees by 15% on the international stroke dataset.
CRAIG selects a coreset for faster machine learning training.
problem Efficiently training machine learning models with limited data.
method Develops CRAIG, a method to select a weighted subset (coreset) of training data.
result CRAIG achieves a speedup that is inversely proportional to the size of the subset.
Proposes a neural framework to select subsets efficiently across different models.
problem Lack of generalizability in subset selection methods for unseen architectures.
method Introduces a trainable subset selection framework, SubSelNet, that uses attention-based neural gadgets and subset samplers.
result SubSelNet generalizes across architectures and outperforms existing methods.
Datamodels predict model outcomes from training data subsets.
problem Understanding model behavior from training data.
method Conceptual framework for analyzing model behavior using datamodels.
result Simple linear datamodels can accurately predict model outputs.
Eigenrank selects images for deep learning training and predicts segmentation failures.
problem Challenges in automating deep learning for medical image segmentation.
method Eigenrank selects images for training and predicts failures using Von Neumann information.
result U-Net trained on Eigenrank-selected images outperforms one trained on random images.
Subset pretraining speeds up neural network training.
problem Efficiently training neural networks with partial gradients.
method Use small subsets to approximate the loss minimum of the full training set.
result Subset minima can approximate the full training set's loss minimum efficiently.
A challenge in training discriminative models like neural networks is obtaining enough labeled training data. Recent approaches use generative models to combine weak supervision sources, like user-defined heuristics or knowledge bases, to label training data. Prior work has explored learning accuracies for these source…
Ensemble AL searches large datasets for optimal training samples.
problem Improving DNN performance and reducing training time with optimal training data.
method Ensemble Active Learning (AL) for large-scale training data subset search.
result Our approach finds favorable subsets of training data that improve DNN accuracy.
Consensus NN learns from noisy data only for medical image denoising.
problem Lack of clean training data for medical image denoising.
method Trains neural network using only noisy data by splitting and combining subsets.
result Improved performance on denoising medical images compared to existing methods.
A decision-theoretic bootstrapping method for robust uncertainty quantification.
problem Uncertainty in finite data sets and distributional shift between training and testing data.
method Partition data, train models, sample UQ subsets, define adversarial game, identify optimal mixed strategies.
result Optimal model mixtures and UQ estimates for robust uncertainty quantification.
DRCS selects a subset of data to minimize worst-case test error under covariate shift.
problem Selecting a subset of data that performs well across different deployment scenarios when data distributions differ.
method DRCS derives an upper bound for the worst-case test error assuming covariate shift and selects instances to minimize this bound.
result DRCS achieves distributionally robust training instance selection.
Federated learning method selects relevant data to improve model accuracy.
problem Noisy and irrelevant data negatively impacts model training in federated learning.
method Use a benchmark model to evaluate and select relevant data subsets from each client.
result Up to 25% improvement in model accuracy compared to training with all data.
Identifies minimal training subset to flip a prediction.
problem Flipping predictions in machine learning models.
method Extended influence function for relabeling minimal subset.
result Relabeling fewer than 2% of training points can flip a prediction.
Proposes coreset method for robust training of neural networks with noisy labels.
problem Overfitting of neural networks trained with noisy labels.
method Selects weighted subsets (coresets) of clean data points to approximate low-rank Jacobian matrix.
result Gradient descent applied to coreset subsets does not overfit noisy labels.
Bayesian optimization framework for hyperparameter tuning with directional derivatives.
problem Efficient hyperparameter tuning for machine learning models.
method Bayesian optimization with directional derivatives to seek more complex models.
result Demonstrated improved performance on various machine learning tasks.
LLMs can be influenced by unseen dataset subtexts, revealing new ways to select data subsets.
problem Understanding how datasets subtly influence LLMs and their properties.
method Logit-Linear-Selection (LLS) method to select subsets of datasets.
result LLS reveals hidden effects in LLMs that persist across different models and architectures.
Apricot selects subsets from large data sets efficiently using submodular optimization.
problem Efficiently selecting representative subsets from large data sets.
method Submodular optimization with efficient greedy algorithm.
result Strong theoretical guarantees on the quality of selected subsets.
Unified analysis of self-training for deep networks on unlabeled data.
problem Theoretical understanding of self-training for deep networks on unlabeled data.
method Unified theoretical analysis using expansion assumption and input-consistency regularization.
result Proves high accuracy of minimizers of population objectives based on self-training and input-consistency regularization.
PixelCNN models can achieve state-of-the-art results on CIFAR-10 with exact likelihood computation.
problem Dequantization gap in modeling discrete data like images.
method Introducing subset flows to allow exact computation of likelihoods for discrete data.
result PixelCNN models trained with exact likelihood computation achieve state-of-the-art results on CIFAR-10.
Develops method to train classifiers on incomplete feature datasets.
problem Training classifiers on datasets with missing feature subsets.
method Simultaneous training of neural networks and sparse coding.
result Classifier trained on incomplete features correctly separates original data.
Algorithm uncovers latent attribute graph from molecular data.
problem Learning latent representations and interpreting them for limited data.
method Perturbation experiments on latent codes of a generative autoencoder.
result Effective graphical model of latent codes and attributes.
Bayesian approach improves deep learning efficiency.
problem Efficient training of deep learning models with limited labeled data.
method Combines active learning and data augmentation with Bayesian generative models.
result Improves training efficiency and classification accuracy compared to existing methods.
DUA-D2C refines D2C for better deep learning generalization.
problem Overfitting in deep learning models due to data outliers and noise.
method Dynamic uncertainty-aware aggregation of D2C subsets.
result Significant improvement in generalization performance.
Deep neural networks require a large amount of labeled training data during supervised learning. However, collecting and labeling so much data might be infeasible in many cases. In this paper, we introduce a source-target selective joint fine-tuning scheme for improving the performance of deep learning tasks with insuf…
This paper uses deep neural networks for one-class classification by splitting normal data into typical and atypical subsets.
problem Training deep neural networks with only one class of data for one-class classification.
method Intra-class splitting to create typical and atypical subsets, using binary loss and auxiliary subnetworks.
result The method outperformed seven baselines and had comparable performance to state-of-the-art methods on image datasets.
Identifies minimal feature subsets for black-box model decisions.
problem Local explanation techniques are limited and sensitive to irrelevant features.
method Sufficient input subsets for same decision with minimal features.
result Reveals global decision-making principles from feature patterns.
Slice-based Learning improves model performance on critical data subsets.
problem Low performance on critical data subsets in machine learning models.
method Proposes a new programming model (Slice-based Learning) that uses slicing functions to specify critical data subsets and combines these with an attention mechanism.
result Improves model performance by up to 19.0 F1 on slices and 4.6 F1 overall.
Study finds 10% redundant images in image classification datasets.
problem Redundancy in large image classification datasets.
method Analysis of CIFAR-10 and ImageNet datasets to identify redundant images.
result 10% of images are redundant and can be removed without significant loss of performance.
A new feature selection method for semi-supervised learning with imperfect labels.
problem Feature selection for semi-supervised learning with imperfectly labeled data.
method Genetic algorithm for proposing feature subsets, probabilistic error model for mislabeling, multi-class C-bound selection criterion.
result Empirical results show the effectiveness of the proposed framework compared to state-of-the-art approaches.
In their standard form Gaussian processes (GPs) provide a powerful non-parametric framework for regression and classificaton tasks. Their one limiting property is their O(N3) scaling where N is the number of training data points. In this paper we present a framework for GP training with sequential sele…
Optimizes model training efficiency with core subset selection.
problem Escalating computational costs in large dataset training.
method Core subset selection for reweighting.
result Efficiently minimizes computational time and improves model performance.
Neural networks learn clean data patterns first, then noisy data, leading to improved performance initially but deteriorating later.
problem Improvement in prediction error on clean data during early training of neural networks with noisy labels.
method Theoretical analysis and experiments to explore the dynamics of gradient descent and the impact of clean and noisy data.
result Neural networks prioritize learning clean data patterns first, leading to improved performance initially but deteriorating later due to diminishing gradient dominance of clean samples over noisy ones.
ModHiFi identifies critical components for model modification without gradients or loss function.
problem Modifying open weight models without access to training data or loss function.
method Theoretical analysis of Lipschitz-continuous networks, Subset Fidelity metric, and ModHiFi algorithm.
result ModHiFi-P and ModHiFi-U achieve significant performance improvements in model pruning and unlearning.
Bayesian approach scores influential training examples for model predictions.
problem Enhance interpretability and safety of machine learning models.
method Formulate TDA as a Bayesian information-theoretic problem, scoring subsets by information loss.
result Method aligns with classical influence scores while promoting diversity for subsets.
The paper tackles approximate unlearning from a subset of training data using variational inference.
problem Unlearning from a small subset of erased training data while maintaining the posterior belief from the full data.
method Formulates unlearning as minimizing KL divergence, equivalent to minimizing an evidence upper bound. Uses variational inference to approximate posterior beliefs and proposes two tricks to handle challenges.
result Demonstrates the effectiveness of the proposed unlearning methods on various Bayesian models.
The study evaluates various ML models for stock market prediction.
problem Predicting the Nifty 50 Index using machine learning models.
method 8 supervised machine learning models (AdaBoost, kNN, LR, ANN, RF, SGD, SVM, DT) applied to historical Nifty 50 Index data.
result Support Vector Machine performed best, but Stochastic Gradient Descent improved performance with larger datasets.
Improved disentanglement of data factors using recursive training.
problem Current unsupervised disentanglement methods are inconsistent and fail to achieve levels of disentanglement seen in supervised approaches.
method Introduced PBT for VAEs, used UDR for heuristic scoring, and developed recursive rPU-VAE approach.
result Recursive training leads to robust disentanglement of data factors across multiple datasets.
METASET selects diverse unit cells for efficient data-driven metamaterial design.
problem Imbalanced datasets in unit cells can bias data-driven metamaterial design.
method METASET uses similarity metrics and DPPs to select diverse subsets of unit cells.
result Smaller, diverse subsets improve search process and structural performance.