Study finds some datasets benefit from small training samples.
problem The effectiveness of large training datasets in modern computer vision.
method Gradient-based importance measure to analyze training images.
result Some datasets can be trained with smaller samples, others not.
The paper predicts loss scaling across different datasets and compute scales.
problem Predicting loss scaling across different datasets and compute scales.
method Derive shifted power law relationships between train and test losses.
result Shifted power law relationships hold for various datasets and tasks, improving prediction accuracy.
Contingency Training improves classifier accuracy and robustness against irrelevant variables.
problem Feature selection leaves irrelevant variables in high-dimensional datasets, reducing classifier performance.
method Subsampling and creating constraints to find proper feature importance weights.
result Contingency Training outperforms traditional training methods, especially with irrelevant variables.
Predicts optimal training dataset sizes per class for machine learning models.
problem Optimizing training dataset sizes for class-specific machine learning models.
method Algorithm based on space-filling design of experiments, models like powerlaw curves and generalized linear models.
result The algorithm predicts optimal training dataset sizes per class for improved model performance.
Study shows partially-typed NER datasets can match fully-typed ones in model performance.
problem Leveraging multiple partially-typed NER datasets for training models without fully-typed annotations.
method Systematic analysis and controlled experiments comparing partially-typed and fully-typed datasets.
result Models trained with partially-typed annotations can achieve similar performance to those trained with fully-typed annotations.
Develops efficient algorithm for distilling supervised and offline RL datasets.
problem Creating synthetic datasets for supervised and offline RL models to match training data performance.
method An efficient algorithm based on matching losses without model training, using randomly sampled regressors.
result Synthetic datasets derived with ildeO(d2) sampled regressors match MSE loss of bounded linear models on training data. Improves neural network performance by enriching training dataset.
problem Achieving worst-case performance guarantees in neural networks.
method Adapting training dataset during training to reduce worst-case violations.
result Improved worst-case performance guarantees in neural networks.
Study explores how dataset breadth and depth affect Siamese Neural Network performance.
problem Impact of dataset breadth and depth on Siamese Neural Network performance.
method Experiments with three keystroke datasets varying breadth and depth factors.
result Increasing dataset breadth improves model performance, while depth's impact varies by dataset type.
Reducing neural network training time by using smaller, representative datasets.
problem Long training times for neural networks.
method Topology-based representative datasets, measured by persistence diagrams.
result Neural network accuracy on representative datasets is similar to original datasets for perceptrons and mean squared error.
GapNet uses incomplete datasets to train neural networks, improving disease detection.
problem Training neural networks with incomplete datasets, especially in health care.
method Split dataset into subsets, train individual neural networks, combine and fine-tune.
result Improved identification of patients with Alzheimer's and Covid-19 risk.
This work studies the impact of intra-/inter-class diversity on pre-training datasets and finds a balance for optimal performance.
problem The impact of intra-/inter-class diversity on supervised pre-training datasets and their effect on downstream tasks.
method Empirical study and theoretical analysis of the relationship between diversity types and downstream performance.
result The optimal class-to-sample ratio is invariant to the size of the pre-training dataset and can be predicted.
Recent work has shown that optical flow estimation can be formulated as a supervised learning task and can be successfully solved with convolutional networks. Training of the so-called FlowNet was enabled by a large synthetically generated dataset. The present paper extends the concept of optical flow estimation via co…
We introduce a new parameter to measure the inhomogeneity of training datasets.
problem The need for non-stationary models in supervised learning.
method We introduce a new parameter, the inhomogeneity parameter, to measure the inhomogeneity of training datasets.
result A training set with a non-zero inhomogeneity parameter requires a non-stationary model for accurate predictions.
Paper distills large datasets into smaller, synthetic ones for model training.
problem Training models on large datasets is computationally expensive.
method Dataset distillation: synthesize small, synthetic datasets from large ones.
result Synthetic datasets can approximate performance of models trained on original large datasets.
KIP meta-learning compresses datasets significantly.
problem Training data size and quality issues in machine learning.
method Kernel Inducing Points (KIP) for dataset compression.
result Significant reduction in dataset size with similar model performance.
Reduces deep learning model training dataset size by 90%.
problem Redundant information from large training datasets.
method Dynamic instance selection based on CNN predictions.
result Maintained predictive power with reduced dataset size.
Graphical heuristic reduces and partitions large datasets for faster supervised training.
problem Training large datasets for classification tasks.
method Clustering and information graph construction for dataset reduction and partitioning.
result Significant speed-up in training run-time without compromising prediction accuracy.
We use smaller proxy datasets to speed up hyperparameter search.
problem Time-consuming and costly training of neural networks on large datasets.
method Generate smaller 'proxy datasets' by sampling, training on easy/hard examples, or generating synthetic data. Compare proxy quality using variance measures.
result Proxy datasets can explain up to 95% of the variance in full dataset results, significantly faster.
This paper investigates how dataset properties affect GAN training outcomes.
problem GANs often fail to reach equilibrium due to instability or mode collapse.
method Experiments to identify patterns in dataset properties and their effects on GAN training.
result Patterns in dataset properties influence GAN training dynamics and outcomes.
Develops a fast method to analyze image similarities without pre-trained models.
problem Lack of efficient methods to analyze fine-level similarities in large image datasets.
method Wavelet decomposition and numerical analysis tools.
result Identifies similar images in standard datasets (like CIFAR) in a few seconds.
Pre-trains deep networks on random labels for improved supervised learning.
problem Improving supervised learning performance with limited labeled data.
method Randomly labels all samples in an unlabeled dataset and trains a deep neural network to memorize these labels.
result Improves classification accuracy by 1.5% to 5% on various action recognition datasets.
Paper improves sound event detection using semi-supervised learning.
problem Weakly labeled sound event detection in polyphonic audio clips.
method Combines tri-training and adversarial learning for semi-supervised learning.
result Significant performance improvement over baseline model.
Optimized RL algorithms perform well on offline datasets, outperforming fully trained agents.
problem Improving reinforcement learning performance on offline datasets.
method Random Ensemble Mixture (REM) algorithm for Q-learning, trained on DQN replay dataset.
result Offline REM outperforms strong RL baselines and fully trained DQN agent.
Self-training outperforms pre-training on COCO object detection and segmentation datasets.
problem The effectiveness of pre-training in improving object detection and segmentation models is limited.
method Investigated self-training as an alternative method to utilize additional data.
result Self-training consistently improves model performance across various dataset sizes and data augmentation levels.
GANs trained on artificial datasets avoid dataset biases, revealing generative model weaknesses.
problem GANs trained on real datasets often underfit or overfit, making analysis difficult.
method Trained GANs on artificial datasets with infinite samples and simple distributions.
result GANs fail to learn optimal parameters, suggesting limitations in generative models.
Automated learning rate tuning for neural networks across datasets and models.
problem Finding optimal learning rates for neural network training, especially in adversarial settings.
method A parameterless, adaptive learning rate trajectory algorithm.
result Consistently achieves top-level accuracy in natural and adversarial training.
Paper proposes a method to make neural networks more robust to visual distortions.
problem Neural networks lack robustness to real-world visual distortions.
method A general training method that makes neural networks more robust to input visual perturbations.
result Proposed training method improves neural network performance in noisy datasets.
Improved disentanglement of data factors using recursive training.
problem Current unsupervised disentanglement methods are inconsistent and fail to achieve levels of disentanglement seen in supervised approaches.
method Introduced PBT for VAEs, used UDR for heuristic scoring, and developed recursive rPU-VAE approach.
result Recursive training leads to robust disentanglement of data factors across multiple datasets.
New framework shows diverse training data improves subgroup and overall performance.
problem Lack of understanding how diverse training data affects subgroup and overall performance.
method Casts data collection as part of the learning process, analyzes dataset compositions, and guides dataset design.
result Diverse representation in training data improves subgroup and overall performance.
Optimal quantization improves dataset distillation for faster training.
problem Efficiently train models with synthetic data.
method Reformulate disentangled methods as optimal quantization problems.
result Better performance and generalization on ImageNet-1K and subsets.
CAOL learns filters from large datasets, and this paper analyzes its performance.
problem Understanding how dataset size impacts CAOL filter updates.
method Analyzes the impact of dataset size on CAOL filter updates using deterministic and probabilistic bounds.
result Empirical evidence suggests that using more training data can improve CAOL performance.
A new method selects the best feature selection technique for datasets.
problem Selecting the best feature selection method for unseen datasets.
method Data synthesis, meta features, fuzzy similarity, classification model training.
result Successfully recommended the best feature selection method for five out of eight datasets.
Defense against backdoor attacks by clustering incompatible data subsets.
problem Backdoor poisoning attacks on deep neural networks.
method Incompatibility clustering of data subsets during training.
result Successfully reduces attack success rate to below 1%.
Poisoning datasets can reveal private details of other users' training points.
problem Integrity and privacy of machine learning training data.
method Active inference attacks that poison a small fraction of the training dataset.
result Poisoning as little as 0.1% of the training dataset can significantly boost inference attacks.
Many of the recent approaches to polyphonic piano note onset transcription require training a machine learning model on a large piano database. However, such approaches are limited by dataset availability; additional training data is difficult to produce, and proposed systems often perform poorly on novel recording con…
Improved CNN training with BDFA reduces computational cost and improves accuracy.
problem Low training performance of DFA in CNN.
method Combining DFA with BP, introducing feedback weight initialization, and proposing BDFA.
result BDFA shows better performance than conventional BP, especially in small datasets.
Algorithm finds best mixture of training datasets for improved validation performance.
problem Learning from mixture distributions with covariate shift.
method Combines SGD with optimistic tree search and model re-use over mixture space.
result Proves simple regret guarantees for recovering optimal mixture.
Collecting large training datasets, annotated with high-quality labels, is costly and time-consuming. This paper proposes a novel framework for training deep convolutional neural networks from noisy labeled datasets that can be obtained cheaply. The problem is formulated using an undirected graphical model that represe…
MLDS dataset reveals hidden model behavior via weight-space analysis.
problem Neural networks' opacity makes them hard to evaluate.
method Presented MLDS dataset of trained neural networks.
result Weight-space analysis reveals meaningful divergence with small changes in training data.
Deep neural networks (DNNs) trained on large-scale datasets have exhibited significant performance in image classification. Many large-scale datasets are collected from websites, however they tend to contain inaccurate labels that are termed as noisy labels. Training on such noisy labeled datasets causes performance de…
Paper tackles privacy issues in LDA model training.
problem Privacy leakage in LDA model training.
method Developed privacy monitoring and locally private LDA training algorithms.
result Demonstrated effectiveness of locally private LDA training on crowdsourced data.
A hybrid ASR system using conformer architecture improves word-error-rate and training speed.
problem Improving word-error-rate and training efficiency for hybrid ASR systems.
method Used conformer architecture, applied time downsampling, and transposed convolutions.
result Conformer-based hybrid model achieves competitive results and significantly outperforms BLSTM-based hybrid model.
Ensemble AL searches large datasets for optimal training samples.
problem Improving DNN performance and reducing training time with optimal training data.
method Ensemble Active Learning (AL) for large-scale training data subset search.
result Our approach finds favorable subsets of training data that improve DNN accuracy.
Collecting the large datasets needed to train deep neural networks can be very difficult, particularly for the many applications for which sharing and pooling data is complicated by practical, ethical, or legal concerns. However, it may be the case that derivative datasets or predictive models developed within individu…
We introduce two Python frameworks to train neural networks on large datasets: Blocks and Fuel. Blocks is based on Theano, a linear algebra compiler with CUDA-support. It facilitates the training of complex neural network models by providing parametrized Theano operations, attaching metadata to Theano's symbolic comput…
Lung segmentation accuracy varies little across diverse datasets.
problem Limited clinical applicability of automated lung segmentation methods.
method Comparison of four deep learning approaches and two standard algorithms on diverse datasets.
result Standard U-net approach yields higher accuracy on routine imaging data.
An important challenge for human-like AI is compositional semantics. Recent research has attempted to address this by using deep neural networks to learn vector space embeddings of sentences, which then serve as input to other tasks. We present a new dataset for one such task, `natural language inference' (NLI), that c…
Foundation models trained on collider data improve jet generation tasks.
problem Improving foundation models for jet generation tasks.
method Pre-training OmniJet-α model on AspenOpenJets dataset. result Pre-trained model improves performance on jet generation tasks with domain shift.