Self-training with noisy student-teacher boosts keyword spotting accuracy.
problem Robust keyword spotting in challenging conditions.
method Aggressive data augmentation and self-training with noisy student-teacher approach.
result Significant accuracy improvement in difficult conditions, up to 60%.
New attacks show data augmentation may not improve privacy.
problem Measuring privacy risk in models trained with data augmentation.
method Formulated membership inference as a set classification problem, designed input permutation invariant features.
result Proposed approach universally outperforms original methods on models trained with data augmentation.
Online data augmentation improves forecasting performance in deep learning.
problem Insufficient training data for forecasting tasks.
method An online data augmentation framework that generates synthetic samples during training.
result Online data augmentation leads to better forecasting performance.
The paper shows how data augmentation and regularization can enforce group equivariance in machine learning models.
problem Improving model performance by leveraging known symmetries in machine learning tasks.
method Training with data augmentation and regularization to enforce group equivariance.
result Equivariance of the trained model can be achieved through training on augmented data in tandem with regularization.
SapAugment learns adaptive augmentation policies for better model training.
problem Fixed data augmentation methods often apply the same augmentation to all samples, ignoring sample difficulty.
method SapAugment adapts augmentation parameters based on training loss, learning a sample-adaptive policy.
result SapAugment achieves up to 21% relative reduction in word error rate on LibriSpeech dataset.
Deep artificial neural networks require a large corpus of training data in order to effectively learn, where collection of such training data is often expensive and laborious. Data augmentation overcomes this issue by artificially inflating the training set with label preserving transformations. Recently there has been…
CNNs encode data augmentation transformations, especially in early layers.
problem Whether neural network features encode data augmentation transformations.
method Systematic approach using pre-trained vision models to predict augmentation transformations.
result Neural network features encode data augmentation transformations, especially in early layers.
Automatically learns optimal data augmentation for image classification.
problem Finding optimal data augmentation hyperparameters is computationally demanding and requires domain knowledge.
method Proposes an online bilevel optimization framework to learn data augmentation parameters directly.
result Jointly trained method achieves comparable or better classification accuracy than hand-crafted data augmentation without an external validation loop.
Transfer learning and data augmentation improve stock classification performance.
problem Challenges in stock classification due to noise and volatility.
method Pre-trained model on S&P500 index features, transfer learning to new models, data augmentation on feature space.
result Augmentation on feature space leads to 20% increase in risk-adjusted returns.
This paper explores the use of adversarial examples in training speech recognition systems to increase robustness of deep neural network acoustic models. During training, the fast gradient sign method is used to generate adversarial examples augmenting the original training data. Different from conventional data augmen…
Data augmentation has been widely applied as an effective methodology to improve generalization in particular when training deep neural networks. Recently, researchers proposed a few intensive data augmentation techniques, which indeed improved accuracy, yet we notice that these methods augment data have also caused a …
Aug-Gen improves music generation by augmenting training data with model-produced examples.
problem Lack of high-quality training data for specific musical styles.
method Dataset augmentation using generated examples during training.
result Aug-Gen improves both training duration and quality of generated music.
This work analyzes benefits and limitations of data augmentation and feature averaging in deep learning models.
problem Theoretical understanding of incorporating invariance into deep learning models is lacking.
method Data augmentation and feature averaging are analyzed in the context of invariance in deep learning.
result Training with data augmentation leads to better estimates of risk and gradients, and feature averaging reduces generalization error with convex losses.
Data augmentation, a technique in which a training set is expanded with class-preserving transformations, is ubiquitous in modern machine learning pipelines. In this paper, we seek to establish a theoretical framework for understanding data augmentation. We approach this from two directions: First, we provide a general…
Data augmentation improves robustness in adversarial training.
problem Adversarial training robust overfitting.
method Common data augmentation schemes combined with model weight averaging.
result Significant boost in robust accuracy, up to +2.93%.
A new data augmentation method selects mixed classes based on class distances for better performance.
problem Improving recognition accuracy in object recognition using deep learning.
method Calculates class distances and selects mixed data from suitable classes dynamically.
result Improves recognition performance on general and long-tailed image recognition datasets.
Data augmentation (DA) is fundamental against overfitting in large convolutional neural networks, especially with a limited training dataset. In images, DA is usually based on heuristic transformations, like geometric or color transformations. Instead of using predefined transformations, our work learns data augmentati…
AugMask trains diffusion models on incomplete tabular data by augmenting missing values and applying denoising supervision.
problem Training diffusion models on incomplete tabular data with missing values.
method AugMask uses stochastic augmentation and denoising supervision to adapt diffusion models to incomplete data.
result AugMask enables diffusion-based tabular generators to outperform specialized missing-aware baselines across various datasets and missingness regimes.
For training the sequence-to-sequence voice conversion model, we need to handle an issue of insufficient data about the number of speech pairs which consist of the same utterance. This study experimentally investigated the effects of Mel-spectrogram augmentation on training the sequence-to-sequence voice conversion (VC…
New measures quantify how data augmentation improves model performance.
problem Understanding the effectiveness of data augmentation in deep learning.
method Introduced Affinity and Diversity measures to quantify augmentation performance.
result Augmentation performance is best achieved by optimizing both Affinity and Diversity.
A recurring problem faced when training neural networks is that there is typically not enough data to maximize the generalization capability of deep neural networks(DNN). There are many techniques to address this, including data augmentation, dropout, and transfer learning. In this paper, we introduce an additional met…
Data augmentation is commonly used to encode invariances in learning methods. However, this process is often performed in an inefficient manner, as artificial examples are created by applying a number of transformations to all points in the training set. The resulting explosion of the dataset size can be an issue in te…
Deep learning models have demonstrated outstanding performance in several problems, but their training process tends to require immense amounts of computational and human resources for training and labeling, constraining the types of problems that can be tackled. Therefore, the design of effective training methods that…
Ensemble models improve prediction calibration for mismatched distributions.
problem Calibration issues in deep neural networks with mismatched train and test distributions.
method Simple data augmentation and mixing techniques for ensemble models.
result Improves calibration and accuracy on CIFAR10 and CIFAR100 benchmarks.
This work investigates image augmentations for GAN training, improving image quality.
problem Improving the accuracy and robustness of GAN models for image synthesis.
method Systematic study of various image augmentation techniques for GAN training.
result Vanilla GANs can achieve state-of-the-art generation quality with image augmentations.
Expert augmentation improves hybrid model generalization.
problem Limited generalization of hybrid models outside training distribution.
method Introducing expert augmentation to improve hybrid model performance.
result Expert augmentation improves generalization of hybrid models.
Data augmentation is a widely used technique in many machine learning tasks, such as image classification, to virtually enlarge the training dataset size and avoid overfitting. Traditional data augmentation techniques for image classification tasks create new samples from the original training data by, for example, fli…
DARE uses GPT-2 to generate relation examples, improving RE performance.
problem Challenges in Relation Extraction, especially limited data and class imbalance.
method Data augmentation with GPT-2 fine-tuning to generate relation examples.
result Improves RE performance by up to 11 F1 score points.
Data augmentation is a widely used trick when training deep neural networks: in addition to the original data, properly transformed data are also added to the training set. However, to the best of our knowledge, a clear mathematical framework to explain the performance benefits of data augmentation is not available. In…
LcGAN generates synthetic CT images for hemorrhagic lesion segmentation.
problem Scarce training data for hemorrhagic lesion segmentation.
method Lesion conditional Generative Adversarial Network (LcGAN) for synthetic image generation.
result Segmentation improved by 12.8% with synthetic data augmentation.
We study the recently introduced stability training as a general-purpose method to increase the robustness of deep neural networks against input perturbations. In particular, we explore its use as an alternative to data augmentation and validate its performance against a number of distortion types and transformations i…
Exemplar VAEs link generative models with nearest neighbor retrieval and data augmentation.
problem Improving generative model performance and data augmentation effectiveness.
method Exemplar VAEs with Parzen window prior, retrieval augmented training, exemplar leave-one-out, and subsampling.
result Generative data augmentation reduces classification error on MNIST and Fashion MNIST.
This paper improves auto-augment efficiency by sharing augmentation weights.
problem Efficient evaluation of augmentation policies for model training.
method Augmentation-Wise Weight Sharing (AWS) to create a fast yet accurate proxy task.
result Augmentation policies found achieve superior accuracies compared to existing methods.
Dropout is typically interpreted as bagging a large number of models sharing parameters. We show that using dropout in a network can also be interpreted as a kind of data augmentation in the input space without domain knowledge. We present an approach to projecting the dropout noise within a network back into the input…
ADA augments data using AR replicas for robust regression.
problem Improving robustness in nonlinear over-parametrized regression.
method Extends Anchor regression (AR) for data augmentation, using replicas of modified samples.
result ADA provides more robust regression predictions compared to state-of-the-art solutions.
Augment small datasets with synthetic backgrounds to train lightweight CNNs for human pose estimation.
problem Training CNNs from limited real-world data for human pose estimation.
method Synthetic background substitution for data augmentation.
result Improves generalization to unseen environments.
Improves fairness in machine learning by adding underrepresented group data.
problem Machine learning biases across subgroups due to under-representation or societal biases.
method Data augmentation via pairwise mixup across subgroups to balance subpopulations.
result Achieves fair outcomes with robust if not improved accuracy.
The variational autoencoder (VAE) framework remains a popular option for training unsupervised generative models, especially for discrete data where generative adversarial networks (GANs) require workaround to create gradient for the generator. In our work modeling US postal addresses, we show that our discrete VAE wit…
SBA improves neural network generalization by dynamically augmenting data.
problem Lack of direct supervision in data augmentation for generalization.
method Stochastic Batch Augmentation with dynamic soft label regularization.
result SBA improves generalization and speeds up training.
WeMix improves data augmentation by correcting bias in deep learning.
problem Data augmentation's effectiveness is limited by data bias.
method Developed AugDrop and MixLoss algorithms to correct data bias.
result WeMix improves data augmentation performance through bias correction.
We extend recent work (Brehmer, et. al., 2018) that use neural networks as surrogate models for likelihood-free inference. As in the previous work, we exploit the fact that the joint likelihood ratio and joint score, conditioned on both observed and latent variables, can often be extracted from an implicit generative m…
Data augmentation doesn't improve robustness, contrary to belief.
problem The effectiveness of data augmentation in improving model robustness is questioned.
method Taking a Domain Generalization viewpoint, the study examines the robustness of augmented representations.
result Augmented representations are not robust to distortions used during training.
This paper tackles overfitting in CTR models by introducing Multi-Epoch learning with Data Augmentation.
problem Overfitting of the embedding layer in CTR models during multi-epoch training.
method Introduces Multi-Epoch learning with Data Augmentation (MEDA) framework to reduce overfitting and enhance performance.
result MEDA minimizes overfitting and achieves data augmentation through varied embedding spaces, improving performance without overfitting.
DRO-Augment framework enhances deep neural network robustness.
problem Robustness of deep neural networks against various perturbations and adversarial attacks.
method Integrates Wasserstein Distributionally Robust Optimization with data augmentation.
result Significantly improves robustness across various corruptions and adversarial attacks.
Improved neural PDEs trained on augmented data enhance model accuracy and efficiency.
problem Training neural PDEs on limited data to accurately represent complex systems.
method Space-filling sampling of local states to generate augmented training data.
result Data-augmented neural PDEs outperform traditional emulators in accuracy and stability.
T-JEPA learns tabular data representations without augmentations, outperforming traditional methods.
problem Challenges in self-supervised learning for tabular data due to lack of data augmentations.
method T-JEPA uses a Joint Embedding Predictive Architecture (JEPA) to predict latent representations of different subsets of features within the same sample.
result Significant improvement in classification and regression tasks, outperforming traditional methods.
Survey of data augmentation methods for improving deep learning on time series data.
problem Limited labeled data in real-world time series applications.
method Review and comparison of data augmentation methods for time series.
result Empirical comparison of data augmentation methods for various time series tasks.
Augments graph node features to improve GNN performance.
problem Improving graph neural networks' performance on large-scale datasets.
method Iteratively augments node features with gradient-based adversarial perturbations.
result Boosts model performance in node classification, link prediction, and graph classification tasks.