New measures quantify how data augmentation improves model performance.
problem Understanding the effectiveness of data augmentation in deep learning.
method Introduced Affinity and Diversity measures to quantify augmentation performance.
result Augmentation performance is best achieved by optimizing both Affinity and Diversity.
Expert augmentation improves hybrid model generalization.
problem Limited generalization of hybrid models outside training distribution.
method Introducing expert augmentation to improve hybrid model performance.
result Expert augmentation improves generalization of hybrid models.
This paper improves auto-augment efficiency by sharing augmentation weights.
problem Efficient evaluation of augmentation policies for model training.
method Augmentation-Wise Weight Sharing (AWS) to create a fast yet accurate proxy task.
result Augmentation policies found achieve superior accuracies compared to existing methods.
New attacks show data augmentation may not improve privacy.
problem Measuring privacy risk in models trained with data augmentation.
method Formulated membership inference as a set classification problem, designed input permutation invariant features.
result Proposed approach universally outperforms original methods on models trained with data augmentation.
Data augmentation doesn't improve robustness, contrary to belief.
problem The effectiveness of data augmentation in improving model robustness is questioned.
method Taking a Domain Generalization viewpoint, the study examines the robustness of augmented representations.
result Augmented representations are not robust to distortions used during training.
Data augmentation has been widely applied as an effective methodology to improve generalization in particular when training deep neural networks. Recently, researchers proposed a few intensive data augmentation techniques, which indeed improved accuracy, yet we notice that these methods augment data have also caused a …
Transfer learning and data augmentation improve stock classification performance.
problem Challenges in stock classification due to noise and volatility.
method Pre-trained model on S&P500 index features, transfer learning to new models, data augmentation on feature space.
result Augmentation on feature space leads to 20% increase in risk-adjusted returns.
Augmentation improves machine learning model performance on small datasets.
problem Suboptimal generalization performance of machine learning models on small datasets.
method Data augmentation to increase sample size and diversity.
result Augmentation improves AUC by 15.55% on average for small datasets.
Data augmentation affects feature importance, enhancing learning for neural networks.
problem Understanding the effect of data augmentation on feature importance and learning dynamics.
method Analyzing a two-layer convolutional neural network in a multi-view data model.
result Data augmentation alters feature importance, making certain features more likely to be learned.
The paper shows how data augmentation and regularization can enforce group equivariance in machine learning models.
problem Improving model performance by leveraging known symmetries in machine learning tasks.
method Training with data augmentation and regularization to enforce group equivariance.
result Equivariance of the trained model can be achieved through training on augmented data in tandem with regularization.
Data augmentation impacts adversarial risk; careful application recommended.
problem Understanding how data augmentation affects adversarial risk in deep learning.
method Empirical analysis using three measures of adversarial risk.
result Data augmentation does not always improve adversarial risk; augmented data influences models more.
CNNs encode data augmentation transformations, especially in early layers.
problem Whether neural network features encode data augmentation transformations.
method Systematic approach using pre-trained vision models to predict augmentation transformations.
result Neural network features encode data augmentation transformations, especially in early layers.
SapAugment learns adaptive augmentation policies for better model training.
problem Fixed data augmentation methods often apply the same augmentation to all samples, ignoring sample difficulty.
method SapAugment adapts augmentation parameters based on training loss, learning a sample-adaptive policy.
result SapAugment achieves up to 21% relative reduction in word error rate on LibriSpeech dataset.
Simple policy search outperforms advanced learnable test-time augmentation techniques.
problem Improving predictive performance through test-time data augmentation.
method Greedy policy search (GPS) for learning test-time augmentation policies.
result Augmentation policies learned with GPS achieve superior predictive performance and robustness.
Data augmentation improves microbiome disease prediction.
problem Improving predictive models for microbiome data.
method Defined novel data augmentation strategies for simplex-valued data.
result Set new state-of-the-art for disease prediction tasks.
This work analyzes the role of data augmentation in self-supervised learning using RKHS approximation and regression.
problem Limited theoretical understanding of the role of data augmentation in self-supervised learning.
method Geometric characterization of the target function given by augmentation, proving generalization bounds.
result Two generalization bounds are derived, one free of model complexity, the other specific to near-optimal encoders.
Data augmentation, a technique in which a training set is expanded with class-preserving transformations, is ubiquitous in modern machine learning pipelines. In this paper, we seek to establish a theoretical framework for understanding data augmentation. We approach this from two directions: First, we provide a general…
Automatically learns optimal data augmentation for image classification.
problem Finding optimal data augmentation hyperparameters is computationally demanding and requires domain knowledge.
method Proposes an online bilevel optimization framework to learn data augmentation parameters directly.
result Jointly trained method achieves comparable or better classification accuracy than hand-crafted data augmentation without an external validation loop.
Bayesian neural networks with data augmentation show a persistent cold posterior effect.
problem Understanding the cold posterior effect in Bayesian neural networks with data augmentation.
method Developed principled Bayesian neural networks using data augmentation, providing exact likelihoods and tight bounds.
result The cold posterior effect persists even in models incorporating data augmentation, suggesting it's not an artifact.
Task-agnostic data augmentation shows little benefit for pretrained transformers.
problem Evaluating the effectiveness of task-agnostic data augmentation on pretrained transformers.
method Conducted a systematic examination of two data augmentation techniques (Easy Data Augmentation and Back-Translation) across 5 tasks, 6 datasets, and 3 pretrained transformer models.
result Data augmentation techniques previously effective for non-pretrained models fail to consistently improve performance for pretrained transformers, even with limited training data.
A study on optimizing data augmentation weights for improved test-time predictions.
problem Improving robustness of predictions during testing with data augmentation methods.
method A weighted Test-Time Augmentation (TTA) approach based on variational Bayesian framework to optimize weights.
result Optimizing weights suppresses unwanted data augmentations and improves prediction performance.
For training the sequence-to-sequence voice conversion model, we need to handle an issue of insufficient data about the number of speech pairs which consist of the same utterance. This study experimentally investigated the effects of Mel-spectrogram augmentation on training the sequence-to-sequence voice conversion (VC…
Data augmentation improves financial prediction models, especially for small datasets.
problem Improving financial prediction models on small, noisy, non-stationary datasets.
method Evaluation of data augmentation methods combined with deep learning models on financial datasets.
result Data augmentation significantly improves financial performance, up to 400% improvement in risk-adjusted return.
Graph data augmentation improves GNN performance in node classification.
problem Improving generalizability of graph neural networks (GNNs) in semi-supervised node classification.
method Introduces GAug framework for graph data augmentation using neural edge predictors.
result GAug framework improves GNN-based node classification performance across various architectures and datasets.
This work investigates image augmentations for GAN training, improving image quality.
problem Improving the accuracy and robustness of GAN models for image synthesis.
method Systematic study of various image augmentation techniques for GAN training.
result Vanilla GANs can achieve state-of-the-art generation quality with image augmentations.
Proposes a new framework for learning image augmentations to improve classification performance.
problem Improving classification performance with a given class of predictors.
method Transformed Risk Minimization (TRM) framework that optimizes both predictive models and data transformations.
result Performance of TRM with SCALE algorithm compares favorably to prior methods on CIFAR10/100.
Augments graph node features to improve GNN performance.
problem Improving graph neural networks' performance on large-scale datasets.
method Iteratively augments node features with gradient-based adversarial perturbations.
result Boosts model performance in node classification, link prediction, and graph classification tasks.
New method speeds up inference for non-conjugate Gaussian processes.
problem Inference for non-conjugate Gaussian processes is slow and unreliable.
method Automated augmented conjugate inference method that constructs auxiliary variables to make the model conditionally conjugate.
result Our method is up to two orders of magnitude faster and more robust than existing methods.
This work characterizes how data augmentation shapes neural representations.
problem Understanding the impact of data augmentation on neural network representations.
method Embedding neural network hidden representations into a metric space invariant to transformations, analyzing shape-space trajectories.
result Increasing data augmentation strength leads to well-behaved trajectories in the embedded space, and different augmentation types steer representations in distinct directions.
Data augmentation can achieve the same statistical benefits as full augmentation up to an approximation error.
problem Data augmentation in learning problems
method Using Fourier analysis and representation theory of finite groups
result Partial data augmentation achieves the same minimax rates as full augmentation
Graph Neural Networks struggle with generalization, especially OOD data; GRATIN solves this with Gaussian Mixture Model-based augmentation.
problem Graph Neural Networks struggle with generalization, particularly to unseen or out-of-distribution data.
method Theoretical framework using Rademacher complexity to compute a regret bound on generalization error. GRATIN algorithm leveraging Gaussian Mixture Models for efficient data augmentation.
result GRATIN outperforms existing augmentation techniques in terms of generalization and offers improved time complexity.
VFlow enhances generative flows by augmenting data dimensions for better expressiveness.
problem Tractable generative flows have limited expressiveness due to fixed intermediate dimensions.
method Augment data with extra dimensions and learn a generative flow for both original and augmented data using variational inference.
result VFlow achieves state-of-the-art performance on CIFAR-10 with improved compactness.
Counterfactual data augmentations may not ensure OOD robustness if performed by a context-guessing machine.
problem Deep learning models lack out-of-distribution robustness due to reliance on spurious features.
method Theoretical analysis and demonstration of counterfactual data augmentations performed by a context-guessing machine.
result Counterfactual data augmentations by a context-guessing machine do not lead to robust OOD classifiers.
ARDA automatically augments datasets for machine learning models.
problem Improving machine learning model performance through data augmentation.
method ARDA combines data searching, joining, and feature selection to augment datasets.
result Training models on ARDA's augmented datasets leads to better performance.
Combining ensembles and data augmentation harms model calibration.
problem Improving model calibration and robustness with ensembles and data augmentation leads to a trade-off.
method Combining ensemble averaging and data augmentation techniques.
result Combining ensembles and data augmentation can harm model calibration.
Bayesian model selection optimizes data augmentation for improved machine learning robustness.
problem Choosing optimal data augmentation parameters is challenging and often done through trial and error.
method Interprets augmentation parameters as model hyperparameters and uses Bayesian model selection to optimize them.
result Our approach improves calibration and robust performance on various tasks.
Self-training with noisy student-teacher boosts keyword spotting accuracy.
problem Robust keyword spotting in challenging conditions.
method Aggressive data augmentation and self-training with noisy student-teacher approach.
result Significant accuracy improvement in difficult conditions, up to 60%.
This work analyzes benefits and limitations of data augmentation and feature averaging in deep learning models.
problem Theoretical understanding of incorporating invariance into deep learning models is lacking.
method Data augmentation and feature averaging are analyzed in the context of invariance in deep learning.
result Training with data augmentation leads to better estimates of risk and gradients, and feature averaging reduces generalization error with convex losses.
Study ablated data augmentation techniques and their mathematical equivalence to penalties.
problem Lack of mathematical understanding of differences between ablated data augmentation techniques.
method Formal model of mean ablated data augmentation and inverted dropout for linear regression; empirical validation for deep networks.
result Ablated data augmentation and inverted dropout are mathematically equivalent to penalties in optimization.
Synthetic augmentation improves financial machine learning performance in variance-dominant regimes.
problem Data scarcity in financial machine learning.
method Formalized synthetic augmentation, introduced size-matched null augmentation, and developed a non-parametric block permutation test.
result Synthetic augmentation is beneficial only in variance-dominant regimes, such as persistent volatility forecasting.
Data augmentation methods improve worst-case model performance.
problem Ensuring fair predictions across subpopulations in large models.
method Linear last layer retraining with data augmentation techniques.
result Optimal worst-group accuracy achieved for Gaussian latent representation distribution.
PANDA augments data to regularize GLM estimation and inference.
problem Regularizing estimation and inference in GLMs with noisy data.
method Iteratively optimizes augmented noise data to converge to regularized model estimates.
result Established convergence and asymptotic distributions for regularized parameters.
DRO-Augment framework enhances deep neural network robustness.
problem Robustness of deep neural networks against various perturbations and adversarial attacks.
method Integrates Wasserstein Distributionally Robust Optimization with data augmentation.
result Significantly improves robustness across various corruptions and adversarial attacks.
Develops a statistical framework for self-supervised representation learning using data augmentation.
problem Lack of theoretical understanding of data augmentation in nonlinear settings.
method Augmentation invariant manifold learning framework and stochastic optimization algorithm.
result Improves downstream analysis by exploiting manifold's geometric structure and invariant property of augmented data.
Augmented KRnet improves flow-based generative modeling by maintaining exact invertibility.
problem Maintaining exact invertibility in flow-based generative models.
method Integrates augmented dimensions into KRnet to achieve full nonlinear updates in two iterations, keeping exact invertibility.
result Augmented KRnet achieves full nonlinear updates in two iterations, maintaining exact invertibility.
Enhanced EEG classification using augmented covariance matrix.
problem Improving motor imagery classification from EEG signals.
method Proposes a new framework based on the augmented covariance matrix derived from an autoregressive model.
result The augmented covariance matrix outperformed state-of-the-art methods.
Meta-learning overfits both model and base learner; meta-augmentation helps.
problem Meta-learning overfits model and base learner.
method Use an information-theoretic framework to describe and demonstrate meta-augmentation.
result Meta-augmentation produces large complementary benefits to meta-regularization.
Paper proposes SiSTA for single-shot domain adaptation using target-aware generative augmentation.
problem Adapting models from source to target domains with limited target data.
method Fine-tunes a generative model on a single-shot target and uses novel sampling strategies for synthetic data.
result Improves performance by up to 20% over existing baselines in face attribute detection.