WeMix improves data augmentation by correcting bias in deep learning.
problem Data augmentation's effectiveness is limited by data bias.
method Developed AugDrop and MixLoss algorithms to correct data bias.
result WeMix improves data augmentation performance through bias correction.
This paper revisits the distribution gap between clean and augmented data in deep learning.
problem The distribution gap between clean and augmented data in deep learning models.
method Analytical perspective using empirical risk and generalization error, highlighting data augmentation as regularization.
result Data augmentation significantly reduces generalization error but slightly increases empirical risk.
A study on optimizing data augmentation weights for improved test-time predictions.
problem Improving robustness of predictions during testing with data augmentation methods.
method A weighted Test-Time Augmentation (TTA) approach based on variational Bayesian framework to optimize weights.
result Optimizing weights suppresses unwanted data augmentations and improves prediction performance.
New measures quantify how data augmentation improves model performance.
problem Understanding the effectiveness of data augmentation in deep learning.
method Introduced Affinity and Diversity measures to quantify augmentation performance.
result Augmentation performance is best achieved by optimizing both Affinity and Diversity.
CNNs encode data augmentation transformations, especially in early layers.
problem Whether neural network features encode data augmentation transformations.
method Systematic approach using pre-trained vision models to predict augmentation transformations.
result Neural network features encode data augmentation transformations, especially in early layers.
Automatically learns optimal data augmentation for image classification.
problem Finding optimal data augmentation hyperparameters is computationally demanding and requires domain knowledge.
method Proposes an online bilevel optimization framework to learn data augmentation parameters directly.
result Jointly trained method achieves comparable or better classification accuracy than hand-crafted data augmentation without an external validation loop.
New attacks show data augmentation may not improve privacy.
problem Measuring privacy risk in models trained with data augmentation.
method Formulated membership inference as a set classification problem, designed input permutation invariant features.
result Proposed approach universally outperforms original methods on models trained with data augmentation.
Data augmentation doesn't improve robustness, contrary to belief.
problem The effectiveness of data augmentation in improving model robustness is questioned.
method Taking a Domain Generalization viewpoint, the study examines the robustness of augmented representations.
result Augmented representations are not robust to distortions used during training.
Graph data augmentation improves GNN performance in node classification.
problem Improving generalizability of graph neural networks (GNNs) in semi-supervised node classification.
method Introduces GAug framework for graph data augmentation using neural edge predictors.
result GAug framework improves GNN-based node classification performance across various architectures and datasets.
Data augmentation can achieve the same statistical benefits as full augmentation up to an approximation error.
problem Data augmentation in learning problems
method Using Fourier analysis and representation theory of finite groups
result Partial data augmentation achieves the same minimax rates as full augmentation
Study examines how data augmentation impacts optimization in linear regression.
problem Understanding how data augmentation schedules affect optimization in linear regression.
method Analyzed the effect of augmentation on optimization in linear regression with MSE loss, using classical convex optimization and recent work on implicit bias.
result Proved that under certain joint schedules for learning rate and augmentation scheme, augmented gradient descent converges and characterized the resulting minimum.
ADA augments data using AR replicas for robust regression.
problem Improving robustness in nonlinear over-parametrized regression.
method Extends Anchor regression (AR) for data augmentation, using replicas of modified samples.
result ADA provides more robust regression predictions compared to state-of-the-art solutions.
Develops a statistical framework for self-supervised representation learning using data augmentation.
problem Lack of theoretical understanding of data augmentation in nonlinear settings.
method Augmentation invariant manifold learning framework and stochastic optimization algorithm.
result Improves downstream analysis by exploiting manifold's geometric structure and invariant property of augmented data.
Data augmentation impacts adversarial risk; careful application recommended.
problem Understanding how data augmentation affects adversarial risk in deep learning.
method Empirical analysis using three measures of adversarial risk.
result Data augmentation does not always improve adversarial risk; augmented data influences models more.
Data augmentation affects feature importance, enhancing learning for neural networks.
problem Understanding the effect of data augmentation on feature importance and learning dynamics.
method Analyzing a two-layer convolutional neural network in a multi-view data model.
result Data augmentation alters feature importance, making certain features more likely to be learned.
Simple policy search outperforms advanced learnable test-time augmentation techniques.
problem Improving predictive performance through test-time data augmentation.
method Greedy policy search (GPS) for learning test-time augmentation policies.
result Augmentation policies learned with GPS achieve superior predictive performance and robustness.
This work characterizes how data augmentation shapes neural representations.
problem Understanding the impact of data augmentation on neural network representations.
method Embedding neural network hidden representations into a metric space invariant to transformations, analyzing shape-space trajectories.
result Increasing data augmentation strength leads to well-behaved trajectories in the embedded space, and different augmentation types steer representations in distinct directions.
Data augmentation improves microbiome disease prediction.
problem Improving predictive models for microbiome data.
method Defined novel data augmentation strategies for simplex-valued data.
result Set new state-of-the-art for disease prediction tasks.
A method to prevent image representation collapse through data-dependent augmentation.
problem Representation collapse due to image augmentations that damage information.
method Formalizing a stochastic encoding process with a tug-of-war between corruption and preserved information, using infoMax objective.
result Learning a data-dependent distribution of augmentations to avoid representation collapse.
Counterfactual data augmentations may not ensure OOD robustness if performed by a context-guessing machine.
problem Deep learning models lack out-of-distribution robustness due to reliance on spurious features.
method Theoretical analysis and demonstration of counterfactual data augmentations performed by a context-guessing machine.
result Counterfactual data augmentations by a context-guessing machine do not lead to robust OOD classifiers.
Survey of data augmentation techniques for time series classification with neural networks.
problem Small datasets in time series recognition.
method Four families of data augmentation: transformation-based, pattern mixing, generative models, and decomposition methods.
result Empirical evaluation of 12 data augmentation methods on 128 datasets.
PBA generates nonstationary augmentation schedules to match AutoAugment's performance with less compute.
problem Choosing an effective augmentation policy from a large search space.
method Population Based Augmentation (PBA) generates nonstationary augmentation policy schedules.
result PBA matches AutoAugment's performance on CIFAR-10, CIFAR-100, and SVHN with less compute.
Self-training with noisy student-teacher boosts keyword spotting accuracy.
problem Robust keyword spotting in challenging conditions.
method Aggressive data augmentation and self-training with noisy student-teacher approach.
result Significant accuracy improvement in difficult conditions, up to 60%.
Develops a mathematical framework for data augmentation.
problem Lack of a clear mathematical explanation for data augmentation's performance benefits.
method Group-theoretic approach to data augmentation, showing it as averaging over orbits of a group.
result Data augmentation leads to variance reduction and improved model performance.
SapAugment learns adaptive augmentation policies for better model training.
problem Fixed data augmentation methods often apply the same augmentation to all samples, ignoring sample difficulty.
method SapAugment adapts augmentation parameters based on training loss, learning a sample-adaptive policy.
result SapAugment achieves up to 21% relative reduction in word error rate on LibriSpeech dataset.
Bayesian neural networks with data augmentation show a persistent cold posterior effect.
problem Understanding the cold posterior effect in Bayesian neural networks with data augmentation.
method Developed principled Bayesian neural networks using data augmentation, providing exact likelihoods and tight bounds.
result The cold posterior effect persists even in models incorporating data augmentation, suggesting it's not an artifact.
The paper shows how data augmentation and regularization can enforce group equivariance in machine learning models.
problem Improving model performance by leveraging known symmetries in machine learning tasks.
method Training with data augmentation and regularization to enforce group equivariance.
result Equivariance of the trained model can be achieved through training on augmented data in tandem with regularization.
Augmentation improves machine learning model performance on small datasets.
problem Suboptimal generalization performance of machine learning models on small datasets.
method Data augmentation to increase sample size and diversity.
result Augmentation improves AUC by 15.55% on average for small datasets.
Online data augmentation improves forecasting performance in deep learning.
problem Insufficient training data for forecasting tasks.
method An online data augmentation framework that generates synthetic samples during training.
result Online data augmentation leads to better forecasting performance.
Study ablated data augmentation techniques and their mathematical equivalence to penalties.
problem Lack of mathematical understanding of differences between ablated data augmentation techniques.
method Formal model of mean ablated data augmentation and inverted dropout for linear regression; empirical validation for deep networks.
result Ablated data augmentation and inverted dropout are mathematically equivalent to penalties in optimization.
Adversarial learning improves image augmentation for neural networks.
problem Improving data augmentation for neural networks with limited data.
method Adversarial learning using an encoder-decoder architecture with a spatial transformer network.
result Our approach outperforms previous generative data augmentation methods.
SODA optimizes data augmentation allocation for deep learning models.
problem Inefficient allocation of data augmentation budget in deep neural networks.
method Online learning to dynamically allocate data augmentation budget during training.
result Optimized data augmentation can save computation time and promote greener machine learning.
A recurring problem faced when training neural networks is that there is typically not enough data to maximize the generalization capability of deep neural networks(DNN). There are many techniques to address this, including data augmentation, dropout, and transfer learning. In this paper, we introduce an additional met…
The paper explains how data augmentation improves semi-supervised learning efficiency.
problem Improving accuracy from a small fraction of labeled data.
method Data augmentation induces a similarity graph, which is graph-Laplacian-regularized for downstream learning.
result A fast transductive rate of O(1/nL) is achieved, reducing the number of labels needed. Data augmentation, a technique in which a training set is expanded with class-preserving transformations, is ubiquitous in modern machine learning pipelines. In this paper, we seek to establish a theoretical framework for understanding data augmentation. We approach this from two directions: First, we provide a general…
Data augmentation improves financial prediction models, especially for small datasets.
problem Improving financial prediction models on small, noisy, non-stationary datasets.
method Evaluation of data augmentation methods combined with deep learning models on financial datasets.
result Data augmentation significantly improves financial performance, up to 400% improvement in risk-adjusted return.
Task-agnostic data augmentation shows little benefit for pretrained transformers.
problem Evaluating the effectiveness of task-agnostic data augmentation on pretrained transformers.
method Conducted a systematic examination of two data augmentation techniques (Easy Data Augmentation and Back-Translation) across 5 tasks, 6 datasets, and 3 pretrained transformer models.
result Data augmentation techniques previously effective for non-pretrained models fail to consistently improve performance for pretrained transformers, even with limited training data.
Graph imputation neural network (GINN) augments datasets by reconstructing damaged nodes.
problem Efficient data augmentation in semi-supervised learning with limited labeled data.
method Graph-based neural network (GINN) for missing data imputation and data augmentation.
result GINN can significantly improve semi-supervised learning performance and augment datasets up to 10x.
Theoretical framework for data augmentation in finance improves portfolio construction.
problem Improving portfolio construction in speculative markets.
method Developed a theoretical framework for data augmentation and regularization in deep learning for finance.
result A simple noise injection algorithm improves portfolio construction over no noise.
ODVICE augments EHR cohorts using ontology to improve analysis robustness.
problem Limited records in cohorts for rare diseases hamper robust analysis.
method Ontology-driven Monte-Carlo graph spanning algorithm for data augmentation.
result ODVICE augmented cohorts show ~30% improvement in AUC over non-augmented datasets.
This paper tackles confounding biases in data augmentation.
problem Mitigating spurious correlations and confounding variables in training data.
method Formal analysis and counterfactual data augmentation.
result Removing confounding biases leads to invariant features and better generalization.
Adaptive weighting schemes enhance time-series data augmentation for financial and UCR datasets.
problem Limited size of time-series datasets hinders model performance.
method Two adaptive weighting schemes for automatic data augmentation.
result Improves annualized returns by over 50% on financial dataset and outperforms state-of-the-art on half of UCR datasets.
This work improves deep learning models for fMRI by generating realistic brain morphology images.
problem Limited dataset sizes for functional MRI limit the accuracy of deep learning models.
method Proposes a method to generate new fMRI images with realistic brain morphology.
result Demonstrates a 26% improvement in predicting antidepressant treatment response using augmented images.
Enhances reinforcement learning from sparse data.
problem Limited data for offline reinforcement learning.
method Trajectory-based data augmentation.
result Improves reinforcement learning performance.
New framework explains data augmentation's role in machine learning.
problem Understanding how data augmentation affects generalization and invariance learning.
method Information-theoretic framework based on mutual information bounds and orbit-averaged loss functions.
result Derives a new generalization bound decomposing the generalization gap into three interpretable terms.
Data augmentation is commonly used to encode invariances in learning methods. However, this process is often performed in an inefficient manner, as artificial examples are created by applying a number of transformations to all points in the training set. The resulting explosion of the dataset size can be an issue in te…
New insights into data augmentation for improving robustness in computer vision.
problem Challenges in achieving robustness to distributional shift in computer vision.
method Investigation of trade-offs between robustness to high and low frequency corruptions using Gaussian data augmentation and adversarial training.
result Improvements in robustness to high frequency corruptions come at the cost of reduced robustness to low frequency corruptions.
DRO-Augment framework enhances deep neural network robustness.
problem Robustness of deep neural networks against various perturbations and adversarial attacks.
method Integrates Wasserstein Distributionally Robust Optimization with data augmentation.
result Significantly improves robustness across various corruptions and adversarial attacks.