WeMix improves data augmentation by correcting bias in deep learning.
problem Data augmentation's effectiveness is limited by data bias.
method Developed AugDrop and MixLoss algorithms to correct data bias.
result WeMix improves data augmentation performance through bias correction.
Data augmentation has been widely applied as an effective methodology to improve generalization in particular when training deep neural networks. Recently, researchers proposed a few intensive data augmentation techniques, which indeed improved accuracy, yet we notice that these methods augment data have also caused a …
A study on optimizing data augmentation weights for improved test-time predictions.
problem Improving robustness of predictions during testing with data augmentation methods.
method A weighted Test-Time Augmentation (TTA) approach based on variational Bayesian framework to optimize weights.
result Optimizing weights suppresses unwanted data augmentations and improves prediction performance.
Automatically learns optimal data augmentation for image classification.
problem Finding optimal data augmentation hyperparameters is computationally demanding and requires domain knowledge.
method Proposes an online bilevel optimization framework to learn data augmentation parameters directly.
result Jointly trained method achieves comparable or better classification accuracy than hand-crafted data augmentation without an external validation loop.
New attacks show data augmentation may not improve privacy.
problem Measuring privacy risk in models trained with data augmentation.
method Formulated membership inference as a set classification problem, designed input permutation invariant features.
result Proposed approach universally outperforms original methods on models trained with data augmentation.
Data augmentation doesn't improve robustness, contrary to belief.
problem The effectiveness of data augmentation in improving model robustness is questioned.
method Taking a Domain Generalization viewpoint, the study examines the robustness of augmented representations.
result Augmented representations are not robust to distortions used during training.
Graph data augmentation improves GNN performance in node classification.
problem Improving generalizability of graph neural networks (GNNs) in semi-supervised node classification.
method Introduces GAug framework for graph data augmentation using neural edge predictors.
result GAug framework improves GNN-based node classification performance across various architectures and datasets.
Data augmentation can achieve the same statistical benefits as full augmentation up to an approximation error.
problem Data augmentation in learning problems
method Using Fourier analysis and representation theory of finite groups
result Partial data augmentation achieves the same minimax rates as full augmentation
Study examines how data augmentation impacts optimization in linear regression.
problem Understanding how data augmentation schedules affect optimization in linear regression.
method Analyzed the effect of augmentation on optimization in linear regression with MSE loss, using classical convex optimization and recent work on implicit bias.
result Proved that under certain joint schedules for learning rate and augmentation scheme, augmented gradient descent converges and characterized the resulting minimum.
ADA augments data using AR replicas for robust regression.
problem Improving robustness in nonlinear over-parametrized regression.
method Extends Anchor regression (AR) for data augmentation, using replicas of modified samples.
result ADA provides more robust regression predictions compared to state-of-the-art solutions.
Develops a statistical framework for self-supervised representation learning using data augmentation.
problem Lack of theoretical understanding of data augmentation in nonlinear settings.
method Augmentation invariant manifold learning framework and stochastic optimization algorithm.
result Improves downstream analysis by exploiting manifold's geometric structure and invariant property of augmented data.
Data augmentation impacts adversarial risk; careful application recommended.
problem Understanding how data augmentation affects adversarial risk in deep learning.
method Empirical analysis using three measures of adversarial risk.
result Data augmentation does not always improve adversarial risk; augmented data influences models more.
Data augmentation affects feature importance, enhancing learning for neural networks.
problem Understanding the effect of data augmentation on feature importance and learning dynamics.
method Analyzing a two-layer convolutional neural network in a multi-view data model.
result Data augmentation alters feature importance, making certain features more likely to be learned.
This work characterizes how data augmentation shapes neural representations.
problem Understanding the impact of data augmentation on neural network representations.
method Embedding neural network hidden representations into a metric space invariant to transformations, analyzing shape-space trajectories.
result Increasing data augmentation strength leads to well-behaved trajectories in the embedded space, and different augmentation types steer representations in distinct directions.
Data augmentation improves microbiome disease prediction.
problem Improving predictive models for microbiome data.
method Defined novel data augmentation strategies for simplex-valued data.
result Set new state-of-the-art for disease prediction tasks.
A method to prevent image representation collapse through data-dependent augmentation.
problem Representation collapse due to image augmentations that damage information.
method Formalizing a stochastic encoding process with a tug-of-war between corruption and preserved information, using infoMax objective.
result Learning a data-dependent distribution of augmentations to avoid representation collapse.
Counterfactual data augmentations may not ensure OOD robustness if performed by a context-guessing machine.
problem Deep learning models lack out-of-distribution robustness due to reliance on spurious features.
method Theoretical analysis and demonstration of counterfactual data augmentations performed by a context-guessing machine.
result Counterfactual data augmentations by a context-guessing machine do not lead to robust OOD classifiers.
Survey of data augmentation techniques for time series classification with neural networks.
problem Small datasets in time series recognition.
method Four families of data augmentation: transformation-based, pattern mixing, generative models, and decomposition methods.
result Empirical evaluation of 12 data augmentation methods on 128 datasets.
Data augmentations are important ingredients in the recipe for training robust neural networks, especially in computer vision. A fundamental question is whether neural network features encode data augmentation transformations. To answer this question, we introduce a systematic approach to investigate which layers of ne…
Self-training with noisy student-teacher boosts keyword spotting accuracy.
problem Robust keyword spotting in challenging conditions.
method Aggressive data augmentation and self-training with noisy student-teacher approach.
result Significant accuracy improvement in difficult conditions, up to 60%.
Though data augmentation has become a standard component of deep neural network training, the underlying mechanism behind the effectiveness of these techniques remains poorly understood. In practice, augmentation policies are often chosen using heuristics of either distribution shift or augmentation diversity. Inspired…
SapAugment learns adaptive augmentation policies for better model training.
problem Fixed data augmentation methods often apply the same augmentation to all samples, ignoring sample difficulty.
method SapAugment adapts augmentation parameters based on training loss, learning a sample-adaptive policy.
result SapAugment achieves up to 21% relative reduction in word error rate on LibriSpeech dataset.
Bayesian neural networks with data augmentation show a persistent cold posterior effect.
problem Understanding the cold posterior effect in Bayesian neural networks with data augmentation.
method Developed principled Bayesian neural networks using data augmentation, providing exact likelihoods and tight bounds.
result The cold posterior effect persists even in models incorporating data augmentation, suggesting it's not an artifact.
The paper shows how data augmentation and regularization can enforce group equivariance in machine learning models.
problem Improving model performance by leveraging known symmetries in machine learning tasks.
method Training with data augmentation and regularization to enforce group equivariance.
result Equivariance of the trained model can be achieved through training on augmented data in tandem with regularization.
Augmentation improves machine learning model performance on small datasets.
problem Suboptimal generalization performance of machine learning models on small datasets.
method Data augmentation to increase sample size and diversity.
result Augmentation improves AUC by 15.55% on average for small datasets.
Online data augmentation improves forecasting performance in deep learning.
problem Insufficient training data for forecasting tasks.
method An online data augmentation framework that generates synthetic samples during training.
result Online data augmentation leads to better forecasting performance.
Study ablated data augmentation techniques and their mathematical equivalence to penalties.
problem Lack of mathematical understanding of differences between ablated data augmentation techniques.
method Formal model of mean ablated data augmentation and inverted dropout for linear regression; empirical validation for deep networks.
result Ablated data augmentation and inverted dropout are mathematically equivalent to penalties in optimization.
SODA optimizes data augmentation allocation for deep learning models.
problem Inefficient allocation of data augmentation budget in deep neural networks.
method Online learning to dynamically allocate data augmentation budget during training.
result Optimized data augmentation can save computation time and promote greener machine learning.
A recurring problem faced when training neural networks is that there is typically not enough data to maximize the generalization capability of deep neural networks(DNN). There are many techniques to address this, including data augmentation, dropout, and transfer learning. In this paper, we introduce an additional met…
The paper explains how data augmentation improves semi-supervised learning efficiency.
problem Improving accuracy from a small fraction of labeled data.
method Data augmentation induces a similarity graph, which is graph-Laplacian-regularized for downstream learning.
result A fast transductive rate of O(1/nL) is achieved, reducing the number of labels needed. Data augmentation, a technique in which a training set is expanded with class-preserving transformations, is ubiquitous in modern machine learning pipelines. In this paper, we seek to establish a theoretical framework for understanding data augmentation. We approach this from two directions: First, we provide a general…
Test-time data augmentation−averaging the predictions of a machine learning model across multiple augmented samples of data−is a widely used technique that improves the predictive performance. While many advanced learnable data augmentation techniques have emerged in recent years, they are focused on the training p…
Data augmentation improves financial prediction models, especially for small datasets.
problem Improving financial prediction models on small, noisy, non-stationary datasets.
method Evaluation of data augmentation methods combined with deep learning models on financial datasets.
result Data augmentation significantly improves financial performance, up to 400% improvement in risk-adjusted return.
Task-agnostic data augmentation shows little benefit for pretrained transformers.
problem Evaluating the effectiveness of task-agnostic data augmentation on pretrained transformers.
method Conducted a systematic examination of two data augmentation techniques (Easy Data Augmentation and Back-Translation) across 5 tasks, 6 datasets, and 3 pretrained transformer models.
result Data augmentation techniques previously effective for non-pretrained models fail to consistently improve performance for pretrained transformers, even with limited training data.
Theoretical framework for data augmentation in finance improves portfolio construction.
problem Improving portfolio construction in speculative markets.
method Developed a theoretical framework for data augmentation and regularization in deep learning for finance.
result A simple noise injection algorithm improves portfolio construction over no noise.
This paper tackles confounding biases in data augmentation.
problem Mitigating spurious correlations and confounding variables in training data.
method Formal analysis and counterfactual data augmentation.
result Removing confounding biases leads to invariant features and better generalization.
ODVICE augments EHR cohorts using ontology to improve analysis robustness.
problem Limited records in cohorts for rare diseases hamper robust analysis.
method Ontology-driven Monte-Carlo graph spanning algorithm for data augmentation.
result ODVICE augmented cohorts show ~30% improvement in AUC over non-augmented datasets.
Adaptive weighting schemes enhance time-series data augmentation for financial and UCR datasets.
problem Limited size of time-series datasets hinders model performance.
method Two adaptive weighting schemes for automatic data augmentation.
result Improves annualized returns by over 50% on financial dataset and outperforms state-of-the-art on half of UCR datasets.
Enhances reinforcement learning from sparse data.
problem Limited data for offline reinforcement learning.
method Trajectory-based data augmentation.
result Improves reinforcement learning performance.
New framework explains data augmentation's role in machine learning.
problem Understanding how data augmentation affects generalization and invariance learning.
method Information-theoretic framework based on mutual information bounds and orbit-averaged loss functions.
result Derives a new generalization bound decomposing the generalization gap into three interpretable terms.
Data augmentation is commonly used to encode invariances in learning methods. However, this process is often performed in an inefficient manner, as artificial examples are created by applying a number of transformations to all points in the training set. The resulting explosion of the dataset size can be an issue in te…
DRO-Augment framework enhances deep neural network robustness.
problem Robustness of deep neural networks against various perturbations and adversarial attacks.
method Integrates Wasserstein Distributionally Robust Optimization with data augmentation.
result Significantly improves robustness across various corruptions and adversarial attacks.
A key challenge in leveraging data augmentation for neural network training is choosing an effective augmentation policy from a large search space of candidate operations. Properly chosen augmentation policies can lead to significant generalization improvements; however, state-of-the-art approaches such as AutoAugment …
Augments graph node features to improve GNN performance.
problem Improving graph neural networks' performance on large-scale datasets.
method Iteratively augments node features with gradient-based adversarial perturbations.
result Boosts model performance in node classification, link prediction, and graph classification tasks.
Transfer learning and data augmentation improve stock classification performance.
problem Challenges in stock classification due to noise and volatility.
method Pre-trained model on S&P500 index features, transfer learning to new models, data augmentation on feature space.
result Augmentation on feature space leads to 20% increase in risk-adjusted returns.
Data augmentation affects estimates' uncertainty and distribution in complex ways.
problem Understanding how data augmentation impacts the variance and limiting distribution of estimates.
method Developed an adaptation of Lindeberg's technique for block dependence.
result Data augmentation can increase rather than decrease uncertainty, and it may shift the double-descent peak of an empirical risk.
New framework explains diverse impacts of data augmentation.
problem Understanding the varied effects of data augmentation on model performance.
method Developed a theoretical framework to characterize DA's impact on linear models.
result Data augmentation induces implicit spectral regularization through two effects.
The application of deep learning to build accurate predictive models from functional neuroimaging data is often hindered by limited dataset sizes. Though data augmentation can help mitigate such training obstacles, most data augmentation methods have been developed for natural images as in computer vision tasks such as…