This work characterizes how data augmentation shapes neural representations.
problem Understanding the impact of data augmentation on neural network representations.
method Embedding neural network hidden representations into a metric space invariant to transformations, analyzing shape-space trajectories.
result Increasing data augmentation strength leads to well-behaved trajectories in the embedded space, and different augmentation types steer representations in distinct directions.
Transfer learning and data augmentation improve stock classification performance.
problem Challenges in stock classification due to noise and volatility.
method Pre-trained model on S&P500 index features, transfer learning to new models, data augmentation on feature space.
result Augmentation on feature space leads to 20% increase in risk-adjusted returns.
Data augmentation can achieve the same statistical benefits as full augmentation up to an approximation error.
problem Data augmentation in learning problems
method Using Fourier analysis and representation theory of finite groups
result Partial data augmentation achieves the same minimax rates as full augmentation
Dropout is typically interpreted as bagging a large number of models sharing parameters. We show that using dropout in a network can also be interpreted as a kind of data augmentation in the input space without domain knowledge. We present an approach to projecting the dropout noise within a network back into the input…
SBA improves neural network generalization by dynamically augmenting data.
problem Lack of direct supervision in data augmentation for generalization.
method Stochastic Batch Augmentation with dynamic soft label regularization.
result SBA improves generalization and speeds up training.
A key challenge in leveraging data augmentation for neural network training is choosing an effective augmentation policy from a large search space of candidate operations. Properly chosen augmentation policies can lead to significant generalization improvements; however, state-of-the-art approaches such as AutoAugment …
Exemplar VAEs link generative models with nearest neighbor retrieval and data augmentation.
problem Improving generative model performance and data augmentation effectiveness.
method Exemplar VAEs with Parzen window prior, retrieval augmented training, exemplar leave-one-out, and subsampling.
result Generative data augmentation reduces classification error on MNIST and Fashion MNIST.
Enhances generative models by improving expressivity without high computational cost.
problem Improving expressivity in generative models without increasing computational complexity.
method Proposes a new family of generative flows on an augmented data space, proving they can approximate a Hamiltonian ODE as a universal transport map.
result Demonstrates state-of-the-art performance on flow-based generative modeling benchmarks.
Learn invariances in neural networks by optimizing over augmentation parameters.
problem Lack of knowledge about present invariances and their extent in data.
method Parameterize a distribution over augmentations and optimize network parameters and augmentation parameters simultaneously.
result Recover correct set and extent of invariances on various tasks from training data alone.
New framework explains data augmentation's role in machine learning.
problem Understanding how data augmentation affects generalization and invariance learning.
method Information-theoretic framework based on mutual information bounds and orbit-averaged loss functions.
result Derives a new generalization bound decomposing the generalization gap into three interpretable terms.
Generative model for creating graphs with new communities.
problem Generating graphs with a new community structure.
method Fit Gaussian mixture model to latent space data and add new clusters based on MDL principle.
result Empirically demonstrated effectiveness of GCA for generating graphs with new community structures.
T-JEPA learns tabular data representations without augmentations, outperforming traditional methods.
problem Challenges in self-supervised learning for tabular data due to lack of data augmentations.
method T-JEPA uses a Joint Embedding Predictive Architecture (JEPA) to predict latent representations of different subsets of features within the same sample.
result Significant improvement in classification and regression tasks, outperforming traditional methods.
Study linear transformations' effects on data augmentation for improved estimation.
problem Improving performance in image and text classification tasks.
method Examined a family of linear transformations in over-parametrized linear regression settings.
result Transformations that preserve labels or mix data can improve estimation.
In this paper, we propose a novel implicit semantic data augmentation (ISDA) approach to complement traditional augmentation techniques like flipping, translation or rotation. Our work is motivated by the intriguing property that deep networks are surprisingly good at linearizing features, such that certain directions …
The study analyzes how data augmentation helps isolate content from style in self-supervised learning.
problem Understanding how data augmentation affects the separation of content and style in self-supervised learning.
method Formulated a latent variable model with content and style components, studied identifiability of latent representation, and introduced a dataset to test the theory.
result Sufficient conditions for identifying the invariant content partition in self-supervised learning.
New method disentangles style features from data augmentations.
problem Difficulty in deducing which data attributes are 'style' and should be discarded.
method Structured data augmentation with multiple style embedding spaces, maximizing joint entropy.
result Empirically demonstrates benefits on synthetic and real-world data.
Proposes a new framework for learning image augmentations to improve classification performance.
problem Improving classification performance with a given class of predictors.
method Transformed Risk Minimization (TRM) framework that optimizes both predictive models and data transformations.
result Performance of TRM with SCALE algorithm compares favorably to prior methods on CIFAR10/100.
Improved neural PDEs trained on augmented data enhance model accuracy and efficiency.
problem Training neural PDEs on limited data to accurately represent complex systems.
method Space-filling sampling of local states to generate augmented training data.
result Data-augmented neural PDEs outperform traditional emulators in accuracy and stability.
WeMix improves data augmentation by correcting bias in deep learning.
problem Data augmentation's effectiveness is limited by data bias.
method Developed AugDrop and MixLoss algorithms to correct data bias.
result WeMix improves data augmentation performance through bias correction.
In this paper we show how to augment classical methods for inverse problems with artificial neural networks. The neural network acts as a prior for the coefficient to be estimated from noisy data. Neural networks are global, smooth function approximators and as such they do not require explicit regularization of the er…
GeoECG augments ECG data to improve heart disease detection.
problem Insufficient labeled ECG data and vulnerability to adversarial attacks.
method Wasserstein geodesic perturbation for data augmentation.
result Improved accuracy and robustness in ECG-based heart disease detection.
This paper tackles overfitting in CTR models by introducing Multi-Epoch learning with Data Augmentation.
problem Overfitting of the embedding layer in CTR models during multi-epoch training.
method Introduces Multi-Epoch learning with Data Augmentation (MEDA) framework to reduce overfitting and enhance performance.
result MEDA minimizes overfitting and achieves data augmentation through varied embedding spaces, improving performance without overfitting.
Data augmentation has been widely applied as an effective methodology to improve generalization in particular when training deep neural networks. Recently, researchers proposed a few intensive data augmentation techniques, which indeed improved accuracy, yet we notice that these methods augment data have also caused a …
i-Mix improves contrastive learning across domains without domain-specific augmentations.
problem Improving contrastive representation learning for unlabeled data across diverse domains.
method i-Mix treats contrastive learning as a non-parametric classifier problem, mixing data in input and virtual label spaces.
result i-Mix consistently improves representation quality across image, speech, and tabular data domains.
A study on optimizing data augmentation weights for improved test-time predictions.
problem Improving robustness of predictions during testing with data augmentation methods.
method A weighted Test-Time Augmentation (TTA) approach based on variational Bayesian framework to optimize weights.
result Optimizing weights suppresses unwanted data augmentations and improves prediction performance.
New measures quantify how data augmentation improves model performance.
problem Understanding the effectiveness of data augmentation in deep learning.
method Introduced Affinity and Diversity measures to quantify augmentation performance.
result Augmentation performance is best achieved by optimizing both Affinity and Diversity.
CNNs encode data augmentation transformations, especially in early layers.
problem Whether neural network features encode data augmentation transformations.
method Systematic approach using pre-trained vision models to predict augmentation transformations.
result Neural network features encode data augmentation transformations, especially in early layers.
Automatically learns optimal data augmentation for image classification.
problem Finding optimal data augmentation hyperparameters is computationally demanding and requires domain knowledge.
method Proposes an online bilevel optimization framework to learn data augmentation parameters directly.
result Jointly trained method achieves comparable or better classification accuracy than hand-crafted data augmentation without an external validation loop.
This thesis explores fast algorithms for large matrices and data augmentation to improve model efficiency.
problem Efficient handling of large models and data in scientific computing and machine learning.
method Randomized low-rank decomposition algorithms and data augmentation techniques.
result Improved sample efficiency and generalization of machine learning models.
Dataset augmentation, the practice of applying a wide array of domain-specific transformations to synthetically expand a training set, is a standard tool in supervised learning. While effective in tasks such as visual recognition, the set of transformations must be carefully designed, implemented, and tested for every …
Improved generalization with semantic perturbations using normalizing flows.
problem Overfitting in deep neural networks training.
method Use normalizing flows for generating semantically meaningful perturbations in latent space.
result Achieved 96.6% test accuracy on CIFAR-10 with ResNet-18, outperforming existing methods.
Improves tabular data augmentation for contrastive learning.
problem Ineffective augmentation techniques for tabular data.
method Class-conditioned and feature-correlation based augmentation.
result Consistently outperforms conventional corruption methods.
Data augmentation (DA) has been widely utilized to improve generalization in training deep neural networks. Recently, human-designed data augmentation has been gradually replaced by automatically learned augmentation policy. Through finding the best policy in well-designed search space of data augmentation, AutoAugment…
We propose regularization strategies for learning discriminative models that are robust to in-class variations of the input data. We use the Wasserstein-2 geometry to capture semantically meaningful neighborhoods in the space of images, and define a corresponding input-dependent additive noise data augmentation model. …
New attacks show data augmentation may not improve privacy.
problem Measuring privacy risk in models trained with data augmentation.
method Formulated membership inference as a set classification problem, designed input permutation invariant features.
result Proposed approach universally outperforms original methods on models trained with data augmentation.
Data augmentation doesn't improve robustness, contrary to belief.
problem The effectiveness of data augmentation in improving model robustness is questioned.
method Taking a Domain Generalization viewpoint, the study examines the robustness of augmented representations.
result Augmented representations are not robust to distortions used during training.
Graph data augmentation improves GNN performance in node classification.
problem Improving generalizability of graph neural networks (GNNs) in semi-supervised node classification.
method Introduces GAug framework for graph data augmentation using neural edge predictors.
result GAug framework improves GNN-based node classification performance across various architectures and datasets.
Study examines how data augmentation impacts optimization in linear regression.
problem Understanding how data augmentation schedules affect optimization in linear regression.
method Analyzed the effect of augmentation on optimization in linear regression with MSE loss, using classical convex optimization and recent work on implicit bias.
result Proved that under certain joint schedules for learning rate and augmentation scheme, augmented gradient descent converges and characterized the resulting minimum.
ADA augments data using AR replicas for robust regression.
problem Improving robustness in nonlinear over-parametrized regression.
method Extends Anchor regression (AR) for data augmentation, using replicas of modified samples.
result ADA provides more robust regression predictions compared to state-of-the-art solutions.
Two new Hie-TAN and Hie-TAN-Lite algorithms improve TAN for hierarchical feature spaces.
problem Learning dependencies in hierarchical feature spaces.
method Exploits hierarchical parent-child relationships as constraints to learn a dependency tree.
result Hie-TAN-Lite outperforms Hie-TAN and other methods in predictive accuracy.
Develops a statistical framework for self-supervised representation learning using data augmentation.
problem Lack of theoretical understanding of data augmentation in nonlinear settings.
method Augmentation invariant manifold learning framework and stochastic optimization algorithm.
result Improves downstream analysis by exploiting manifold's geometric structure and invariant property of augmented data.
Data augmentation impacts adversarial risk; careful application recommended.
problem Understanding how data augmentation affects adversarial risk in deep learning.
method Empirical analysis using three measures of adversarial risk.
result Data augmentation does not always improve adversarial risk; augmented data influences models more.
A major problem in data augmentation is to ensure that the generated new samples cover the search space. This is a challenging problem and requires exploration for data augmentation policies to ensure their effectiveness in covering the search space. In this paper, we propose Greedy AutoAugment as a highly efficient se…
Data augmentation affects feature importance, enhancing learning for neural networks.
problem Understanding the effect of data augmentation on feature importance and learning dynamics.
method Analyzing a two-layer convolutional neural network in a multi-view data model.
result Data augmentation alters feature importance, making certain features more likely to be learned.
Simple policy search outperforms advanced learnable test-time augmentation techniques.
problem Improving predictive performance through test-time data augmentation.
method Greedy policy search (GPS) for learning test-time augmentation policies.
result Augmentation policies learned with GPS achieve superior predictive performance and robustness.
Data augmentation improves microbiome disease prediction.
problem Improving predictive models for microbiome data.
method Defined novel data augmentation strategies for simplex-valued data.
result Set new state-of-the-art for disease prediction tasks.
A method to prevent image representation collapse through data-dependent augmentation.
problem Representation collapse due to image augmentations that damage information.
method Formalizing a stochastic encoding process with a tug-of-war between corruption and preserved information, using infoMax objective.
result Learning a data-dependent distribution of augmentations to avoid representation collapse.
Counterfactual data augmentations may not ensure OOD robustness if performed by a context-guessing machine.
problem Deep learning models lack out-of-distribution robustness due to reliance on spurious features.
method Theoretical analysis and demonstration of counterfactual data augmentations performed by a context-guessing machine.
result Counterfactual data augmentations by a context-guessing machine do not lead to robust OOD classifiers.