CNNs encode data augmentation transformations, especially in early layers.
problem Whether neural network features encode data augmentation transformations.
method Systematic approach using pre-trained vision models to predict augmentation transformations.
result Neural network features encode data augmentation transformations, especially in early layers.
Survey of data augmentation techniques for time series classification with neural networks.
problem Small datasets in time series recognition.
method Four families of data augmentation: transformation-based, pattern mixing, generative models, and decomposition methods.
result Empirical evaluation of 12 data augmentation methods on 128 datasets.
Bayesian neural networks with data augmentation show a persistent cold posterior effect.
problem Understanding the cold posterior effect in Bayesian neural networks with data augmentation.
method Developed principled Bayesian neural networks using data augmentation, providing exact likelihoods and tight bounds.
result The cold posterior effect persists even in models incorporating data augmentation, suggesting it's not an artifact.
Bayesian neural networks simplified with input augmentation.
problem Uncertainty in deep learning models.
method Layer-wise input augmentation to induce uncertainty distributions.
result State-of-the-art performance in uncertainty representation.
This work characterizes how data augmentation shapes neural representations.
problem Understanding the impact of data augmentation on neural network representations.
method Embedding neural network hidden representations into a metric space invariant to transformations, analyzing shape-space trajectories.
result Increasing data augmentation strength leads to well-behaved trajectories in the embedded space, and different augmentation types steer representations in distinct directions.
Memory-augmented neural networks (MANNs) have been shown to outperform other recurrent neural network architectures on a series of artificial sequence learning tasks, yet they have had limited application to real-world tasks. We evaluate direct application of Neural Turing Machines (NTM) and Differentiable Neural Compu…
Learn invariances in neural networks by optimizing over augmentation parameters.
problem Lack of knowledge about present invariances and their extent in data.
method Parameterize a distribution over augmentations and optimize network parameters and augmentation parameters simultaneously.
result Recover correct set and extent of invariances on various tasks from training data alone.
Bayesian network learns data invariances without augmentation.
problem Learning invariances in neural networks without manual design.
method Bayesian approach infers weight-sharing schemes from data.
result Model outperforms non-invariant networks on specific tasks.
In this paper we show how to augment classical methods for inverse problems with artificial neural networks. The neural network acts as a prior for the coefficient to be estimated from noisy data. Neural networks are global, smooth function approximators and as such they do not require explicit regularization of the er…
Data augmentation affects feature importance, enhancing learning for neural networks.
problem Understanding the effect of data augmentation on feature importance and learning dynamics.
method Analyzing a two-layer convolutional neural network in a multi-view data model.
result Data augmentation alters feature importance, making certain features more likely to be learned.
Graph data augmentation improves GNN performance in node classification.
problem Improving generalizability of graph neural networks (GNNs) in semi-supervised node classification.
method Introduces GAug framework for graph data augmentation using neural edge predictors.
result GAug framework improves GNN-based node classification performance across various architectures and datasets.
A recurring problem faced when training neural networks is that there is typically not enough data to maximize the generalization capability of deep neural networks(DNN). There are many techniques to address this, including data augmentation, dropout, and transfer learning. In this paper, we introduce an additional met…
Graph Neural Networks struggle with generalization, especially OOD data; GRATIN solves this with Gaussian Mixture Model-based augmentation.
problem Graph Neural Networks struggle with generalization, particularly to unseen or out-of-distribution data.
method Theoretical framework using Rademacher complexity to compute a regret bound on generalization error. GRATIN algorithm leveraging Gaussian Mixture Models for efficient data augmentation.
result GRATIN outperforms existing augmentation techniques in terms of generalization and offers improved time complexity.
Guided warping augments time series data by aligning features with a teacher.
problem Small time series datasets limit neural network performance.
method Guided warping with a discriminative teacher to augment data deterministically.
result Significant improvement in performance on various time series datasets.
DRO-Augment framework enhances deep neural network robustness.
problem Robustness of deep neural networks against various perturbations and adversarial attacks.
method Integrates Wasserstein Distributionally Robust Optimization with data augmentation.
result Significantly improves robustness across various corruptions and adversarial attacks.
SODA optimizes data augmentation allocation for deep learning models.
problem Inefficient allocation of data augmentation budget in deep neural networks.
method Online learning to dynamically allocate data augmentation budget during training.
result Optimized data augmentation can save computation time and promote greener machine learning.
Today, the dominant paradigm for training neural networks involves minimizing task loss on a large dataset. Using world knowledge to inform a model, and yet retain the ability to perform end-to-end training remains an open question. In this paper, we present a novel framework for introducing declarative knowledge to ne…
Study adapts AI research methods to analyze image augmentation impacts on neural network operations.
problem Understanding how image augmentation affects neural network performance and sensitivity.
method Adapted treatment-control paradigm, uses variance decomposition, Sobol indices, and Shapley values for sensitivity analysis.
result Visualizes and quantifies sensitivity to different image augmentation parameters.
Method learns invariances in deep nets without human validation.
problem Manual selection of data augmentation parameters is cumbersome.
method Differentiable Laplace approximation for Bayesian model selection.
result Method successfully recovers invariances and improves generalization.
Stochastic approach improves neural network training for kinetic simulations.
problem Training neural networks under physical constraints in kinetic fusion simulations.
method Stochastic augmented Lagrangian approach using pyTorch.
result Higher model prediction accuracy achieved compared to fixed penalty method.
State-augmented algorithm optimizes wireless network resource management.
problem Optimizing resource allocation in multi-user wireless networks.
method Proposes a state-augmented algorithm using dual variables.
result Feasible and near-optimal resource decisions achieved.
DAERNN models censored data using neural networks with data augmentation.
problem Handling censored data in expectile regression.
method Data augmentation based Expectile Regression Neural Networks (ERNNs).
result DAERNN outperforms existing censored ERNNs methods and achieves comparable predictive performance to fully observed data.
Online data augmentation improves forecasting performance in deep learning.
problem Insufficient training data for forecasting tasks.
method An online data augmentation framework that generates synthetic samples during training.
result Online data augmentation leads to better forecasting performance.
This work explains how tempering improves Bayesian neural networks by reducing the impact of data augmentation.
problem Improper sharpening of Bayesian neural networks leads to suboptimal performance.
method Theoretical analysis and empirical evaluations of simplified settings and group convolutions.
result Tempering reduces the misspecification due to modeling augmentations as independent and identically distributed (i.i.d.) data.
A multi-objective prediction method of multi-stage pump method based on neural network with data augmentation is proposed. In order to study the highly nonlinear relationship between key design variables and centrifugal pump external characteristic values (head and power), the neural network model (NN) is built in comp…
Optimizes wireless network resource management with state-augmented policies.
problem Optimizing network-wide utility with user performance constraints.
method State-augmented parameterization of RRM policy, using dual variables.
result Superior trade-off between mean, minimum, and 5th percentile rates.
New method combines neural nets with epidemic models for better prediction.
problem Improving epidemic prediction and forecasting using deep neural networks.
method Integrates machine learning with compartmental disease models for data-driven analysis.
result Data augmentation strategy improves neural network reliability for epidemic forecasting.
Augments graph node features to improve GNN performance.
problem Improving graph neural networks' performance on large-scale datasets.
method Iteratively augments node features with gradient-based adversarial perturbations.
result Boosts model performance in node classification, link prediction, and graph classification tasks.
Recently, data augmentation in the semi-supervised regime, where unlabeled data vastly outnumbers labeled data, has received a considerable attention. In this paper, we describe an efficient technique for this task, exploiting a recent framework we proposed for missing data imputation called graph imputation neural net…
Deep multi-task learning attracts much attention in recent years as it achieves good performance in many applications. Feature learning is important to deep multi-task learning for sharing common information among tasks. In this paper, we propose a Hierarchical Graph Neural Network (HGNN) to learn augmented features fo…
A new framework for systematic graph neural network data augmentation.
problem Diversity and difficulty in choosing graph neural network data augmentation techniques.
method Comprehensive framework capturing all previous RDAs, formal universality proof, automatic training method.
result Improved state of the art through new RDAs and impartial comparison.
Study evaluates data augmentation methods for prostate cancer detection in MRI.
problem Limited data for prostate cancer detection in MRI.
method Static application of five augmentation techniques (rotation, flip, crop, translation) to MRI dataset.
result Rotation method improved 2D slice-based AUC to 0.85.
Large-batch SGD is important for scaling training of deep neural networks. However, without fine-tuning hyperparameter schedules, the generalization of the model may be hampered. We propose to use batch augmentation: replicating instances of samples within the same batch with different data augmentations. Batch augment…
Enhances neural networks with prior function values to improve accuracy.
problem Improving neural network accuracy in regions without training data.
method Develops a probabilistic approach to augment BNNs with prior function values.
result Predictions rely more on prior information in uncertain regions.
SBA improves neural network generalization by dynamically augmenting data.
problem Lack of direct supervision in data augmentation for generalization.
method Stochastic Batch Augmentation with dynamic soft label regularization.
result SBA improves generalization and speeds up training.
Data augmentation improves financial prediction models, especially for small datasets.
problem Improving financial prediction models on small, noisy, non-stationary datasets.
method Evaluation of data augmentation methods combined with deep learning models on financial datasets.
result Data augmentation significantly improves financial performance, up to 400% improvement in risk-adjusted return.
Enhances graph neural networks by creating virtual data examples.
problem Lack of examples to identify optimal graph rationales in graph applications.
method Introduces environment replacement to create virtual data examples and proposes a framework for rationale-environment separation and representation learning.
result Demonstrates the effectiveness and efficiency of the augmentation-based graph rationalization framework on molecular and polymer datasets.
Enhances neural network robustness with Mixup and TLAT.
problem Neural networks are sensitive to various perturbations and adversarial examples.
method Combines Mixup augmentation with Targeted Labeling Adversarial Training (TLAT).
result M-TLAT increases robustness against 19 corruptions and 5 adversarial attacks without reducing clean sample accuracy.
We augment recurrent neural networks with an external memory mechanism that builds upon recent progress in metalearning. We conceptualize this memory as a rapidly adaptable function that we parameterize as a deep neural network. Reading from the neural memory function amounts to pushing an input (the key vector) throug…
Deep artificial neural networks require a large corpus of training data in order to effectively learn, where collection of such training data is often expensive and laborious. Data augmentation overcomes this issue by artificially inflating the training set with label preserving transformations. Recently there has been…
Data augmentation is a widely used trick when training deep neural networks: in addition to the original data, properly transformed data are also added to the training set. However, to the best of our knowledge, a clear mathematical framework to explain the performance benefits of data augmentation is not available. In…
Graph Random Neural Network improves semi-supervised learning on graphs.
problem Over-smoothing, non-robustness, and weak-generalization in GNNs with few labeled nodes.
method Random propagation strategy and consistency regularization.
result Significantly outperforms state-of-the-art GNN baselines on semi-supervised node classification.
CutMix enhances feature learning in neural networks, improving test accuracy.
problem Understanding and improving feature learning in neural networks using patch-level augmentation.
method Three distinct methods: vanilla training, Cutout training, and CutMix training were studied.
result CutMix training yields the highest test accuracy and learns all features and noise vectors evenly.
Lifelong learning aims to develop machine learning systems that can learn new tasks while preserving the performance on previous learned tasks. In this paper we present a method to overcome catastrophic forgetting on convolutional neural networks, that learns new tasks and preserves the performance on old tasks without…
The recent adoption of recurrent neural networks (RNNs) for session modeling has yielded substantial performance gains compared to previous approaches. In terms of context-aware session modeling, however, the existing RNN-based models are limited in that they are not designed to explicitly model rich static user-side c…
Mixes higher-order simplicial complexes for data augmentation.
problem Lack of labeled data for complex systems with multiway interactions.
method Proposes mixup mechanisms for simplicial complexes, including linear and nonlinear mixup, and a convex clustering mixup.
result Synthetic simplicial complexes interpolate between existing data based on homomorphism densities.
New model improves neural network robustness against input manipulations.
problem Improving neural network robustness against input manipulations.
method Causal view and deep causal manipulation augmented model (deep CAMA) with data augmentation and test-time fine-tuning.
result Deep CAMA shows superior robustness against unseen manipulations compared to traditional models.
Recent empirical results on long-term dependency tasks have shown that neural networks augmented with an external memory can learn the long-term dependency tasks more easily and achieve better generalization than vanilla recurrent neural networks (RNN). We suggest that memory augmented neural networks can reduce the ef…