Method learns invariances in deep nets without human validation.
problem Manual selection of data augmentation parameters is cumbersome.
method Differentiable Laplace approximation for Bayesian model selection.
result Method successfully recovers invariances and improves generalization.
Proposes FATTNN for tensor-on-tensor regression with improved prediction and reduced computation.
problem Tensor-on-tensor regression with complex tensor structures and nonlinear relationships.
method Integrates tensor factor models into deep neural networks to handle nonlinearity and reduce data dimensionality.
result Significant improvements in prediction accuracy and computational efficiency over traditional methods.
Stochastic approach improves neural network training for kinetic simulations.
problem Training neural networks under physical constraints in kinetic fusion simulations.
method Stochastic augmented Lagrangian approach using pyTorch.
result Higher model prediction accuracy achieved compared to fixed penalty method.
Recently, data augmentation in the semi-supervised regime, where unlabeled data vastly outnumbers labeled data, has received a considerable attention. In this paper, we describe an efficient technique for this task, exploiting a recent framework we proposed for missing data imputation called graph imputation neural net…
SMART-FAN-Lasso fine-tunes neural networks for high-dimensional nonparametric regression.
problem Fine-tuning neural networks for high-dimensional nonparametric regression with variable selection.
method Source-model-augmented residual tuning (SMART) framework for neural Lasso.
result SMART-FAN-Lasso achieves statistical acceleration over single-task learning under precise conditions.
Online data augmentation improves forecasting performance in deep learning.
problem Insufficient training data for forecasting tasks.
method An online data augmentation framework that generates synthetic samples during training.
result Online data augmentation leads to better forecasting performance.
This work explains how tempering improves Bayesian neural networks by reducing the impact of data augmentation.
problem Improper sharpening of Bayesian neural networks leads to suboptimal performance.
method Theoretical analysis and empirical evaluations of simplified settings and group convolutions.
result Tempering reduces the misspecification due to modeling augmentations as independent and identically distributed (i.i.d.) data.
CNNs encode data augmentation transformations, especially in early layers.
problem Whether neural network features encode data augmentation transformations.
method Systematic approach using pre-trained vision models to predict augmentation transformations.
result Neural network features encode data augmentation transformations, especially in early layers.
Memory-augmented neural networks (MANNs) are designed for question-answering tasks. It is difficult to run a MANN effectively on accelerators designed for other neural networks (NNs), in particular on mobile devices, because MANNs require recurrent data paths and various types of operations related to external memory a…
In this paper, we use variational recurrent neural network to investigate the anomaly detection problem on graph time series. The temporal correlation is modeled by the combination of recurrent neural network (RNN) and variational inference (VI), while the spatial information is captured by the graph convolutional netw…
Augments GNNs with diversification to preserve node identity.
problem Current GNNs filter node information, potentially losing node identity.
method Integrates diversification operators with aggregation to enrich node representations.
result Significant performance boost on 9 node classification tasks.
Survey of data augmentation techniques for time series classification with neural networks.
problem Small datasets in time series recognition.
method Four families of data augmentation: transformation-based, pattern mixing, generative models, and decomposition methods.
result Empirical evaluation of 12 data augmentation methods on 128 datasets.
Bayesian neural networks with data augmentation show a persistent cold posterior effect.
problem Understanding the cold posterior effect in Bayesian neural networks with data augmentation.
method Developed principled Bayesian neural networks using data augmentation, providing exact likelihoods and tight bounds.
result The cold posterior effect persists even in models incorporating data augmentation, suggesting it's not an artifact.
Bayesian neural networks simplified with input augmentation.
problem Uncertainty in deep learning models.
method Layer-wise input augmentation to induce uncertainty distributions.
result State-of-the-art performance in uncertainty representation.
This work characterizes how data augmentation shapes neural representations.
problem Understanding the impact of data augmentation on neural network representations.
method Embedding neural network hidden representations into a metric space invariant to transformations, analyzing shape-space trajectories.
result Increasing data augmentation strength leads to well-behaved trajectories in the embedded space, and different augmentation types steer representations in distinct directions.
Memory-augmented neural networks (MANNs) have been shown to outperform other recurrent neural network architectures on a series of artificial sequence learning tasks, yet they have had limited application to real-world tasks. We evaluate direct application of Neural Turing Machines (NTM) and Differentiable Neural Compu…
Learn invariances in neural networks by optimizing over augmentation parameters.
problem Lack of knowledge about present invariances and their extent in data.
method Parameterize a distribution over augmentations and optimize network parameters and augmentation parameters simultaneously.
result Recover correct set and extent of invariances on various tasks from training data alone.
Bayesian network learns data invariances without augmentation.
problem Learning invariances in neural networks without manual design.
method Bayesian approach infers weight-sharing schemes from data.
result Model outperforms non-invariant networks on specific tasks.
Study shows low-complexity models can perform as well as state-of-the-art on small datasets.
problem Performance of deep learning models on small datasets.
method Wide variety of experiments with different deep learning architectures on small datasets.
result Low-complexity models can perform comparably well or better than state-of-the-art models on small datasets.
Topic-aware chatbot learns from NMF topic vectors.
problem Improving chatbot relevance based on user topics.
method Combines RNN with NMF for topic learning and attention.
result Chatbot provides more relevant answers based on topic.
In this paper we show how to augment classical methods for inverse problems with artificial neural networks. The neural network acts as a prior for the coefficient to be estimated from noisy data. Neural networks are global, smooth function approximators and as such they do not require explicit regularization of the er…
Data augmentation affects feature importance, enhancing learning for neural networks.
problem Understanding the effect of data augmentation on feature importance and learning dynamics.
method Analyzing a two-layer convolutional neural network in a multi-view data model.
result Data augmentation alters feature importance, making certain features more likely to be learned.
Graph data augmentation improves GNN performance in node classification.
problem Improving generalizability of graph neural networks (GNNs) in semi-supervised node classification.
method Introduces GAug framework for graph data augmentation using neural edge predictors.
result GAug framework improves GNN-based node classification performance across various architectures and datasets.
Neural network with data augmentation improves multi-stage pump prediction accuracy.
problem Predicting multi-stage pump external characteristics with high accuracy.
method Neural network model with data augmentation for multi-objective prediction.
result Neural network model with data augmentation outperforms other models in accuracy.
A recurring problem faced when training neural networks is that there is typically not enough data to maximize the generalization capability of deep neural networks(DNN). There are many techniques to address this, including data augmentation, dropout, and transfer learning. In this paper, we introduce an additional met…
Paper proposes SDDP for improving time series forecasting with high-dimensional predictors.
problem Improving time series forecasting with high-dimensional predictors.
method SDDP framework that incorporates target variable and lagged observations into factor extraction process.
result SDDP improves predictive accuracy in time series forecasting.
Graph Neural Networks struggle with generalization, especially OOD data; GRATIN solves this with Gaussian Mixture Model-based augmentation.
problem Graph Neural Networks struggle with generalization, particularly to unseen or out-of-distribution data.
method Theoretical framework using Rademacher complexity to compute a regret bound on generalization error. GRATIN algorithm leveraging Gaussian Mixture Models for efficient data augmentation.
result GRATIN outperforms existing augmentation techniques in terms of generalization and offers improved time complexity.
Guided warping augments time series data by aligning features with a teacher.
problem Small time series datasets limit neural network performance.
method Guided warping with a discriminative teacher to augment data deterministically.
result Significant improvement in performance on various time series datasets.
DRO-Augment framework enhances deep neural network robustness.
problem Robustness of deep neural networks against various perturbations and adversarial attacks.
method Integrates Wasserstein Distributionally Robust Optimization with data augmentation.
result Significantly improves robustness across various corruptions and adversarial attacks.
HGNN learns augmented features for deep multi-task learning.
problem Feature learning for deep multi-task learning.
method Hierarchical Graph Neural Network (HGNN) with two levels of graph neural networks.
result Significant performance improvement in classification tasks.
SODA optimizes data augmentation allocation for deep learning models.
problem Inefficient allocation of data augmentation budget in deep neural networks.
method Online learning to dynamically allocate data augmentation budget during training.
result Optimized data augmentation can save computation time and promote greener machine learning.
Tackling pattern recognition problems in areas such as computer vision, bioinformatics, speech or text recognition is often done best by taking into account task-specific statistical relations between output variables. In structured prediction, this internal structure is used to predict multiple outputs simultaneously,…
FIDDLE uses deep learning to estimate ATE from complex data.
problem Estimating ATE from high-dimensional, correlated covariates with sparse nonlinear effects.
method Factor-augmented deep learning for propensity and outcome models.
result FIDDLE consistently estimates ATE under model misspecification and is semiparametrically efficient.
Today, the dominant paradigm for training neural networks involves minimizing task loss on a large dataset. Using world knowledge to inform a model, and yet retain the ability to perform end-to-end training remains an open question. In this paper, we present a novel framework for introducing declarative knowledge to ne…
Study adapts AI research methods to analyze image augmentation impacts on neural network operations.
problem Understanding how image augmentation affects neural network performance and sensitivity.
method Adapted treatment-control paradigm, uses variance decomposition, Sobol indices, and Shapley values for sensitivity analysis.
result Visualizes and quantifies sensitivity to different image augmentation parameters.
State-augmented algorithm optimizes wireless network resource management.
problem Optimizing resource allocation in multi-user wireless networks.
method Proposes a state-augmented algorithm using dual variables.
result Feasible and near-optimal resource decisions achieved.
DAERNN models censored data using neural networks with data augmentation.
problem Handling censored data in expectile regression.
method Data augmentation based Expectile Regression Neural Networks (ERNNs).
result DAERNN outperforms existing censored ERNNs methods and achieves comparable predictive performance to fully observed data.
Optimizes wireless network resource management with state-augmented policies.
problem Optimizing network-wide utility with user performance constraints.
method State-augmented parameterization of RRM policy, using dual variables.
result Superior trade-off between mean, minimum, and 5th percentile rates.
New method combines neural nets with epidemic models for better prediction.
problem Improving epidemic prediction and forecasting using deep neural networks.
method Integrates machine learning with compartmental disease models for data-driven analysis.
result Data augmentation strategy improves neural network reliability for epidemic forecasting.
In practice it is often found that large over-parameterized neural networks generalize better than their smaller counterparts, an observation that appears to conflict with classical notions of function complexity, which typically favor smaller models. In this work, we investigate this tension between complexity and gen…
Augments graph node features to improve GNN performance.
problem Improving graph neural networks' performance on large-scale datasets.
method Iteratively augments node features with gradient-based adversarial perturbations.
result Boosts model performance in node classification, link prediction, and graph classification tasks.
Data representations that contain all the information about target variables but are invariant to nuisance factors benefit supervised learning algorithms by preventing them from learning associations between these factors and the targets, thus reducing overfitting. We present a novel unsupervised invariance induction f…
A new framework for systematic graph neural network data augmentation.
problem Diversity and difficulty in choosing graph neural network data augmentation techniques.
method Comprehensive framework capturing all previous RDAs, formal universality proof, automatic training method.
result Improved state of the art through new RDAs and impartial comparison.
Study evaluates data augmentation methods for prostate cancer detection in MRI.
problem Limited data for prostate cancer detection in MRI.
method Static application of five augmentation techniques (rotation, flip, crop, translation) to MRI dataset.
result Rotation method improved 2D slice-based AUC to 0.85.
Large-batch SGD is important for scaling training of deep neural networks. However, without fine-tuning hyperparameter schedules, the generalization of the model may be hampered. We propose to use batch augmentation: replicating instances of samples within the same batch with different data augmentations. Batch augment…
Enhances neural networks with prior function values to improve accuracy.
problem Improving neural network accuracy in regions without training data.
method Develops a probabilistic approach to augment BNNs with prior function values.
result Predictions rely more on prior information in uncertain regions.
SBA improves neural network generalization by dynamically augmenting data.
problem Lack of direct supervision in data augmentation for generalization.
method Stochastic Batch Augmentation with dynamic soft label regularization.
result SBA improves generalization and speeds up training.
Data augmentation improves financial prediction models, especially for small datasets.
problem Improving financial prediction models on small, noisy, non-stationary datasets.
method Evaluation of data augmentation methods combined with deep learning models on financial datasets.
result Data augmentation significantly improves financial performance, up to 400% improvement in risk-adjusted return.