Study ablated data augmentation techniques and their mathematical equivalence to penalties.
problem Lack of mathematical understanding of differences between ablated data augmentation techniques.
method Formal model of mean ablated data augmentation and inverted dropout for linear regression; empirical validation for deep networks.
result Ablated data augmentation and inverted dropout are mathematically equivalent to penalties in optimization.
Augments graph node features to improve GNN performance.
problem Improving graph neural networks' performance on large-scale datasets.
method Iteratively augments node features with gradient-based adversarial perturbations.
result Boosts model performance in node classification, link prediction, and graph classification tasks.
i-Mix improves contrastive learning across domains without domain-specific augmentations.
problem Improving contrastive representation learning for unlabeled data across diverse domains.
method i-Mix treats contrastive learning as a non-parametric classifier problem, mixing data in input and virtual label spaces.
result i-Mix consistently improves representation quality across image, speech, and tabular data domains.
Aims to create a world model without baggage, achieving good performance.
problem Creating world models without baggage and limitations.
method Self-supervised representation learning, frame and action stacking, data augmentation.
result Good performance on the Atari 100k benchmark.
Semi-supervised learning (SSL) provides an effective means of leveraging unlabeled data to improve a model's performance. In this paper, we demonstrate the power of a simple combination of two common SSL methods: consistency regularization and pseudo-labeling. Our algorithm, FixMatch, first generates pseudo-labels usin…
MixBoost generates synthetic instances to balance imbalanced datasets.
problem Training models on imbalanced datasets.
method Iterative data augmentation method that selects and combines instances from majority and minority classes.
result MixBoost outperforms existing approaches on 20 benchmark datasets.
New unsupervised learning task improves RL performance.
problem Reward-driven feature learning limitations in RL from images.
method Introduce Augmented Temporal Contrast (ATC) for unsupervised learning of image representations.
result Training encoders using ATC matches or outperforms end-to-end RL in most environments.
iGCL preserves graph semantics in latent space augmentations.
problem Manual tuning of augmentation ratios and unexpected graph changes.
method iGCL uses a Variational Graph Auto-Encoder to learn augmentations in the latent space, optimizing an upper bound for contrastive loss.
result iGCL achieves state-of-the-art performance on graph-level and node-level tasks.
A conjugate Bayesian method detects change points in Hawkes processes efficiently.
problem Non-conjugacy between Hawkes process likelihood and prior causes inefficiency in change point detection.
method Data augmentation to propose a conjugate Bayesian two-step change point detection method.
result The conjugate method is more accurate and efficient than non-conjugate methods.
Curriculum learning and imitation learning improve control over financial time-series data.
problem Improving control performance over complex financial time-series data.
method Data augmentation for curriculum learning and policy distillation for imitation learning.
result Curriculum learning shows significant improvement over time-series control tasks.
In microsurgery, lasers have emerged as precise tools for bone ablation. A challenge is automatic control of laser bone ablation with 4D optical coherence tomography (OCT). OCT as high resolution imaging modality provides volumetric images of tissue and foresees information of bone position and orientation (pose) as we…
The paper analyzes how data augmentation affects the test error in regression models.
problem Understanding the impact of data augmentation on the test error in regression models.
method Characterizes the test error in terms of population quantities and augmentation statistics.
result Provides a tight characterization of the test error in mean squared error.
Enhances neural forecasting for hierarchically organized time series data.
problem Probabilistic coherent forecasting of time series data across different levels of aggregation.
method Proposes a coherent multivariate mixture output for neural forecasting architectures, optimizing with a composite likelihood objective.
result 13.2% average accuracy improvements on most datasets compared to state-of-the-art baselines.
Generates realistic stock market order streams using GANs.
problem Creating high-fidelity stock market data.
method Conditional Wasserstein GAN with auction mechanism and order-book augmentation.
result Generated data is close to real market data.
We review some of the latest approaches to analysing cardiac electrophysiology data using machine learning and predictive modelling. Cardiac arrhythmias, particularly atrial fibrillation, are a major global healthcare challenge. Treatment is often through catheter ablation, which involves the targeted localized destruc…
ABC method analyzes diffusion model dependence without retraining.
problem Difficulties in characterizing diffusion models' dependence on training data.
method Ablation-based counterfactuals using model components trained on different splits.
result Demonstrates limits of training data attribution and existence of unattributable samples.
This paper tackles confounding biases in data augmentation.
problem Mitigating spurious correlations and confounding variables in training data.
method Formal analysis and counterfactual data augmentation.
result Removing confounding biases leads to invariant features and better generalization.
A key challenge in leveraging data augmentation for neural network training is choosing an effective augmentation policy from a large search space of candidate operations. Properly chosen augmentation policies can lead to significant generalization improvements; however, state-of-the-art approaches such as AutoAugment …
Semi-supervised learning has proven to be a powerful paradigm for leveraging unlabeled data to mitigate the reliance on large labeled datasets. In this work, we unify the current dominant approaches for semi-supervised learning to produce a new algorithm, MixMatch, that works by guessing low-entropy labels for data-aug…
Identifying a set of homogeneous clusters in a heterogeneous dataset is one of the most important classes of problems in statistical modeling. In the realm of unsupervised partitional clustering, k-means is a very important algorithm for this. In this technical report, we develop a new k-means variant called Augmented …
New approach improves human activity recognition with wearables.
problem Improving human activity recognition with wearables.
method Exploiting latent relationships between multi-channel sensor modalities, data-agnostic augmentation, and a classification loss criterion.
result Achieves new state-of-the-art performance on four diverse activity recognition benchmarks.
Gaussian Processes enhance financial forecasting by predicting mean-reverting time series with probability distributions.
problem Accurate long-term financial predictions with probability distributions.
method Functional and augmented data structures for Gaussian Processes.
result Gaussian Processes offer improved long-term predictions with probability distributions.
FastSGT improves accuracy in BERT-based DST for SGD datasets.
problem Dialog State Tracking in goal-oriented dialogue systems.
method BERT-based model with two carry-over procedures and multi-head attention.
result Significantly improved accuracy compared to baseline model.
Data augmented bootstrap unifies various confidence interval construction methods.
problem Constructing confidence intervals from data transformations.
method Data augmented bootstrap (DAB) framework.
result Establishes theoretical coverage results for DAB methods.
MixKD improves large-scale language model compression and generalization.
problem Inefficient and resource-intensive large-scale language models.
method MixKD uses mixup data augmentation to enhance student model's generalization ability.
result MixKD leads to significant performance gains over standard KD and competitive baselines.
Ablation studies show BCF model's propensity score is not essential for treatment effect estimation.
problem Understanding the necessity of propensity score in nonparametric treatment effect estimation.
method Partial ablation studies of Bayesian Causal Forest (BCF) model.
result Excluding estimated propensity score does not affect treatment effect estimation or uncertainty quantification.
LSTMs were introduced to combat vanishing gradients in simple RNNs by augmenting them with gated additive recurrent connections. We present an alternative view to explain the success of LSTMs: the gates themselves are versatile recurrent models that provide more representational power than previously appreciated. We do…
MET learns tabular data representations without data augmentations.
problem Lack of effective self-supervised learning methods for tabular data.
method Reconstruction-based approach using masked encoding, with separate representations for each coordinate and adversarial reconstruction loss.
result MET achieves state-of-the-art performance on five diverse tabular datasets, improving up to 9% over current methods.
Theoretical comparison of three invariance approaches in deep linear networks.
problem Understanding invariance in deep linear networks.
method Data augmentation, regularization, and hard-wiring approaches.
result Regularization introduces additional critical points, but they remain saddles except for the global optimum.
Study improves data quality assessment for structural monitoring data.
problem Ensuring reliability of structural health monitoring data.
method Probabilistic data quality assessment using a conditional diffusion model.
result Significantly improves accuracy of data quality assessment.
New model improves data augmentation for causal tasks.
problem Optimizing causal models robustly under Wasserstein distances.
method Proposes a new G-Causal Normalizing Flow architecture.
result Empirically outperforms standard generative models.
Paper extends multi-task Gaussian Cox processes for heterogeneous tasks.
problem Modeling multiple heterogeneous correlated tasks jointly.
method Data augmentation and mean-field approximation for non-conjugate Bayesian inference.
result Demonstrates improved performance and inference on synthetic and real data.
DiffSlack learns neural networks with nonlinear constraints via learnable slack variables.
problem Enforcing nonlinear inequality constraints in neural networks.
method DiffSlack reformulates inequalities as equalities with learnable slack variables, predicting them as part of the network output.
result DiffSlack achieves higher planning success rates and stronger geometric constraint satisfaction compared to existing methods.
In this paper, we propose a novel implicit semantic data augmentation (ISDA) approach to complement traditional augmentation techniques like flipping, translation or rotation. Our work is motivated by the intriguing property that deep networks are surprisingly good at linearizing features, such that certain directions …
Left atrium shape has been shown to be an independent predictor of recurrence after atrial fibrillation (AF) ablation. Shape-based representation is imperative to such an estimation process, where correspondence-based representation offers the most flexibility and ease-of-computation for population-level shape statisti…
This paper evaluates data enrichment techniques for rare event detection in manufacturing.
problem Rare events in manufacturing lead to unplanned downtime and high energy consumption.
method Time series data augmentation, sampling, and imputation techniques combined with supervised machine learning.
result Data enrichment enhances rare failure event detection and prediction by up to 48%.
FraudTransformer detects payment fraud by preserving event order and time gaps.
problem Detecting payment fraud in real-world banking streams with irregular time gaps.
method Augments a GPT-style architecture with a dedicated time encoder and a learned positional encoder.
result FraudTransformer outperforms classical and transformer baselines, achieving highest AUROC and PRAUC on held-out test set.
Develops Bayesian inference methods for gamma models.
problem Challenges in inference for models with gamma functions.
method Data augmentation scheme using Exponential Reciprocal Gamma distributions.
result Scalable EM and MCMC algorithms developed.
For machine learning task, lacking sufficient samples mean the trained model has low confidence to approach the ground truth function. Until recently, after the generative adversarial networks (GAN) had been proposed, we see the hope of small samples data augmentation (DA) with realistic fake data, and many works valid…
We develop a general variational inference method that preserves dependency among the latent variables. Our method uses copulas to augment the families of distributions used in mean-field and structured approximations. Copulas model the dependency that is not captured by the original variational distribution, and thus …
The paper constructs Yang-Baxter solutions using categorical augmented racks.
problem Solutions to the Yang-Baxter equation in knot theory.
method Interpreting augmented racks in tensor categories and constructing solutions using quantum heaps and Hopf algebra modules.
result Explicit constructions and infinite families of Yang-Baxter solutions are provided.
Given a model f that predicts a target y from a vector of input features x=x1,x2,…,xM, we seek to measure the importance of each feature with respect to the model's ability to make a good prediction. To this end, we consider how (on average) some measure of goodness or badness of prediction (wh…
Generative model for creating graphs with new communities.
problem Generating graphs with a new community structure.
method Fit Gaussian mixture model to latent space data and add new clusters based on MDL principle.
result Empirically demonstrated effectiveness of GCA for generating graphs with new community structures.
The study analyzes consistency-based SSL methods and proposes a new framework.
problem Improving performance in semi-supervised learning with minimal labeled data.
method Analysis of consistency-based SSL methods, including the Π-model, and proposing a new framework. result A simple extension of the Hidden Manifold Model that incorporates data-augmentation schemes.
Proposes Moment Exchange to use moments in image recognition models, improving generalization.
problem Discarding moments in image recognition models reduces stability and training time.
method Moment Exchange: replaces moments of learned features with another image's moments and interpolates labels.
result Improves generalization of recognition models across multiple datasets.
We reconsider a nonparametric density model based on Gaussian processes. By augmenting the model with latent Pólya--Gamma random variables and a latent marked Poisson process we obtain a new likelihood which is conjugate to the model's Gaussian process prior. The augmented posterior allows for efficient inference by Gi…
AEALT uses autoencoders to reduce text embedding dimensions for improved efficiency.
problem High dimensionality of text embeddings hinders downstream tasks.
method Factor-augmented supervised learning with autoencoders.
result AEALT outperforms conventional deep-learning approaches.
In this paper we propose a novel environmental sound classification approach incorporating unsupervised feature learning from codebook via spherical K-Means++ algorithm and a new architecture for high-level data augmentation. The audio signal is transformed into a 2D representation using a discrete wavelet transform …