Data augmentation (DA) is fundamental against overfitting in large convolutional neural networks, especially with a limited training dataset. In images, DA is usually based on heuristic transformations, like geometric or color transformations. Instead of using predefined transformations, our work learns data augmentati…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “data transformations”
Firm size data usually do not show the normality that is often assumed in statistical analysis such as regression analysis. In this study we focus on two firm size data: the number of employees and sale. Those data deviate considerably from a normal distribution. To improve the normality of those data we transform them…
Improves data normality with robust transformations.
problem Skewed data distribution.
method Modified Box-Cox and Yeo-Johnson transformations with robust parameter estimation.
result Transformed data approximates normality in the center with outliers.
Data is said to follow the transform (or analysis) sparsity model if it becomes sparse when acted on by a linear operator called a sparsifying transform. Several algorithms have been designed to learn such a transform directly from data, and data-adaptive sparsifying transforms have demonstrated excellent performance i…
New transforms improve signal classification and data analysis.
problem Improving signal classification and data analysis.
method Algebraic generative models and transport transforms.
result Classes of signals are transformed into convex sets, simplifying classification.
Diffusion Transformer Captures Spatial-Temporal Dependencies: A Theory for Gaussian Process Datacs.LG
Diffusion Transformer captures spatial-temporal dependencies in sequential data.
problem Capturing rich spatial and temporal dependencies in sequential data.
method Established theoretical guarantees for diffusion transformers learning Gaussian process data.
result Spatial-temporal dependencies are captured within attention layers of diffusion transformers.
Transformer-M learns molecular data in 2D or 3D formats.
problem Learning models for molecules are limited to specific data formats.
method Developed a Transformer-based model that can handle 2D and 3D molecular data.
result Transformer-M achieves strong performance on both 2D and 3D molecular tasks.
This paper explains how model invariance improves generalization using data transformations.
problem Understanding why model invariance leads to better generalization performance.
method Introducing sample cover induced by transformations and refining generalization bounds.
result The sample covering number can be used to evaluate and select suitable data transformations.
CNNs encode data augmentation transformations, especially in early layers.
problem Whether neural network features encode data augmentation transformations.
method Systematic approach using pre-trained vision models to predict augmentation transformations.
result Neural network features encode data augmentation transformations, especially in early layers.
Transformers fine-tuned on synthetic data boost tabular data classification performance.
problem Improving tabular data classification accuracy.
method Fine-tuning ICL-transformers on synthetic datasets with complex decision boundaries.
result Fine-tuned ICL-transformers outperform regular neural networks on real-world datasets.
A new method for discrete data normalizing flows using latent transformations.
problem Challenges in parameterizing bijective transformations for discrete data.
method Predict a distribution over latent transformations to make the marginal likelihood differentiable.
result Discrete-data normalizing flows can be trained using gradient-based learning with unbiased score function estimation.
The Weyl transform is introduced as a rich framework for data representation. Transform coefficients are connected to the Walsh-Hadamard transform of multiscale autocorrelations, and different forms of dyadic periodicity in a signal are shown to appear as different features in its Weyl coefficients. The Weyl transform …
Self-supervision is key to extending use of deep learning for label scarce domains. For most of self-supervised approaches data transformations play an important role. However, up until now the impact of transformations have not been studied. Furthermore, different transformations may have different impact on the syste…
The article recovers tensor fields from partial data using weighted divergent ray transforms.
problem Recovering tensor fields from partial data.
method Weighted divergent ray transforms, unique continuation property of fractional Laplacian, explicit reconstruction formulas.
result Recovery of symmetric -tensor fields and unique continuation for vector fields and symmetric 2-tensor fields.
Study linear transformations' effects on data augmentation for improved estimation.
problem Improving performance in image and text classification tasks.
method Examined a family of linear transformations in over-parametrized linear regression settings.
result Transformations that preserve labels or mix data can improve estimation.
Probabilistic STNs improve image classification and robustness.
problem Training and robustness issues in STNs.
method Probabilistic extension of STNs that estimates stochastic transformations.
result Improved classification performance, robustness, and model calibration.
Bayesian model for discrete data with conditional transformations.
problem Handling discrete ordinal and count data with excess zeros.
method Bayesian framework with conditional transformation functions and modular MCMC algorithm.
result Flexible modeling of linear and nonlinear covariate effects for ordinal and count data.
Kendall transformation converts continuous data into categorical vectors for robust information theory.
problem Handling small number of observations and preserving ranking in continuous data.
method Kendall transformation converts ordered features into categorical vectors of pairwise order relations.
result Kendall transformation makes information theory methods applicable to continuous data robustly.
Transformer architecture improved with credibility mechanism for better model performance.
problem Improving predictive models in tabular data.
method Introducing a credibility mechanism to the Transformer architecture.
result Credibility Transformer leads to superior predictive models compared to state-of-the-art models.
Transformers can learn noisy linear systems with depth and IID data.
problem Learning noisy linear dynamical systems with transformers.
method Theoretical analysis of multi-layer and single-layer transformers with respect to -testing loss.
result Single-layer transformers have a non-diminishing lower bound on approximation error, suggesting depth separation.
The visibility transformation embeds data position into signature features for efficient pattern recognition.
problem Embedding absolute position into signature features for efficient pattern recognition.
method The visibility transformation is put on a theoretical footing and used to embed absolute position into signature features efficiently.
result The generated feature set simplifies pattern recognition by accommodating nonlinear functions of absolute and relative values.
The study explains transformer scaling laws using statistical and approximation theories.
problem Understanding why transformer scaling laws exist for large models trained on low-dimensional data.
method Established statistical estimation and mathematical approximation theories for transformers on low-dimensional manifolds.
result Predicted a power law between generalization error and model and data sizes, with power depending on intrinsic data dimension.
Task-agnostic data augmentation shows little benefit for pretrained transformers.
problem Evaluating the effectiveness of task-agnostic data augmentation on pretrained transformers.
method Conducted a systematic examination of two data augmentation techniques (Easy Data Augmentation and Back-Translation) across 5 tasks, 6 datasets, and 3 pretrained transformer models.
result Data augmentation techniques previously effective for non-pretrained models fail to consistently improve performance for pretrained transformers, even with limited training data.
Pre-training on different modalities improves Transformer performance in offline reinforcement learning.
problem Improving Transformer-based models for offline reinforcement learning tasks.
method Empirical investigation of pre-training on language and vision data, analysis of internal representations, and fine-tuning with no context.
result Pre-trained Transformers with language data can utilize context-like information to solve reinforcement learning tasks efficiently.
Deep generative networks have been widely used for learning mappings from a low-dimensional latent space to a high-dimensional data space. In many cases, data transformations are defined by linear paths in this latent space. However, the Euclidean structure of the latent space may be a poor match for the underlying lat…
Scattering transforms are non-trainable deep convolutional architectures that exploit the multi-scale resolution of a wavelet filter bank to obtain an appropriate representation of data. More importantly, they are proven invariant to translations, and stable to perturbations that are close to translations. This stabili…
Optimizing data movement significantly improves transformer training efficiency.
problem Data movement is a major bottleneck in training transformers.
method Developed a recipe to globally optimize data movement in transformers.
result Achieved up to 1.30x performance improvement over state-of-the-art frameworks.
Our goal is to extract meaningful transformations from raw images, such as varying the thickness of lines in handwriting or the lighting in a portrait. We propose an unsupervised approach to learn such transformations by attempting to reconstruct an image from a linear combination of transformations of its nearest neig…
This paper recovers input data from transformer models using attention weights.
problem Recovering input data from transformer models for security and privacy concerns.
method Introducing an algorithm to minimize the loss function between expected and actual outputs of transformers.
result The algorithm successfully recovers input data from attention weights and outputs of transformers.
New STKR estimators use unlabeled data for smoother function learning.
problem Leveraging unlabeled data for smoother function learning.
method Spectrally transformed kernel regression (STKR) with scalable implementations.
result STKR can learn any sufficiently smooth function.
Paper unifies propositionalization and embedding for relational learning.
problem Data fusion from diverse input formats into a single table.
method Unified framework combining propositionalization and embedding.
result New algorithms outperform existing relational learners.
Data augmentation is a ubiquitous technique for increasing the size of labeled training sets by leveraging task-specific data transformations that preserve class labels. While it is often easy for domain experts to specify individual transformations, constructing and tuning the more sophisticated compositions typically…
Transforms solutions of Davey-Stewartson II equation geometrically.
problem Solving the Davey-Stewartson II equation.
method Moutard transform and spinor representation of surfaces.
result Constructs examples of solutions with smooth initial data losing regularity.
Transformers can learn new tasks from diverse pretraining data but struggle with out-of-domain tasks.
problem Transformer models' ability to learn new tasks in-context is limited by their pretraining data coverage.
method Investigation of transformer models trained on pairs, comparing in-context learning capabilities across different task families.
result Transformers can identify and learn within task families in their pretraining data but fail with out-of-domain tasks.
Study compares LSTM and Transformer models in financial time series prediction.
problem Comparing LSTM and Transformer models for financial time series prediction.
method Various LSTM-based and Transformer-based models compared on financial tasks; DLSTM and new Transformer architecture designed.
result Transformer-based models show limited advantage in absolute price sequence prediction, while LSTM-based models perform better on difference sequences.
Transformers improve stock forecasting with federated learning.
problem Overfitting, data scarcity, and privacy issues in transformer-based time series forecasting.
method Attentive federated transformers for time series stock forecasting.
result Proposed scheme outperforms conventional training schemes in stock forecasting.
Proposes a new feature preprocessing method using kernel density integral transformation.
problem Feature preprocessing for tabular data in machine learning and statistics.
method Kernel density integral transformation as a drop-in replacement or improved alternative to min-max scaling and quantile transformation.
result Frequently outperforms min-max scaling and quantile transformation with hyperparameter tuning.
A great number of deep learning based models have been recently proposed for automatic music composition. Among these models, the Transformer stands out as a prominent approach for generating expressive classical piano performance with a coherent structure of up to one minute. The model is powerful in that it learns ab…
L-GATr transforms high-energy physics data using geometric algebra and Lorentz symmetry.
problem Extracting scientific understanding from particle-physics experiments with high precision and efficiency.
method L-GATr, a geometric algebra Transformer, representing data in 4D space-time and being equivariant under Lorentz transformations.
result L-GATr achieves performance comparable to or better than domain-specific baselines on regression, classification, and generative tasks.
The characteristics (or numerical patterns) of a feature vector in the transform domain of a perturbation model differ significantly from those of its corresponding feature vector in the input domain. These differences - caused by the perturbation techniques used for the transformation of feature patterns - degrade the…
GT-PCA improves PCA for image and time series data.
problem Lack of robustness to transformations in PCA.
method GT-PCA is a neural network that estimates components invariant to specific transformations.
result GT-PCA outperforms alternative methods in synthetic and real data experiments.
Develops a Causal Transformer for estimating counterfactual outcomes from longitudinal data.
problem Estimating counterfactual outcomes over time from observational data is challenging due to complex, long-range dependencies.
method Combines three transformer subnetworks with separate inputs for time-varying covariates, previous treatments, and previous outcomes into a joint network with in-between cross-attentions. Uses a custom, end-to-end training procedure with a counterfactual domain confusion loss to address confounding bias.
result Achieves superior performance over current baselines in synthetic and real-world datasets.
We reduce boundary determination of an unknown function and its normal derivatives from the (possibly weighted and attenuated) broken ray data to the injectivity of certain geodesic ray transforms on the boundary. For determination of the values of the function itself we obtain the usual geodesic ray transform, but for…
Transformers simplify modeling of small longitudinal cohort data by reducing parameters and incorporating attention mechanisms.
problem Challenges in modeling longitudinal cohort data due to complex temporal dependencies and large dataset requirements.
method Simplified transformer architecture with attention mechanism, autoregressive model, and kernel-based temporal decay.
result The approach recovers contextual dependencies even with small datasets, identifying temporal patterns in stress and mental health.
The signature is an infinite graded sequence of statistics known to characterise a stream of data up to a negligible equivalence class. It is a transform which has previously been treated as a fixed feature transformation, on top of which a model may be built. We propose a novel approach which combines the advantages o…
Deep learning model estimates uncertainty in complex regression tasks.
problem Uncertainty quantification in probabilistic regression predictions.
method Combines statistical and deep learning transformation models using gradient descent.
result State-of-the-art performance on small datasets and complex image data.
We study the invariance characteristics of pre-trained predictive models by empirically learning transformations on the input that leave the prediction function approximately unchanged. To learn invariant transformations, we minimize the Wasserstein distance between the predictive distribution conditioned on the data i…
This paper offers a simple method for Bayesian regression with unknown transformations.
problem Joint inference of unknown transformations and model parameters in Bayesian regression is computationally inefficient and cumbersome.
method The paper introduces a Bayesian nonparametric model via the Bayesian bootstrap to directly target the posterior distribution of the transformation.
result The approach delivers joint posterior consistency and efficient Monte Carlo inference for the transformation and all parameters.