The visibility transformation embeds data position into signature features for efficient pattern recognition.
problem Embedding absolute position into signature features for efficient pattern recognition.
method The visibility transformation is put on a theoretical footing and used to embed absolute position into signature features efficiently.
result The generated feature set simplifies pattern recognition by accommodating nonlinear functions of absolute and relative values.
Transformers handle infinite dimensional inputs effectively by feature extraction and dynamic feature selection.
problem Understanding the approximation and estimation ability of Transformers with infinite dimensional inputs.
method Anisotropic smoothness analysis and feature extraction properties of Transformers.
result Transformers avoid the curse of dimensionality and dynamically select important features.
Proposes a new feature preprocessing method using kernel density integral transformation.
problem Feature preprocessing for tabular data in machine learning and statistics.
method Kernel density integral transformation as a drop-in replacement or improved alternative to min-max scaling and quantile transformation.
result Frequently outperforms min-max scaling and quantile transformation with hyperparameter tuning.
This paper analyzes kNN convergence over feature transformations.
problem The curse of dimensionality affects kNN performance in transformed feature spaces.
method Developed a novel analysis on kNN convergence rates over transformed features, linking properties of the transformed space to raw feature space.
result Theoretical analysis explains why some feature transformations are better for kNN.
Objective: A variety of pattern analysis techniques for model training in brain interfaces exploit neural feature dimensionality reduction based on feature ranking and selection heuristics. In the light of broad evidence demonstrating the potential sub-optimality of ranking based feature selection by any criterion, we …
Data augmentations are important ingredients in the recipe for training robust neural networks, especially in computer vision. A fundamental question is whether neural network features encode data augmentation transformations. To answer this question, we introduce a systematic approach to investigate which layers of ne…
Conventional mutual information (MI) based feature selection (FS) methods are unable to handle heterogeneous feature subset selection properly because of data format differences or estimation methods of MI between feature subset and class label. A way to solve this problem is feature transformation (FT). In this study,…
Sparse coding is a common approach to learning local features for object recognition. Recently, there has been an increasing interest in learning features from spatio-temporal, binocular, or other multi-observation data, where the goal is to encode the relationship between images rather than the content of a single ima…
Proposes a tabular transformer model to maintain feature effect intelligibility.
problem Losing marginal feature effects in deep tabular transformer networks.
method Adapts tabular transformer networks to identify marginal feature effects.
result The model accurately identifies marginal feature effects, matching black-box performance while maintaining intelligibility.
An audiovisual speaker conversion method is presented for simultaneously transforming the facial expressions and voice of a source speaker into those of a target speaker. Transforming the facial and acoustic features together makes it possible for the converted voice and facial expressions to be highly correlated and f…
FLFE improves machine learning by efficiently and securely transforming features.
problem Efficiently and securely transforming features in a multi-party setting.
method FLFE uses a pre-learning pattern to selectively transform features, reducing communication overhead.
result FLFE outperforms evaluation-based approaches in feature transformation efficiency.
LLT transforms time series features based on linear laws.
problem Classifying univariate and multivariate time series.
method Time-delay embedding, spectral decomposition, and feature transformation.
result Transformed features improve classification accuracy.
Learning invariant representations is an important problem in machine learning and pattern recognition. In this paper, we present a novel framework of transformation-invariant feature learning by incorporating linear transformations into the feature learning algorithms. For example, we present the transformation-invari…
Transformers learn unseen tasks via prompts without fine-tuning.
problem Understanding how transformers learn unseen tasks without additional fine-tuning.
method Structured data model, gradient descent, two-phase convergence analysis.
result Transformers can learn linear function classes via in-context learning.
How can we recognise social roles of people, given a completely unlabelled social network? We present a transfer learning approach to network role classification based on feature transformations from each network's local feature distribution to a global feature space. Experiments are carried out on real-world datasets.…
The characteristics (or numerical patterns) of a feature vector in the transform domain of a perturbation model differ significantly from those of its corresponding feature vector in the input domain. These differences - caused by the perturbation techniques used for the transformation of feature patterns - degrade the…
ALT improves TSC by capturing complex patterns in time series data.
problem Challenges in traditional TSC methods with time series complexity and variability.
method ALT incorporates variable-length shifted time windows to enhance LLT for better feature representation.
result ALT achieves state-of-the-art performance with few hyperparameters.
New approach interprets machine learning models through feature space transformations.
problem Interpreting models with strongly dependent features in high-dimensional spaces.
method Feature space transformations, including PCA and partial orthogonalization.
result Enhances model interpretation tools for domain experts.
The study explains how transformer components enable in-context learning.
problem Understanding how transformer components contribute to in-context learning.
method Analyzed a two-attention-layer transformer model trained on Markov chain data.
result Gradient flow converges to a limiting model with a copier, selector, and classifier mechanism.
Deep learning uses layers of transformations to predict structured data with uncertainty.
problem Predicting structured high-dimensional data efficiently and with uncertainty.
method Applying layers of semi-affine input transformations to find features for probabilistic statistical methods.
result Achieves scalable prediction rules with uncertainty quantification and feature selection.
Study examines infinite limits of transformer dynamics, identifying key parameterizations.
problem Understanding the training dynamics of transformer models in the feature learning regime.
method Analysis of infinite scaling limits using dynamical mean field theory.
result Identified parameterizations that admit well-defined infinite width and depth limits.
Optimizes LightGBM for stock market forecasting with novel feature engineering and transformation methods.
problem Accurately forecasting stock market fluctuations to mitigate risks.
method Feature engineering and transformation methods for LightGBM optimization.
result Log Returns, Returns and EMA Difference Ratio are the most effective target variable transformations.
CascadeXML improves multi-resolution learning for XMC with transformer features.
problem Learning subset labels from millions of choices with trade-offs between performance and computation.
method End-to-end multi-resolution learning pipeline using transformer multi-layer architecture.
result Significantly outperforms existing approaches on benchmark datasets.
Latent feature models are attractive for image modeling, since images generally contain multiple objects. However, many latent feature models ignore that objects can appear at different locations or require pre-segmentation of images. While the transformed Indian buffet process (tIBP) provides a method for modeling tra…
Kernel models learn low-dimensional predictive subspaces from input data.
problem Learning effective feature transformations in kernel models.
method Study of a compositional kernel ridge regression model.
result Global minimizers of the objective function identify the subspace with high probability.
Equivariance is a nice property to have as it produces much more parameter efficient neural architectures and preserves the structure of the input through the feature mapping. Even though some combinations of transformations might never appear (e.g. an upright face with a horizontal nose), current equivariant architect…
A deep learning framework learns wavelet packet transforms for efficient feature extraction.
problem Efficiently extracting meaningful time-frequency features from high-frequency signals.
method Learnable wavelet packet transforms using deep learning.
result Improved spectral leakage and enhanced anomaly detection performance.
A method uses ITD and XGBoost for precise power transformer fault diagnosis.
problem Fault diagnosis of power transformers using DGA data.
method Ranking DGA parameters by skewness, extracting ITD features, and using an XGBoost classifier.
result The method achieves over 95% accuracy in classification.
Convolutional neural networks (CNNs) have achieved state-of-the-art results on many visual recognition tasks. However, current CNN models still exhibit a poor ability to be invariant to spatial transformations of images. Intuitively, with sufficient layers and parameters, hierarchical combinations of convolution (matri…
Method extracts features from signals for classification with explainability.
problem Lack of interpretability in signal classification models.
method Combining scattering transform and multiclass logistic regression with zeroth-order optimization.
result Uncovered the meaning of scattering transform coefficients.
A wide variety of deep learning techniques from style transfer to multitask learning rely on training affine transformations of features. Most prominent among these is the popular feature normalization technique BatchNorm, which normalizes activations and then subsequently applies a learned affine transform. In this pa…
CAM-GAN improves GANs for continual learning with efficient feature map transformations.
problem Efficient continual learning for GANs with reduced parameter growth.
method Designing and leveraging parameter-efficient feature map transformations, including global and task-specific parameters, residual bias, and Fisher information matrix.
result Significantly improved model performance and high-quality samples with fewer parameters.
SISR improves feature attribution in complex payoff schemes.
problem Distorted feature attributions due to non-additive payoff functions and high-dimensional feature spaces.
method Sparse Isotonic Shapley Regression (SISR) learns a monotonic transformation to restore additivity and enforces L0 sparsity.
result SISR achieves strong support recovery and stable attributions across various payoff schemes.
Theoretical analysis of vision transformers' performance with MAE and CL objectives.
problem Understanding the distinct representations learned by vision transformers with MAE and CL objectives.
method Modeling visual data distribution and analyzing ViTs training dynamics with gradient descent.
result ViTs trained with MAE objectives learn both global and local features, while CL-trained ViTs favor global features.
Relational data representations have become an increasingly important topic due to the recent proliferation of network datasets (e.g., social, biological, information networks) and a corresponding increase in the application of statistical relational learning (SRL) algorithms to these domains. In this article, we exami…
ModelDiff compares learning algorithms by identifying feature transformations.
problem Comparing models trained with different learning algorithms.
method ModelDiff uses datamodels framework to find distinguishing feature transformations.
result ModelDiff can compare models trained with/without data augmentation, pre-training, and different SGD hyperparameters.
Neural networks' feature geometry evolves like discrete Ricci flow.
problem Understanding neural feature representations and their geometric transformations.
method Approximating input manifold with geometric graphs and analyzing their evolution during training.
result Neural feature geometry evolves like discrete Ricci flow, with nonlinear activations playing a crucial role.
Enhances GNNs with text features for better fake news detection.
problem Detecting disinformation on social media using GNNs.
method Integrates Transformer-based textual features into GNNs.
result Contextual text representations improve GNN performance by 33.8% in Macro F1.
This research examines how data transformations affect adversarial robustness in recurrent neural networks.
problem Adversarial examples reduce machine learning accuracy, especially in high-dimensional datasets.
method Analysis of feature selection, dimensionality reduction, and trend extraction techniques on recurrent neural networks.
result Data transformations may increase vulnerability to adversarial samples, but only if they approximate intrinsic dimensionality and maintain manifold coverage.
Enhanced transformer converts whispered speech to natural speech.
problem Machine recognition of whispered speech is challenging.
method Proposes an enhanced transformer architecture trained end-to-end using supervised learning.
result Similar formant distributions of converted speech to groundtruth.
Transformers can scale both context and task, but MLPs can only scale task.
problem Understanding and scaling In-Context Learning in transformers.
method Simplified transformer architecture, feature map, and MLP combination.
result Simplified transformer can perform ICL and context-scaling but not task-scaling.
B-cos transforms improve neural network interpretability by aligning weights.
problem Improving interpretability of deep neural networks.
method Replacing linear transforms with B-cos transforms that promote weight-input alignment during training.
result B-cos transforms lead to highly interpretable and task-relevant linear summaries of neural network computations.
New method for curve comparison using iterated integrals and moving frames.
problem Comparing curves robustly to noise and transformations.
method Moving frame method paired with log-signature transform.
result Algorithmic construction of invariants for curve equivalence under rigid motions.
New RFs reduce kernel approximation variance and improve Transformer performance.
problem Efficient approximation of Gaussian and softmax kernels for kernel methods and Transformers.
method Parameterized, positive, non-trigonometric RFs optimized for variance reduction.
result Significant variance reduction in practice, outperforming previous methods.
A novel text-independent speaker identification (SI) method is proposed. This method uses the Mel-frequency Cepstral coefficients (MFCCs) and the dynamic information among adjacent frames as feature sets to capture speaker's characteristics. In order to utilize dynamic information, we design super-MFCCs features by cas…
Study excess risk in statistical inference with transformations.
problem Excess risk in estimating random variables from feature vectors and transformations.
method Characterize lossless transformations, develop test statistics, and information-theoretic bounds.
result Strongly consistent partitioning test statistic for lossless transformations.
A novel graph spectral method for mixed categorical and numerical data.
problem Feature learning for mixed data types (numerical and categorical).
method Graph spectral decomposition of the graph Laplacian to model probabilistic dependence structure.
result Increased separability and clusterability of observations in the transformed feature space.
In trying to generalize Bianchi's Bäcklund transformation of quadrics to Bäcklund transformations of isometric deformations of other (classes of) surfaces, we investigate basic features of the isometric deformation of surfaces via the Bäcklund transformation with isometric correspondence of leaves of a general nature (…