Transformers cluster meaningless words around leaders for sentiment analysis.
problem Capturing context in sentiment analysis using transformers.
method Characterized transformers with hardmax self-attention and normalization, showing asymptotic convergence to clustered equilibrium.
result Transformers can effectively capture context by clustering meaningless words around leader words.
GT-PCA improves PCA for image and time series data.
problem Lack of robustness to transformations in PCA.
method GT-PCA is a neural network that estimates components invariant to specific transformations.
result GT-PCA outperforms alternative methods in synthetic and real data experiments.
New transforms improve signal classification and data analysis.
problem Improving signal classification and data analysis.
method Algebraic generative models and transport transforms.
result Classes of signals are transformed into convex sets, simplifying classification.
New analysis shows transformer models can't learn effectively.
problem Transformer models learning in context can't achieve general predictive accuracy.
method Empirical evidence and mathematical analysis of transformer architecture limitations.
result Transformers cannot achieve general predictive accuracy due to architectural limitations.
Transformers improve time series modeling by capturing long-range dependencies.
problem Capturing long-range dependencies in time series data.
method Summarized and reviewed adaptations of Transformers for time series analysis.
result Transformers enhance time series forecasting, anomaly detection, and classification.
The paper models musical motif transformations in Beethoven's works.
problem Understanding how motifs transform in symbolic music.
method Developed a probabilistic framework using Conditional Random Fields.
result Identified patterns of motif transformations and their co-occurrences.
Effective theory for Transformer initialization improves model performance.
problem Improving performance of Transformers at initialization.
method Effective-theory analysis of signal propagation in wide and deep Transformers.
result Particular width scalings of initialization and training hyperparameters.
Transformer predicts Ethereum prices using cross-currency correlation and sentiment analysis.
problem Predicting Ethereum cryptocurrency prices with limited data.
method Transformer-based neural network with cross-currency correlation and sentiment analysis.
result Transformer model outperforms other models on some parameters.
Firm size data usually do not show the normality that is often assumed in statistical analysis such as regression analysis. In this study we focus on two firm size data: the number of employees and sale. Those data deviate considerably from a normal distribution. To improve the normality of those data we transform them…
Transformers learn rich in-context dependencies efficiently.
problem Understanding how transformers learn long-range dependencies efficiently.
method Approximation and dynamics analysis of induction head mechanisms.
result Abrupt transition from lazy to rich mechanisms during training.
Transform learning improves K-means clustering for document analysis.
problem Improving K-means clustering for document analysis.
method Embedding K-means clustering loss into transform learning framework and solving jointly using ADMM.
result Improves over state-of-the-art in document clustering.
Alternative approach to model selection using transformation analysis.
problem Over-simplistic models lead to erroneous interpretations.
method Step-wise complexity reduction to identify simpler, better-interpretable models.
result Transformation models improve model fit and interpretability.
Transforms curves and surfaces for efficient geometric analysis.
problem Efficiently analyzing and comparing curves and surfaces.
method Square root velocity transformation for curves and intrinsic comparison for surfaces.
result Fundamental geometric properties of curves under the transformation.
The paper provides convergence guarantees for ODE-based generative models using transformers.
problem Theoretical guarantees for ODE-based generative models.
method A pre-trained autoencoder maps inputs to a latent space, and a transformer predicts the velocity field.
result The distribution of samples generated via estimated ODE flow converges to the target distribution in Wasserstein-2 distance.
Sparse manifold transform linearizes non-linear signal transformations.
problem Non-linear signal transformations in sensory data.
method Combines sparse coding, manifold learning, and slow feature analysis.
result Models sparse discreteness and low-dimensional manifold structure in natural scenes.
Study normal operators of double fibration transforms with conjugate points.
problem Normal operators of double fibration transforms with conjugate points.
method Stable conditions on the distribution of conjugate points, splitting into elliptic and Fourier integral operators.
result Normal operator splits into an elliptic pseudodifferential operator and Fourier integral operators.
This paper reviews digital transformation research from 2011-2024, focusing on corporate finance.
problem Lack of systematic review in digital transformation from corporate finance perspective.
method Combines bibliometric and content analysis methods.
result Emerging and rapidly growing focus on digital transformation, particularly in developed countries.
Injectivity of X-ray transform proven for non-smooth metrics.
problem Injectivity of X-ray transform on non-smooth metrics.
method Microlocal analysis of the normal operator, establishing ellipticity and smoothing properties.
result Injectivity of X-ray transform on L2 for metrics with finitely differentiable tensor. New filter bank sparsifying transforms outperform patch-based methods for image denoising.
problem Improving image denoising performance using data-adaptive sparsifying transforms.
method Proposes a new transform learning framework using undecimated perfect reconstruction filter banks, allowing independent filter length choice.
result Filter bank sparsifying transforms outperform existing patch-based methods for image denoising.
Sparse coding is a common approach to learning local features for object recognition. Recently, there has been an increasing interest in learning features from spatio-temporal, binocular, or other multi-observation data, where the goal is to encode the relationship between images rather than the content of a single ima…
New method eliminates domain size restrictions for X-ray transform inversion.
problem Injectivity and stability of X-ray transform in convex domains.
method Semiclassical analysis to invert X-ray transform without small domain assumptions.
result Elimination of domain size restrictions for injectivity and stability.
Transformers show strengths and weaknesses in complexity analysis.
problem Understanding the strengths and limitations of attention layers in transformers.
method Analysis of representation power through complexity parameters and task-specific constructions.
result Transformers can solve sparse averaging tasks with logarithmic complexity, but triple detection tasks require linear complexity.
Proposes a new feature preprocessing method using kernel density integral transformation.
problem Feature preprocessing for tabular data in machine learning and statistics.
method Kernel density integral transformation as a drop-in replacement or improved alternative to min-max scaling and quantile transformation.
result Frequently outperforms min-max scaling and quantile transformation with hyperparameter tuning.
Transforms game optimization dynamics into frequency domain for precise hyperparameter analysis.
problem Analyzing convergence of hyperparameters in game optimization.
method Frequency-domain framework using High-Resolution Differential Equations (HRDEs) and Laplace transforms.
result Derives precise convergence criteria for the Lookahead algorithm.
Hidformer improves stock price prediction accuracy using Transformer techniques.
problem Improving stock price prediction accuracy using machine learning.
method Adapted Transformer model (Hidformer) for stock price forecasting.
result Hidformer shows promising performance in stock price prediction.
A new deep learning approach enhances signature transforms for data analysis.
problem Improving data analysis through more flexible signature transforms.
method Combining deep learning with signature transforms to select terms data-dependently.
result Signature transform can be used as a pooling operation within neural networks.
DPI quantifies phase differences in 1D and multidimensional signals using Riesz transform.
problem Quantifying phase differences in signals of varying dimensions.
method Riesz transform framework for harmonic analysis.
result DPI detects hypersynchronization and subtle changes in images and artworks.
The paper explains how microlocal analysis solves geometric inverse problems.
problem Recovering geometric information from boundary measurements.
method Microlocal analysis applied to three inverse problems.
result Microlocal techniques solve specific inverse problems in Riemannian geometry.
Develops first robustness verification for complex Transformers.
problem Certify prediction behavior of Transformers with complex self-attention layers.
method Resolves challenges of cross-nonlinearity and cross-position dependency in Transformers.
result Certified robustness bounds are significantly tighter than Interval Bound Propagation.
Algorithm extracts wavefront sets from images using deep learning.
problem Extracting wavefront sets from images for imaging sciences.
method Combines shearlet transform and deep neural networks.
result Algorithm outperforms other methods in edge and ramp orientation detection.
This paper offers a simple method for Bayesian regression with unknown transformations.
problem Joint inference of unknown transformations and model parameters in Bayesian regression is computationally inefficient and cumbersome.
method The paper introduces a Bayesian nonparametric model via the Bayesian bootstrap to directly target the posterior distribution of the transformation.
result The approach delivers joint posterior consistency and efficient Monte Carlo inference for the transformation and all parameters.
The paper finds transformation formulas for quaternionic complex structures.
problem Quaternionic projective invariance of k-Cauchy-Fueter complex. method Explicit transformation formulae under mSL(n+1,H). result Quaternionic projectively invariant operator and defining density.
We verify CNNs using reachability analysis and transformations.
problem Verifying neural-based perception systems implemented by CNNs.
method Reachability analysis for feed-forward neural networks with MILP encodings.
result The notion of local robustness cannot be captured by previous robustness notions.
FineMorphs models smooth transformations for multivariate regression.
problem Efficiently modeling complex transformations for multivariate regression.
method Optimal control of affine and diffeomorphic transformations using smooth vector fields.
result FineMorphs can reduce dimensionality and adapt to large datasets.
SurvTRACE uses transformers to analyze survival times with competing events.
problem Analyzing survival times with multiple competing events and confounders.
method Transformer-based model for survival analysis, handling competing events and confounders.
result SurvTRACE outperforms existing methods in multi-event scenarios.
Generalizes shape analysis to Lie groups and homogeneous spaces.
problem Shape analysis on non-Euclidean spaces.
method Square Root Velocity Transform (SRVT) generalization to Lie groups and homogeneous manifolds.
result Shape analysis on Lie groups and homogeneous spaces.
New analysis shows PE in Transformers increases generalization gap and vulnerability.
problem Understanding the impact of PE on Transformer generalization and robustness.
method Generalization analysis and adversarial Rademacher bounds for a single-layer Transformer with trainable PE.
result PE systematically enlarges the generalization gap and makes models more vulnerable to attacks.
A method for vectorizing persistence diagrams simplifies topological data analysis.
problem Challenges in integrating persistence diagrams into machine learning pipelines.
method Quantized Persistence and Integral transforms of Diagrams (Qupid) using binning and discrete transforms.
result Qupid preserves highly competitive performances compared to state-of-the-art methods across various classification tasks.
Kendall transformation converts continuous data into categorical vectors for robust information theory.
problem Handling small number of observations and preserving ranking in continuous data.
method Kendall transformation converts ordered features into categorical vectors of pairwise order relations.
result Kendall transformation makes information theory methods applicable to continuous data robustly.
Study improves probabilistic circuits using transformations for better predictions.
problem Predictive limitations of probabilistic circuits in robotic scenarios.
method Integrates transformations into joint probability trees, extending their capabilities.
result Achieves higher likelihoods with fewer parameters on various data sets.
New Fourier transform method handles missing data and asynchronous observations.
problem Volatility inference with missing or asynchronous data.
method Spectral framework using Fourier transforms.
result Consistent volatility functional estimation with limit distributions.
This work solves TRPCA under linear transforms, recovering low-rank and sparse components.
problem Exact recovery of tensor low-rank and sparse components from their sum.
method Convex optimization with weighted tensor nuclear norm and ℓ1-norm.
result The convex program exactly recovers the components under certain incoherence conditions.
TraCeR uses transformers to analyze survival data with longitudinal covariates.
problem Handling longitudinal covariates and assessing model calibration in survival analysis.
method Transformer-based survival analysis framework with factorized self-attention architecture.
result TraCeR achieves significant performance improvements over state-of-the-art methods.
Deep LSTMs learn correlated representations of time-series data.
problem Learning nonlinear transformations and correlated embeddings of variable-length sequences.
method Use LSTMs to transform multi-view time-series data, then correlate outputs to find a fixed-dimensional representation.
result Deep LSTMs can effectively learn and project correlated representations of time-series data.
Transformer-based method for causal discovery with prior knowledge integration.
problem Complex nonlinear dependencies and spurious correlations in time series data.
method Multi-layer Transformer forecaster with gradient-based causal structure extraction and attention masking for prior knowledge integration.
result Significant improvement in causal discovery and causal lag estimation compared to state-of-the-art methods.
We present a generalization of independent component analysis (ICA), where instead of looking for a linear transform that makes the data components independent, we look for a transform that makes the data components well fit by a tree-structured graphical model. Treating the problem as a semiparametric statistical prob…
Geodesic X-ray transform proves injective for smooth one-forms on gas giant manifolds.
problem Injectivity of geodesic X-ray transform for one-forms on specific manifolds.
method Pestov identity and asymptotic analysis of short geodesics.
result Geodesic X-ray transform is solenoidally injective for smooth one-forms on gas giant manifolds.
This paper extends hypergraph construction to multivariate time series using signature transforms.
problem Constructing hypergraphs from collections of multivariate time series.
method Leveraging signature transforms to introduce controlled randomness and robustness.
result Validated on synthetic datasets, the method enhances robustness in hypergraph construction.