In this paper, we propose several dictionary learning algorithms for sparse representations that also impose specific structures on the learned dictionaries such that they are numerically efficient to use: reduced number of addition/multiplications and even avoiding multiplications altogether. We base our work on facto…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “multiplicative transformations”
Hybrid approach combines transformer and Bayesian filtering for robust multiple particle tracking.
problem Challenges in tracking multiple particles in noisy scenes due to combinatorial explosion of hypotheses.
method Attention-Bayesian hybrid framework using transformer for association and Bayesian filtering for pruning hypotheses.
result Improved tracking accuracy and robustness against spurious detections.
Improved machine translation with INT8 hardware using a novel training method.
problem Training accurate machine translation models with limited hardware precision.
method Convert all Transformer matrix multiplications to 8-bit integer (INT8) without sacrificing accuracy.
result INT8 Transformer models achieve BLEU scores 99.3% to 100% relative to FP32 models.
Study very stable Higgs bundles on Riemann surfaces, linking to multiplicity and mirror symmetry.
problem Existence and properties of very stable Higgs bundles.
method Bialynicki-Birula theory, -actions, Hecke transformations, Fourier-Mukai transforms.
result Precise formula for multiplicity of very stable components of global nilpotent cone.
MPP trains a transformer to predict multiple physical systems, improving accuracy across various tasks.
problem Training models for specific physical systems is inefficient and requires fine-tuning.
method MPP trains a shared transformer on multiple heterogeneous physical systems, projecting fields into a shared embedding space.
result A single MPP-pretrained transformer outperforms task-specific models on all pretraining sub-tasks and downstream tasks.
We develop a deep autoencoder architecture that can be used to find a coordinate transformation which turns a nonlinear PDE into a linear PDE. Our architecture is motivated by the linearizing transformations provided by the Cole-Hopf transform for Burgers equation and the inverse scattering transform for completely int…
HFformer outperforms LSTM in high-frequency trading with multiple signals.
problem Improving high-frequency trading performance using deep learning models.
method Introducing HFformer, a hybrid Transformer model for time series forecasting.
result HFformer achieves higher cumulative PnL than LSTM in backtesting.
Study multiplicity-free covering of graded manifolds, proving equivalence of categories.
problem Equivalence of categories of graded manifolds and symmetric vector bundles.
method Defined and computed multiplicity-free covering, showed deck transformation group isomorphic to .
result Equivalence of categories of graded manifolds and symmetric -fold vector bundles.
New CMC surfaces with dihedral symmetry constructed from Darboux transforms.
problem Constructing closed CMC surfaces with specific symmetries.
method Using Darboux transforms on multiple covers to create new CMC surfaces.
result Explicit parametrisations of new closed CMC surfaces with dihedral symmetry.
SurvTRACE uses transformers to analyze survival times with competing events.
problem Analyzing survival times with multiple competing events and confounders.
method Transformer-based model for survival analysis, handling competing events and confounders.
result SurvTRACE outperforms existing methods in multi-event scenarios.
Learning multiple tasks across heterogeneous domains is a challenging problem since the feature space may not be the same for different tasks. We assume the data in multiple tasks are generated from a latent common domain via sparse domain transforms and propose a latent probit model (LPM) to jointly learn the domain t…
Transformers use ReLUs to approximate softmax efficiently.
problem Analyzing resource usage in softmax transformer models.
method Translating ReLU approximation results to softmax attention mechanisms.
result Economic resource bounds for softmax attention mechanisms.
Fast linear transforms are ubiquitous in machine learning, including the discrete Fourier transform, discrete cosine transform, and other structured transformations such as convolutions. All of these transforms can be represented by dense matrix-vector multiplication, yet each has a specialized and highly efficient (su…
BoostTransformer uses boosting to improve transformer efficiency and accuracy.
problem Heavy computational resources and hyperparameter tuning in transformer architectures.
method Augments transformers with boosting principles through subgrid token selection and importance-weighted sampling, incorporating a least square boosting objective directly into the pipeline.
result BoostTransformer demonstrates faster convergence and higher accuracy compared to standard transformers.
Fusion of transformer networks using optimal transport for improved performance.
problem Improving performance of transformer-based models through fusion.
method Exploiting optimal transport for soft alignment of transformer components.
result Consistently outperforms vanilla fusion and individual parent models.
Paper proposes a robust framework for detecting multiple periodic components in time series.
problem Detecting multiple periodic components in time series with interlaced patterns and external noise.
method Applying maximal overlap discrete wavelet transform to isolate periodic components, ranking them by wavelet variance, and detecting single periodicity robustly.
result The proposed algorithm outperforms other methods for both single and multiple periodicity detection.
MultiRocket boosts TSC speed and accuracy with pooling and transformations.
problem Efficient time series classification with high accuracy.
method Multiple pooling operators and transformations applied to raw and differenced series.
result MultiRocket outperforms MiniRocket and is competitive with state-of-the-art methods in terms of accuracy and speed.
We consider an application involving a financial quadratic portfolio of options, when the joint underlying log-returns changes with multivariate elliptic distribution. This motivates the needs for methods for the approximation of multiple integrals over hyperboloids. A transformation is used to reduce the hyperboloid i…
The classical Liouville Theorem on conformal transformations determines local conformal transformations on the Euclidean space of dimension . Its natural adaptation to the general framework of Riemannian structures is the 2-rigidity of conformal transformations, that is such a transformation is fully determined…
Improved signal classification using multiple wavelets and their smooth coefficients.
problem Signal classification accuracy declines with reduced attributes.
method Transform data with multiple wavelets, combine outputs, apply ensemble classifiers.
result Proposed technique outperforms raw data and single wavelet approaches.
LOFT separates subspace rotation and transformation for orthogonal fine-tuning.
problem Conflating subspace rotation and transformation in orthogonal fine-tuning.
method LOFT explicitly separates subspace rotation and transformation, using task-aware support selection.
result LOFT recovers principal-subspace orthogonal adaptation and improves efficiency-performance trade-off.
We show that for any complete connected Kähler manifold the index of the group of complex affine transformations in the group of c-projective transformations is at most two unless the Kähler manifold is isometric to complex projective space equipped with a positive constant multiple of the Fubini-Study metric. This est…
Improves contrastive learning invariance with novel training objectives and feature averaging.
problem Contrastive learning's implicit invariance is insufficient for robust performance.
method Introduces a novel training objective and feature averaging approach to enforce invariance.
result Improved performance and robustness to transformations on downstream tasks.
MIMONets: Multiple-Input-Multiple-Output Neural Networks Exploiting Computation in Superpositioncs.LG
MIMONets speed up neural network inference by processing multiple inputs in parallel.
problem Reducing computational cost in neural network inference for large datasets.
method Proposes MIMONets, which augment neural network architectures with variable binding mechanisms to handle multiple inputs in superposition.
result Achieves significant speedups (2-4x) with minimal accuracy loss, demonstrating adaptability across different architectures.
Outlier detection aims to identify unusual data instances that deviate from expected patterns. The outlier detection is particularly challenging when outliers are context dependent and when they are defined by unusual combinations of multiple outcome variable values. In this paper, we develop and study a new conditiona…
We analyze a simple asset transfer model in which the transfer amount is a fixed fraction of the giver's wealth. The model is analyzed in a new way by Laplace transforming the master equation, solving it analytically and numerically for the steady-state distribution, and exploring the solutions for various values o…
Transformers learn to recall with non-orthogonal embeddings in realistic settings.
problem Understanding how transformers store and retrieve knowledge in practical scenarios.
method Analyzing a single-layer transformer with random embeddings trained on a token-retrieval task.
result Explicit formulas for the model's storage capacity reveal a multiplicative dependence on sample size, embedding dimension, and sequence length.
Hierarchical Hybrid Sliced Wasserstein: A Scalable Metric for Heterogeneous Joint Distributionscs.CV
Extends SW and GSW to compare heterogeneous joint distributions.
problem Limited applicability of SW and GSW to heterogeneous joint distributions.
method Introduces HHRT and PGRT to extend SW and GSW.
result H2SW distance for heterogeneous joint distributions.
Transformers can learn optimal variable selection in group-sparse classification.
problem Understanding how transformers leverage attention to select relevant variables in group-sparse classification.
method Training a one-layer transformer using gradient descent to select variables from one group of input variables.
result A one-layer transformer can correctly leverage the attention mechanism to select variables, disregarding irrelevant ones.
Transformer models show robustness across domains with domain adversarial training.
problem Domain adaptation from multiple sources with no labeled data.
method Domain adversarial training and mixture of experts.
result Domain adversarial training improves representation but not performance.
Probabilistic STNs improve image classification and robustness.
problem Training and robustness issues in STNs.
method Probabilistic extension of STNs that estimates stochastic transformations.
result Improved classification performance, robustness, and model calibration.
Transformers simplify modeling of small longitudinal cohort data by reducing parameters and incorporating attention mechanisms.
problem Challenges in modeling longitudinal cohort data due to complex temporal dependencies and large dataset requirements.
method Simplified transformer architecture with attention mechanism, autoregressive model, and kernel-based temporal decay.
result The approach recovers contextual dependencies even with small datasets, identifying temporal patterns in stress and mental health.
The alignment of a set of objects by means of transformations plays an important role in computer vision. Whilst the case for only two objects can be solved globally, when multiple objects are considered usually iterative methods are used. In practice the iterative methods perform well if the relative transformations b…
We incorporate Tensor-Product Representations within the Transformer in order to better support the explicit representation of relation structure. Our Tensor-Product Transformer (TP-Transformer) sets a new state of the art on the recently-introduced Mathematics Dataset containing 56 categories of free-form math word-pr…
Many-to-Many VTN improves voice conversion across multiple speakers.
problem Voice conversion across multiple speakers.
method Sequence-to-sequence learning framework with many-to-many VTN architecture.
result Improved sound quality and speaker similarity compared to baseline methods.
Multi-View Treelet Transformstat.ML
Current multi-view factorization methods make assumptions that are not acceptable for many kinds of data, and in particular, for graphical data with hierarchical structure. At the same time, current hierarchical methods work only in the single-view setting. We generalize the Treelet Transform to the Multi-View Treelet …
Sparse coding is a common approach to learning local features for object recognition. Recently, there has been an increasing interest in learning features from spatio-temporal, binocular, or other multi-observation data, where the goal is to encode the relationship between images rather than the content of a single ima…
Investigates the fundamental components of attention mechanisms.
problem Understanding the building blocks of attention in deep learning.
method Classified and studied three key mechanisms: additive, multiplicative output, and synaptic attention.
result Additive activation attention is central in proofs of lower bounds.
Segmentation maps of medical images annotated by medical experts contain rich spatial information. In this paper, we propose to decompose annotation maps to learn disentangled and richer feature transforms for segmentation problems in medical images. Our new scheme consists of two main stages: decompose and integrate. …
Unified framework for learning function representations using INRs and Transformers.
problem Scalability and efficiency limitations in existing generative models.
method Integrates INRs and Transformer-based hypernetworks into latent variable models.
result Improved scalability, expressiveness, and generalization over existing models.
Most existing fingerprints-based indoor localization approaches are based on some single fingerprints, such as received signal strength (RSS), channel impulse response (CIR), and signal subspace. However, the localization accuracy obtained by the single fingerprint approach is rather susceptible to the changing environ…
Although nonstationary data are more common in the real world, most existing causal discovery methods do not take nonstationarity into consideration. In this letter, we propose a kernel embedding-based approach, ENCI, for nonstationary causal model inference where data are collected from multiple domains with varying d…
Transforming Causality: Transformer-Based Temporal Causal Discovery with Prior Knowledge Integrationcs.LG
Transformer-based method for causal discovery with prior knowledge integration.
problem Complex nonlinear dependencies and spurious correlations in time series data.
method Multi-layer Transformer forecaster with gradient-based causal structure extraction and attention masking for prior knowledge integration.
result Significant improvement in causal discovery and causal lag estimation compared to state-of-the-art methods.
Paper improves neural network models for MOOC student course prediction.
problem Improving neural network-based predictive methods for MOOC student course trajectory modeling.
method Enhanced LSTM networks and Transformer architecture applied to MOOC student data.
result Improved accuracy in predicting MOOC student course trajectories.
GOAD improves anomaly detection across various data types.
problem Finding anomalies in diverse data types.
method GOAD combines classification and transformation-based methods.
result GOAD achieves state-of-the-art accuracy on multiple datasets.
The sub-Riemannian X-ray Transform on H-type groups: Fourier-Slice Theorems and Injectivity setsmath.DG
X-ray transform on H-type groups solved, revealing function injectivity.
problem Injectivity in sub-Riemannian geometry.
method Fourier Slice Theorem adapted to H-type groups.
result Integrable functions on H-type groups are uniquely determined by their integrals over geodesics.
Deep interpretable ensemblesstat.ML
Transforms ensemble predictions to maintain interpretability.
problem Loss of interpretability in deep ensembles.
method Proposes transformation ensembles that aggregate predictions while preserving interpretability.
result Transformation ensembles yield better predictions than individual models and maintain interpretability.
Improved SVMs learn from few samples with composition and multiple scales.
problem Learning with small sample sizes.
method Transformation-invariant SVMs with composition and locality at multiple scales.
result Kernels based on maximum similarity are positive definite and yield superior accuracy.