Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

55110164219 · Jun 202019922001200920172026
48 results for non-linear transformations

Unified framework for non-linear attention using modern Hopfield networks.

problem Improving transformer model's understanding of complex relationships and efficiency.
method Proposes an energy functional based on Modern Hopfield Networks (MNH) to unify linear and non-linear attention mechanisms.
result Context wells encapsulate contextual relationships among tokens, offering a richer representation of non-linear data.

Regularizes GAMs to improve interpretability by reducing concurvity.

problem Susceptibility of GAMs to concurvity reduces interpretability.
method Proposes a regularizer to penalize pairwise correlations of non-linearly transformed features.
result Improves interpretability and reduces concurvity without sacrificing prediction quality.

Paper converts deep networks to flat, equivalent kernel machines.

problem Capacity control and uniform convergence in deep learning.
method Push-forward transformation from deep networks to indefinite kernel machines.
result Flat network weights are Lp-norm regularized (0<p<1).

In this paper, we generalize the famous Hasimoto's transformation by showing that the dynamics of a closed unidimensional vortex filament embedded in a three-dimensional manifold of constant curvature gives rise under Hasimoto's transformation to the non-linear Schrodinger equation. We also give a natural interpretatio…

2012-04-24abs ↗pdf ↗

We introduce the Randomized Dependence Coefficient (RDC), a measure of non-linear dependence between random variables of arbitrary dimension based on the Hirschfeld-Gebelein-Rényi Maximum Correlation Coefficient. RDC is defined in terms of correlation of random non-linear copula projections; it is invariant with respec…

2013-04-29abs ↗pdf ↗

Transformers use a unique Hessian structure that differs from classical networks, affecting optimization.

problem Understanding the unique optimization landscape of Transformers.
method Theoretical Hessian analysis of a single self-attention layer in Transformers.
result Transformers have a highly non-linear Hessian structure, distinguishing them from classical networks.

This paper explains a mechanism called phase collapse that improves image classification accuracy.

problem Understanding the role of non-linearities and convolutional filters in image classification.
method Demonstrates phase collapse as a mechanism that eliminates spatial variability and linearly separates classes.
result Phase collapse improves classification accuracy, while thresholding operators degrade performance.

We present a signal representation framework called the sparse manifold transform that combines key ideas from sparse coding, manifold learning, and slow feature analysis. It turns non-linear transformations in the primary sensory signal space into linear interpolations in a representational embedding space while maint…

2018-06-23abs ↗pdf ↗

Paper investigates Lipschitz constants of self-attention modules in neural networks.

problem Lipschitz constants of self-attention modules in neural networks.
method Proved standard dot-product self-attention is not Lipschitz for unbounded input domain. Proposed L2 self-attention that is Lipschitz. Derived upper bound on L2 self-attention's Lipschitz constant.
result Proved standard self-attention is not Lipschitz for unbounded input domain and proposed an alternative L2 self-attention that is Lipschitz.

This paper investigates how transformers can learn to generalize to unseen examples in context.

problem Understanding how transformers can generalize to unseen examples in a prompt.
method Gradient descent analysis of one-layer multi-head transformers for in-context learning.
result The training loss for a one-layer multi-head transformer converges linearly to a global minimum, effectively learning ridge regression over basis functions.

The paper generalizes equivariant neural networks on homogeneous spaces to the non-linear setting.

problem Equivariant neural networks on homogeneous spaces.
method Deriving generalized steerability constraints for non-linear equivariant layers.
result The universality of the derived construction for non-linear equivariant layers.

Paper introduces non-linearity signature to measure deep neural network performance.

problem Difficulty in explaining performance differences among similar DNN architectures.
method Affine Optimal Transport mappings to measure non-linearity.
result Signature provides better understanding of DNN inner workings.

PatternLocal improves XAI for non-linear models by suppressing suppressor variables.

problem Suppressor variables cause false-positive feature attributions in non-linear models.
method PatternLocal uses locally linear surrogate models and transforms weights into a generative representation.
result PatternLocal reduces false-positive attributions and provides more reliable explanations.

Agents learn state ambiguity from non-linear sensor data using Gaussian approximations.

problem Learning state representation from non-linear sensor data.
method Second-order Taylor approximation of Gaussian distribution for non-linear measurement functions.
result Induces a preference for states based on inferability from observations.

This work optimizes reservoir computing models by linking recurrence and non-linear dynamics.

problem Understanding how recurrence and non-linear dynamics in cortical networks contribute to their function.
method Transformed time-continuous, recurrent dynamics into an effective feed-forward structure of linear and non-linear temporal kernels.
result Optimal time-series classifiers can be built from random reservoir networks, demonstrating significant performance gains.

The paper analyzes how market prices respond to information processing and non-linear dynamics.

problem Understanding how market prices change in response to information.
method Logistic Continuous Wavelet Transformation method applied to SP 500 market data.
result Identifies patterns in market dynamics and describes them using a new theory of reflexive communication.

ALAMO is a computational methodology for leaning algebraic functions from data. Given a data set, the approach begins by building a low-complexity, linear model composed of explicit non-linear transformations of the independent variables. Linear combinations of these non-linear transformations allow a linear model to b…

2017-05-31abs ↗pdf ↗

SL(N,C) is the phase space of the Poisson SU(N). We calculate explicitly the symplectic structure of SL(N,C), define an analogue of the Hamiltonian of the free motion on SU(N) and solve the corresponding equations of motion. Velocity is related to the momentum by a non-linear Legendre transformation.

1996-12-04abs ↗pdf ↗

One 'problem' with the 21st century world, particularly the economic and business worlds, is the phenomenal and increasing number of interconnections between economic agents (consumers, firms, banks, markets, national economies). This implies that such agents are all interacting and consequently giving raise to enormou…

2012-08-27abs ↗pdf ↗

We propose a novel method of introducing structure into existing machine learning techniques by developing structure-based similarity and distance measures. To learn structural information, low-dimensional structure of the data is captured by solving a non-linear, low-rank representation problem. We show that this low-…

2011-10-26abs ↗pdf ↗

In this note, we study non-linear gauge theories for principal bundles, where the structure group is replaced by a Lie groupoid. We follow the approach of Moerdijk-Mrcun and establish its relation with the existing physics literature. In particular, we derive a new formula for the gauge transformation which closely res…

2008-07-21abs ↗pdf ↗

Subspace clustering assumes that the data is sepa-rable into separate subspaces. Such a simple as-sumption, does not always hold. We assume that, even if the raw data is not separable into subspac-es, one can learn a representation (transform coef-ficients) such that the learnt representation is sep-arable into subspac…

2019-12-10abs ↗pdf ↗

The paper proposes a new variant of a decision tree, called an Extreme Learning Tree. It consists of an extremely random tree with non-linear data transformation, and a linear observer that provides predictions based on the leaf index where the data samples fall. The proposed method outperforms linear models on a bench…

2019-12-19abs ↗pdf ↗

Geometric Variational Inference improves efficiency in complex probability distributions.

problem Efficiently accessing information in non-linear and high-dimensional probability distributions.
method Geometric Variational Inference (geoVI) uses Riemannian geometry and the Fisher information metric to construct a coordinate transformation.
result geoVI provides a more efficient variational approximation by a normal distribution, demonstrated on various problems.

Deep structured models are widely used for tasks like semantic segmentation, where explicit correlations between variables provide important prior information which generally helps to reduce the data needs of deep nets. However, current deep structured models are restricted by oftentimes very local neighborhood structu…

2018-11-01abs ↗pdf ↗