A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Token-adaptive FFN design improves LLM expressivity.
Two of the most popular modelling paradigms in computer vision are feed-forward neural networks (FFNs) and probabilistic graphical models (GMs). Various connections between the two have been studied in recent works, such as e.g. expressing mean-field based inference in a GM as an FFN. This paper establishes a new conne…
FMA-ETA predicts travel time using FFN with attention.
Spark Transformer achieves high sparsity in FFN and attention without sacrificing model quality.
The Transformer architecture is widely used in natural language processing. Despite its success, the design principle of the Transformer remains elusive. In this paper, we provide a novel perspective towards understanding the architecture: we show that the Transformer can be mathematically interpreted as a numerical Or…
We present a Statistical Mechanics (SM) model of deep neural networks, connecting the energy-based and the feed forward networks (FFN) approach. We infer that FFN can be understood as performing three basic steps: encoding, representation validation and propagation. From the meanfield solution of the model, we obtain a…
In this paper, we predict the likelihood of a player making a shot in basketball from multiagent trajectories. Previous approaches to similar problems center on hand-crafting features to capture domain specific knowledge. Although intuitive, recent work in deep learning has shown this approach is prone to missing impor…
FFN addresses spectral bias in neural value approximation, improving reinforcement learning performance.
Training very deep networks is an important open problem in machine learning. One of many difficulties is that the norm of the back-propagated error gradient can grow or decay exponentially. Here we show that training very deep feed-forward networks (FFNs) is not as difficult as previously thought. Unlike when back-pro…
Minimalist softmax attention learns constrained Boolean functions with supervision.
Tensor Neural Networks improve regression accuracy and efficiency.
Model projection transfers convolutional network properties to feedforward networks.
This work explains the structural origins of attention sinks in LLMs.
Weibull framework diagnoses transformer weight distributions, revealing distinct patterns across modules.