arXiv research

The paper explores how sinks and diagonal patterns prevent attention oversmoothing.

This work explains the structural origins of attention sinks in LLMs.

Multinational corporations use highly complex structures of parents and subsidiaries to organize their operations and ownership. Offshore Financial Centers (OFCs) facilitate these structures through low taxation and lenient regulation, but are increasingly under scrutiny, for instance for enabling tax avoidance. Theref…

New proof of injectivity for broken non-abelian X-ray transform in Minkowski space.

This paper surveys various methods for dimensionality reduction and nearest neighbor search.

A new method reduces the computational cost of Sinkhorn algorithm for OT and UOT problems.

An efficient method to price bonds with optional sinking feature is presented. Such instruments equip their issuer with the option (but not the obligation) to redeem parts of the notional prior to maturity, therefore the future cash flows are random. In a one-factor model for the issuer's default intensity we show that…

We define a laminar branched surface to be a branched surface satisfying the following conditions: (1) Its horizontal boundary is incompressible; (2) there is no monogon; (3) there is no Reeb component; (4) there is no sink disk (after eliminating trivial bubbles in the branched surface). The first three conditions are…

Proves analyticity of quasinormal modes in Kerr and Kerr-de Sitter spacetimes.

Randomly initialized transformers show extreme token preferences.

Gradient flow in softmax models tends to produce low-entropy outputs.

Graph Convolutional Networks (GCNs) have gained significant developments in representation learning on graphs. However, current GCNs suffer from two common challenges: 1) GCNs are only effective with shallow structures; stacking multiple GCN layers will lead to over-smoothing. 2) GCNs do not scale well with large, dens…

This study analyzes information flow networks in Chinese stock sectors using transfer entropy.

A graph (digraph) $G=(V,E)$ with a set $T\subseteq V$ of terminals is called inner Eulerian if each nonterminal node $v$ has even degree (resp. the numbers of edges entering and leaving $v$ are equal). Cherkassky and Lovász showed that the maximum number of pairwise edge-disjoint $T$-paths in an inner Eulerian graph $G…

The Poincare-Hopf theorem tells us that given a smooth, structurally stable vector field on a surface of genus g, the number of saddles is 2-2g less than the number of sinks and sources. We generalize this result by introducing a more complex combinatorial invariant. Using this tool, we demonstrate that many such struc…

Despite their successes, what makes kernel methods difficult to use in many large scale problems is the fact that storing and computing the decision function is typically expensive, especially at prediction time. In this paper, we overcome this difficulty by proposing Fastfood, an approximation that accelerates such co…

New Alexander polynomial defined for transverse graphs.

The paper studies grid homology for spatial graphs and proves a Künneth formula for connected sums.

We present a scalable Gaussian process model for identifying and characterizing smooth multidimensional changepoints, and automatically learning changes in expressive covariance structure. We use Random Kitchen Sink features to flexibly define a change surface in combination with expressive spectral mixture kernels to …

Paper introduces metrics for evaluating multi-agent policies using best response dynamics.

Irrespective of the fact that Machine learning has produced groundbreaking results, it demands an enormous amount of data in order to perform so. Even though data production has been in its all-time high, almost all the data is unlabelled, hence making them unsuitable for training the algorithms. This paper proposes a …

A challenging problem in complex networks is the network reconstruction problem from data. This work deals with a class of networks denoted as conserved networks, in which a flow associated with every edge and the flows are conserved at all non-source and non-sink nodes. We propose a novel polynomial time algorithm to …

We define sink marks for branched complexes and find conditions for them to determine a branched surface structure. These will be used to construct branched surfaces in knot and tangle complements. We will extend Delman's theorem and prove that a Montesinos knot $K$ of length at least 3 has a persistently laminar branc…

The paper uses topological concepts to analyze neural networks, revealing complex structure and dynamics.

Study compares information flow between Chinese and US stock sectors.

Classifies Morse flows on 3-sphere with specific saddle connections.

GOAT improves attention mechanisms by learning better priors.

We build a model using Gaussian processes to infer a spatio-temporal vector field from observed agent trajectories. Significant landmarks or influence points in agent surroundings are jointly derived through vector calculus operations that indicate presence of sources and sinks. We evaluate these influence points by us…

BSG learns dynamic network spillovers and uncertainty quantification.

Economy is demanding new models, able to understand and predict the evolution of markets. To this respect, Econophysics is offering models of markets as complex systems, such as the gas-like model, able to predict money distributions observed in real economies. However, this model reveals some technical hitches to expl…

In this article we will show that the Macro-Economy and its growth can be modelled and explained exactly in principle by commonly known Field Theory from theoretical physics. We will show the main concepts and calculations needed and show that calculation and prediction of economic growth then gets indeed possible in D…

We propose a novel method that makes use of deep neural networks and gradient decent to perform automated design on complex real world engineering tasks. Our approach works by training a neural network to mimic the fitness function of a design optimization task and then, using the differential nature of the neural netw…

The paper proves the existence of a horizon for certain spacetimes.

McKernel introduces a framework to use kernel approximates in the mini-batch setting with Stochastic Gradient Descent (SGD) as an alternative to Deep Learning. Based on Random Kitchen Sinks [Rahimi and Recht 2007], we provide a C++ library for Large-scale Machine Learning. It contains a CPU optimized implementation of …

This is a sequel to [arXiv:1708.09092v2]. For an oriented trivalent graph $G$ without source or sink embedded in $S^3$, we prove that the $\mathfrak{gl}(1| 1)$-Alexander polynomial $\underlineΔ(G, c)$ defined by Viro satisfies a series of relations, which we call MOY-type relations in [arXiv:1708.09092v2]. As a corolla…

Study on uniquely determining thermal properties from boundary temperature and heat flux measurements.

Crypto markets show negative spillovers between chains, not positive co-movements.

Kernels are powerful and versatile tools in machine learning and statistics. Although the notion of universal kernels and characteristic kernels has been studied, kernel selection still greatly influences the empirical performance. While learning the kernel in a data driven way has been investigated, in this paper we e…

It is common for CCTV operators to overlook inter- esting events taking place within the crowd due to large number of people in the crowded scene (i.e. marathon, rally). Thus, there is a dire need to automate the detection of salient crowd regions acquiring immediate attention for a more effective and proactive surveil…

Poisson learning improves graph-based semi-supervised learning at very low label rates.

Bayesian learning has been recently considered as an effective means of accounting for uncertainty in trained deep network parameters. This is of crucial importance when dealing with small or sparse training datasets. On the other hand, shallow models that compute weighted sums of their inputs, after passing them throu…

New metrics improve scRNA-seq perturbation modeling by reducing mode collapse.

This dissertation uses ILP to learn Bayesian network structures efficiently.

We consider the class non-surjective irreducible endomorphisms of the free group $F_n$. We show that such an endomorphism $φ$ is topologically represented by a simplicial immersion $f:G \rightarrow G$ of a marked graph $G$; along the way we classify the dynamics of $\partial φ$ acting on $\partial F_n$: there are at mo…

A method for authorship attribution based on function word adjacency networks (WANs) is introduced. Function words are parts of speech that express grammatical relationships between other words but do not carry lexical meaning on their own. In the WANs in this paper, nodes are function words and directed edges stand in…

The study analyzes numerical stability in large language models using mixed-precision arithmetic.

Study trace systoles on surfaces, finding optimal bounds and implications.

Wealth redistribution through Fokker-Planck equation controls preserves Gini coefficient.