Graph neural networks speed up nonnegative matrix factorization.
problem Efficiently factorize nonnegative matrices for various applications.
method Developed a graph neural network that combines bipartite self-attention with ADMM updates.
result Significant acceleration achieved in nonnegative matrix factorization.
Learning latent representations of nodes in graphs is an important and ubiquitous task with widespread applications such as link prediction, node classification, and graph visualization. Previous methods on graph representation learning mainly focus on static graphs, however, many real-world graphs are dynamic and evol…
Study explores properties of bipartite knots.
problem None explicitly stated; focuses on properties of bipartite knots.
method Exploration of combinatorial structure.
result Rich combinatorial structure of bipartite knots.
New method extends knot theory to non-bipartite knots, revealing PDs.
problem Extending knot theory to non-bipartite knots.
method Developed a new positive decomposition (PD) for HOMFLY polynomials of non-bipartite knots.
result PD exists for non-bipartite knots, not just bipartite ones.
Simplified Khovanov polynomials for bipartite links.
problem Computing Khovanov polynomials for bipartite links.
method Reduced Khovanov-Rozansky technique to Kauffman-Khovanov cycle calculus.
result Consistency demonstrated between reduced technique and bipartite Khovanov polynomials.
Bipartite networks are a common type of network data in which there are two types of vertices, and only vertices of different types can be connected. While bipartite networks exhibit community structure like their unipartite counterparts, existing approaches to bipartite community detection have drawbacks, including im…
GSA-Nets apply group equivariance to self-attention for vision tasks.
problem Improving self-attention networks for vision tasks.
method Define group-equivariant positional encodings.
result GSA-Nets outperform non-equivariant self-attention networks on vision benchmarks.
New model for detecting communities in weighted bipartite networks.
problem Lack of models for weighted bipartite networks.
method Introducing Bipartite Distribution-Free model and its extension.
result Spectral algorithms for consistent estimation of node labels.
Self-attention prefers sparse functions of input sequences, reducing sample complexity.
problem Understanding the inductive biases of self-attention in modeling long-range dependencies.
method Theoretical analysis and synthetic experiments to probe sample complexity of learning sparse functions with Transformers.
result Bounded-norm Transformer networks can represent sparse functions of the input sequence with logarithmic sample complexity.
Self-attention models benefit equally from width and depth, but beyond a certain point, depth becomes less efficient.
problem Understanding the optimal balance between depth and width in self-attention models.
method Theoretical predictions and empirical ablations on networks of varying depths and widths.
result An optimal width of 30K is recommended for a 1-Trillion parameter network, marking a significant width for self-attention models.
Study algebraic invariants from lightning self-attention models.
problem Understanding polynomial coefficients of self-attention mechanisms.
method Identify algebraic invariants using polynomial coefficients and coordinate geometry.
result Found linear and nonlinear families of algebraic invariants.
Random forests with attention and self-attention improve regression performance.
problem Improving regression model performance on various datasets.
method Proposes new models using attention and self-attention mechanisms to solve regression problems.
result The models improve model performance on many datasets.
Paper investigates Lipschitz constants of self-attention modules in neural networks.
problem Lipschitz constants of self-attention modules in neural networks.
method Proved standard dot-product self-attention is not Lipschitz for unbounded input domain. Proposed L2 self-attention that is Lipschitz. Derived upper bound on L2 self-attention's Lipschitz constant.
result Proved standard self-attention is not Lipschitz for unbounded input domain and proposed an alternative L2 self-attention that is Lipschitz.
New research finds six bipartite intrinsically knotted graphs with 23 edges.
problem Identifying intrinsically knotted bipartite graphs with 23 edges.
method Analyzing embeddings and graph minors to find minimal intrinsically knotted graphs.
result No minor minimal intrinsically knotted bipartite graph exists with 23 edges.
Simplified Khovanov-Rozansky calculus for bipartite knots.
problem Complexity in calculating superpolynomials for knots.
method Bipartite calculus generalizes Khovanov-Rozansky calculus for a restricted class of knots.
result Simplification of Khovanov-Rozansky polynomials for bipartite knots.
Proves Khovanov homology has no torsion for bipartite circle graphs.
problem Proving properties of Khovanov homology for bipartite circle graphs.
method Proved homotopy equivalence of independence complexes to wedges of spheres.
result Extreme Khovanov homology has no torsion.
The key to a Transformer model is the self-attention mechanism, which allows the model to analyze an entire sequence in a computationally efficient manner. Recent work has suggested the possibility that general attention mechanisms used by RNNs could be replaced by active-memory mechanisms. In this work, we evaluate wh…
Self attention mechanisms have become a key building block in many state-of-the-art language understanding models. In this paper, we show that the self attention operator can be formulated in terms of 1x1 convolution operations. Following this observation, we propose several novel operators: First, we introduce a 2D ve…
Improved bipartite link prediction using 2-hop paths.
problem Link prediction in bipartite networks without node attributes.
method Multiply reconstructed adjacency matrix with symmetrically normalized training adjacency matrix to form 2-hop paths.
result 2-hop paths improve link prediction performance.
New model for detecting communities in weighted bipartite networks.
problem No model for community detection in overlapping bipartite weighted networks.
method Introduces BiMMDF model allowing any distribution with block structure.
result Efficient algorithm with theoretical guarantee of consistent estimation.
Kernel PCA explains self-attention mechanisms in deep learning models.
problem Understanding and explaining self-attention mechanisms in deep learning models.
method Deriving self-attention from kernel principal component analysis (kernel PCA).
result RPC-Attention, a robust attention mechanism, outperforms softmax attention in various tasks.
Bipartite graphs have been used to represent data relationships in many data-mining applications such as in E-commerce recommendation systems. Since learning in graph space is more complicated than in Euclidian space, recent studies have extensively utilized neural nets to effectively and efficiently embed a graph's no…
Study reveals self-attention's role in learning and generalizing interactions.
problem Understanding self-attention's theoretical role in neural architectures.
method Interacting entities analysis, including multi-agent RL and genetic sequences.
result Self-attention efficiently represents, learns, and generalizes pairwise interactions.
A new method calculates HOMFLY-PT polynomials for bipartite links.
problem Computing HOMFLY-PT polynomials for bipartite links efficiently.
method Generalizes Goeritz matrix method for bipartite links.
result Reduces HOMFLY-PT polynomial calculation to matrix algebra.
Linformer reduces transformer complexity to linear, improving efficiency.
problem High cost of training and deploying large transformer models for long sequences.
method Approximates self-attention with low-rank matrix, proposing Linformer with O(n) complexity. result Linformer performs similarly to standard transformers but is more memory- and time-efficient.
We define integral odd Khovanov homology of principally unimodular bipartite graph-links.
Recent studies identified that sequential Recommendation is improved by the attention mechanism. By following this development, we propose Relation-Aware Kernelized Self-Attention (RKSA) adopting a self-attention mechanism of the Transformer with augmentation of a probabilistic model. The original self-attention of Tra…
Skeinformer accelerates self-attention for long sequences with linear complexity.
problem Efficiency of Transformer models in processing long sequences.
method Matrix sketching and column sampling to reduce quadratic complexity to linear.
result Skeinformer outperforms alternatives with smaller time/space footprint.
Paper connects MoE and self-attention, proposing active-attention.
problem Improving efficiency and performance of self-attention mechanisms.
method Established connection between MoE and self-attention, analyzed quadratic gating functions, proposed active-attention mechanism.
result Active-attention outperforms standard self-attention in various tasks.
PAC learning simplified as bipartite matching.
problem Efficiently solving PAC learning problems.
method Transductive learning and one-inclusion graphs.
result PAC learning can be reduced to bipartite matching.
Gradient descent converges geometrically to optimal self-attention parameters.
problem Training softmax self-attention layers for linear regression.
method Structure-aware gradient descent with preconditioner and regularizer.
result Gradient descent converges geometrically to global minima.
We present a simple combinatorial model for quasipositive surfaces and positive braids, based on embedded bipartite graphs. As a first application, we extend the well-known duality on standard diagrams of torus links to twisted torus links. We then introduce a combinatorial notion of adjacency for bipartite graph links…
We present evidence in support of a conjecture that a bipartite graph with at least five vertices in each part and |E(G)| \geq 4 |V(G)| - 17 is intrinsically knotted. We prove the conjecture for graphs that have exactly five or exactly six vertices in one part. We also show that there is a constant C_n such that a bipa…
Develops a new variational estimator for node popularity in bipartite networks.
problem Estimating node popularity in bipartite networks with varying patterns.
method Variational Expectation-Maximization (VEM) framework for the Two-Way Node Popularity Model (TNPM).
result The proposed method achieves superior estimation accuracy across different types of networks.
Improved text summarization using belief propagation on weighted bipartite graphs.
problem Text summarization from a graph theory perspective.
method Generalized belief propagation algorithm for weighted bipartite graphs.
result Our algorithm outperforms greedy methods in text summarization tasks.
New method for matching bipartite and unipartite graphs without collapsing.
problem Matching between bipartite and unipartite networks without losing information.
method Formulated as an undirected graphical model, aligns graphs without collapsing.
result Consistent method with conditions for exact recovery of matching solution.
Bipartite Riemann-Finsler geometries with complementary Finsler structures are constructed. Calculable examples are presented based on a bilinear-form coefficient for explicit Lorentz violation.
Incorrect parity-based descriptions of realizable Gauss diagrams found, but bipartite graphs provide a valid approach.
problem Incorrect descriptions of realizable Gauss diagrams using parity conditions.
method Used bipartite graphs to describe realizable Gauss diagrams.
result Realizable Gauss diagrams can be accurately described using bipartite graphs.
New method for calculating HOMFLY polynomials in symmetric representations.
problem Calculating HOMFLY polynomials for symmetric representations.
method Planar decomposition and projection to symmetric representations.
result Restoration of planarity and new insights into HOMFLY polynomials.
MCSAE improves speaker embedding by focusing on both high- and low-level features.
problem Reduced effect of low-level features in speaker embedding encoding.
method Masked cross self-attentive encoding using ResNet with multi-layer aggregation and random masking regularization.
result Improved speaker embedding with equal error rate of 2.63% and minimum detection cost function of 0.1453.
Transformers' self-attention mechanism is mapped to a generalized Potts model.
problem Uncertainty in what type of data distribution self-attention can efficiently learn.
method Decouple word positions and embeddings, then show self-attention learns a generalized Potts model.
result Training self-attention is equivalent to solving the inverse Potts problem.
A new model detects common patterns in pollination networks.
problem Comparing organization of bipartite networks to understand community structure.
method colBiSBM, a family of probabilistic models for collections of bipartite networks.
result The method uncovers shared ecological roles and partitions networks.
Model detects electricity theft with high accuracy.
problem Detecting electricity theft on imbalanced datasets.
method Multi-head self-attention mechanism with dilated convolutions and binary mask.
result Achieved AUC of 0.926, improving previous work by 17%.
We characterize which automorphisms of an arbitrary complete bipartite graph Kn,m can be induced by a homeomorphism of some embedding of the graph in S3.
New analysis shows how attention masks and LayerNorm prevent rank collapse in transformers.
problem Rank collapse in transformer models with increasing depth.
method General analysis of rank collapse under self-attention, considering attention masks and LayerNorm.
result Self-attention with LayerNorm can prevent rank collapse and maintain a rich set of equilibria.
Neural execution solves complex graph problems like bipartite matching.
problem Solving complex graph algorithms like maximum bipartite matching.
method Reduces bipartite matching to a flow problem and uses Ford-Fulkerson for maximum flow.
result Neural network achieves optimal matching almost 100% of the time.
A graph is intrinsically knotted if every embedding contains a knotted cycle. It is known that intrinsically knotted graphs have at least 21 edges and that the KS graphs, K7 and the 13 graphs obtained from K7 by ∇Y moves, are the only minor minimal intrinsically knotted graphs with 21 edges. This set incl…
Improved speech enhancement with MNTFA using time-frequency attention.
problem Speech enhancement with limited model size and memory.
method Designing MNTFA with self-attention modules for long sequences and joint training.
result MNTFA achieves better performance with fewer parameters than DPCRN.