Graph neural networks speed up nonnegative matrix factorization.
problem Efficiently factorize nonnegative matrices for various applications.
method Developed a graph neural network that combines bipartite self-attention with ADMM updates.
result Significant acceleration achieved in nonnegative matrix factorization.
Learning latent representations of nodes in graphs is an important and ubiquitous task with widespread applications such as link prediction, node classification, and graph visualization. Previous methods on graph representation learning mainly focus on static graphs, however, many real-world graphs are dynamic and evol…
Study explores properties of bipartite knots.
problem None explicitly stated; focuses on properties of bipartite knots.
method Exploration of combinatorial structure.
result Rich combinatorial structure of bipartite knots.
New method extends knot theory to non-bipartite knots, revealing PDs.
problem Extending knot theory to non-bipartite knots.
method Developed a new positive decomposition (PD) for HOMFLY polynomials of non-bipartite knots.
result PD exists for non-bipartite knots, not just bipartite ones.
Simplified Khovanov polynomials for bipartite links.
problem Computing Khovanov polynomials for bipartite links.
method Reduced Khovanov-Rozansky technique to Kauffman-Khovanov cycle calculus.
result Consistency demonstrated between reduced technique and bipartite Khovanov polynomials.
Bipartite networks are a common type of network data in which there are two types of vertices, and only vertices of different types can be connected. While bipartite networks exhibit community structure like their unipartite counterparts, existing approaches to bipartite community detection have drawbacks, including im…
GSA-Nets apply group equivariance to self-attention for vision tasks.
problem Improving self-attention networks for vision tasks.
method Define group-equivariant positional encodings.
result GSA-Nets outperform non-equivariant self-attention networks on vision benchmarks.
New model for detecting communities in weighted bipartite networks.
problem Lack of models for weighted bipartite networks.
method Introducing Bipartite Distribution-Free model and its extension.
result Spectral algorithms for consistent estimation of node labels.
Paper proposes multiscale self-attentive convolutions for vision and language.
problem Improving language and vision understanding models using self-attention.
method Developed 1D and 2D Self Attentive Convolutions (SAC), multiscale SAC (MSAC).
result MSAC enhances model performance for vision and language tasks.
Active-memory mechanisms can replace self-attention in Transformers, but optimal results often require both.
problem Replacing self-attention with active-memory mechanisms in Transformers.
method Evaluation of various active-memory mechanisms in a Transformer model.
result Active-memory mechanisms can achieve comparable results to self-attention for language modeling, but optimal results are often achieved by combining both mechanisms.
Self-attention prefers sparse functions of input sequences, reducing sample complexity.
problem Understanding the inductive biases of self-attention in modeling long-range dependencies.
method Theoretical analysis and synthetic experiments to probe sample complexity of learning sparse functions with Transformers.
result Bounded-norm Transformer networks can represent sparse functions of the input sequence with logarithmic sample complexity.
Self-attention models benefit equally from width and depth, but beyond a certain point, depth becomes less efficient.
problem Understanding the optimal balance between depth and width in self-attention models.
method Theoretical predictions and empirical ablations on networks of varying depths and widths.
result An optimal width of 30K is recommended for a 1-Trillion parameter network, marking a significant width for self-attention models.
Improves sequential recommendation with relation-aware self-attention.
problem Improving accuracy in sequential recommendation.
method Integrates Transformer's self-attention mechanism with a probabilistic model of recommendation context.
result Significant improvements over recent baseline models.
Study algebraic invariants from lightning self-attention models.
problem Understanding polynomial coefficients of self-attention mechanisms.
method Identify algebraic invariants using polynomial coefficients and coordinate geometry.
result Found linear and nonlinear families of algebraic invariants.
Random forests with attention and self-attention improve regression performance.
problem Improving regression model performance on various datasets.
method Proposes new models using attention and self-attention mechanisms to solve regression problems.
result The models improve model performance on many datasets.
Paper investigates Lipschitz constants of self-attention modules in neural networks.
problem Lipschitz constants of self-attention modules in neural networks.
method Proved standard dot-product self-attention is not Lipschitz for unbounded input domain. Proposed L2 self-attention that is Lipschitz. Derived upper bound on L2 self-attention's Lipschitz constant.
result Proved standard self-attention is not Lipschitz for unbounded input domain and proposed an alternative L2 self-attention that is Lipschitz.
New research finds six bipartite intrinsically knotted graphs with 23 edges.
problem Identifying intrinsically knotted bipartite graphs with 23 edges.
method Analyzing embeddings and graph minors to find minimal intrinsically knotted graphs.
result No minor minimal intrinsically knotted bipartite graph exists with 23 edges.
Simplified Khovanov-Rozansky calculus for bipartite knots.
problem Complexity in calculating superpolynomials for knots.
method Bipartite calculus generalizes Khovanov-Rozansky calculus for a restricted class of knots.
result Simplification of Khovanov-Rozansky polynomials for bipartite knots.
Proves Khovanov homology has no torsion for bipartite circle graphs.
problem Proving properties of Khovanov homology for bipartite circle graphs.
method Proved homotopy equivalence of independence complexes to wedges of spheres.
result Extreme Khovanov homology has no torsion.
Improved bipartite link prediction using 2-hop paths.
problem Link prediction in bipartite networks without node attributes.
method Multiply reconstructed adjacency matrix with symmetrically normalized training adjacency matrix to form 2-hop paths.
result 2-hop paths improve link prediction performance.
New model for detecting communities in weighted bipartite networks.
problem No model for community detection in overlapping bipartite weighted networks.
method Introduces BiMMDF model allowing any distribution with block structure.
result Efficient algorithm with theoretical guarantee of consistent estimation.
Kernel PCA explains self-attention mechanisms in deep learning models.
problem Understanding and explaining self-attention mechanisms in deep learning models.
method Deriving self-attention from kernel principal component analysis (kernel PCA).
result RPC-Attention, a robust attention mechanism, outperforms softmax attention in various tasks.
Proposes a faster Transformer decoding method by truncating target-side self-attention windows.
problem Efficiency in Transformer decoding with minimal BLEU score loss.
method N-gram assumption to truncate target-side self-attention windows.
result N-gram masked self-attention model maintains BLEU score for N values from 4 to 8. Bipartite graphs have been used to represent data relationships in many data-mining applications such as in E-commerce recommendation systems. Since learning in graph space is more complicated than in Euclidian space, recent studies have extensively utilized neural nets to effectively and efficiently embed a graph's no…
Study reveals self-attention's role in learning and generalizing interactions.
problem Understanding self-attention's theoretical role in neural architectures.
method Interacting entities analysis, including multi-agent RL and genetic sequences.
result Self-attention efficiently represents, learns, and generalizes pairwise interactions.
A new method calculates HOMFLY-PT polynomials for bipartite links.
problem Computing HOMFLY-PT polynomials for bipartite links efficiently.
method Generalizes Goeritz matrix method for bipartite links.
result Reduces HOMFLY-PT polynomial calculation to matrix algebra.
Linformer reduces transformer complexity to linear, improving efficiency.
problem High cost of training and deploying large transformer models for long sequences.
method Approximates self-attention with low-rank matrix, proposing Linformer with O(n) complexity. result Linformer performs similarly to standard transformers but is more memory- and time-efficient.
We define integral odd Khovanov homology of principally unimodular bipartite graph-links.
A new SBM for bipartite networks improves community detection in noisy data.
problem Community detection in bipartite networks with stochastic blockmodels.
method Bayesian nonparametric formulation of SBM for bipartite networks, algorithm to find communities efficiently.
result Improves community detection results over general SBMs, especially in noisy data.
Skeinformer accelerates self-attention for long sequences with linear complexity.
problem Efficiency of Transformer models in processing long sequences.
method Matrix sketching and column sampling to reduce quadratic complexity to linear.
result Skeinformer outperforms alternatives with smaller time/space footprint.
Paper connects MoE and self-attention, proposing active-attention.
problem Improving efficiency and performance of self-attention mechanisms.
method Established connection between MoE and self-attention, analyzed quadratic gating functions, proposed active-attention mechanism.
result Active-attention outperforms standard self-attention in various tasks.
PAC learning simplified as bipartite matching.
problem Efficiently solving PAC learning problems.
method Transductive learning and one-inclusion graphs.
result PAC learning can be reduced to bipartite matching.
Gradient descent converges geometrically to optimal self-attention parameters.
problem Training softmax self-attention layers for linear regression.
method Structure-aware gradient descent with preconditioner and regularizer.
result Gradient descent converges geometrically to global minima.
We present a simple combinatorial model for quasipositive surfaces and positive braids, based on embedded bipartite graphs. As a first application, we extend the well-known duality on standard diagrams of torus links to twisted torus links. We then introduce a combinatorial notion of adjacency for bipartite graph links…
We present evidence in support of a conjecture that a bipartite graph with at least five vertices in each part and |E(G)| \geq 4 |V(G)| - 17 is intrinsically knotted. We prove the conjecture for graphs that have exactly five or exactly six vertices in one part. We also show that there is a constant C_n such that a bipa…
Develops a new variational estimator for node popularity in bipartite networks.
problem Estimating node popularity in bipartite networks with varying patterns.
method Variational Expectation-Maximization (VEM) framework for the Two-Way Node Popularity Model (TNPM).
result The proposed method achieves superior estimation accuracy across different types of networks.
Improved text summarization using belief propagation on weighted bipartite graphs.
problem Text summarization from a graph theory perspective.
method Generalized belief propagation algorithm for weighted bipartite graphs.
result Our algorithm outperforms greedy methods in text summarization tasks.
New method for matching bipartite and unipartite graphs without collapsing.
problem Matching between bipartite and unipartite networks without losing information.
method Formulated as an undirected graphical model, aligns graphs without collapsing.
result Consistent method with conditions for exact recovery of matching solution.
New method embeds time span into self-attention for better temporal pattern recognition.
problem Capturing temporal patterns in event sequences without recurrent networks.
method Functional time representation learning with Bochner's and Mercer's Theorems.
result Proposed methods outperform baseline models in various continuous-time event sequence prediction tasks.
Incorrect parity-based descriptions of realizable Gauss diagrams found, but bipartite graphs provide a valid approach.
problem Incorrect descriptions of realizable Gauss diagrams using parity conditions.
method Used bipartite graphs to describe realizable Gauss diagrams.
result Realizable Gauss diagrams can be accurately described using bipartite graphs.
Bipartite Riemann-Finsler geometries with complementary Finsler structures are constructed. Calculable examples are presented based on a bilinear-form coefficient for explicit Lorentz violation.
New method for calculating HOMFLY polynomials in symmetric representations.
problem Calculating HOMFLY polynomials for symmetric representations.
method Planar decomposition and projection to symmetric representations.
result Restoration of planarity and new insights into HOMFLY polynomials.
MCSAE improves speaker embedding by focusing on both high- and low-level features.
problem Reduced effect of low-level features in speaker embedding encoding.
method Masked cross self-attentive encoding using ResNet with multi-layer aggregation and random masking regularization.
result Improved speaker embedding with equal error rate of 2.63% and minimum detection cost function of 0.1453.
SANST uses self-attentive networks with spatial and temporal embeddings for better POI recommendations.
problem Next point-of-interest (POI) recommendation for users based on their history.
method SANST incorporates spatio-temporal patterns into self-attentive networks.
result SANST outperforms state-of-the-art models by up to 13.65% in nDCG@10.
Transformers' self-attention mechanism is mapped to a generalized Potts model.
problem Uncertainty in what type of data distribution self-attention can efficiently learn.
method Decouple word positions and embeddings, then show self-attention learns a generalized Potts model.
result Training self-attention is equivalent to solving the inverse Potts problem.
A new model detects common patterns in pollination networks.
problem Comparing organization of bipartite networks to understand community structure.
method colBiSBM, a family of probabilistic models for collections of bipartite networks.
result The method uncovers shared ecological roles and partitions networks.
Model detects electricity theft with high accuracy.
problem Detecting electricity theft on imbalanced datasets.
method Multi-head self-attention mechanism with dilated convolutions and binary mask.
result Achieved AUC of 0.926, improving previous work by 17%.
We characterize which automorphisms of an arbitrary complete bipartite graph Kn,m can be induced by a homeomorphism of some embedding of the graph in S3.