NetSMF efficiently embeds large networks by sparse matrix factorization.
problem Learning latent representations for large-scale networks efficiently.
method NetSMF leverages spectral sparsification to efficiently sparsify and factorize a dense matrix.
result NetSMF achieves high efficiency and effectiveness on large-scale networks.
A model for POI recommendation using relation embedding.
problem Challenges in POI recommendation due to sparse user-POI matrix and varying context.
method Translation-based relation embedding using Knowledge Graph Embedding techniques, combined matrix factorization framework.
result Demonstrates effectiveness of the proposed model on real-world datasets.
FREDE efficiently embeds graphs using linear space and guarantees quality.
problem Efficiently embedding graphs with quality guarantees and linear space complexity.
method FREDE combines matrix sketching with a nonlinear transform of PageRank similarities to achieve linear space and quality guarantees.
result FREDE provides column-covariance approximation guarantees that are nearly as good as SVD, even with limited node similarities.
Predict missing movie ratings or graph embeddings with low rank matrices.
problem Predicting missing entries in a ratings matrix or graph embeddings with known linear relations.
method Low rank matrix completion approach applied to graph embeddings.
result Effective methods for predicting missing entries in matrices and graph embeddings.
Paper studies efficient regression for dynamic graphs.
problem Regression over dynamic graphs.
method Update-efficient matrix embedding, O(nm) time complexity for updates.
result Exact optimal solution can be updated efficiently in dynamic graphs.
We simplify word embeddings by removing sigmoid in SGNS, revealing connections to hyperbolic spaces.
problem Improving word embeddings quality and understanding their relationship with hyperbolic spaces.
method Analyzing squashed shifted PMI matrix and its relation to graph properties and hyperbolic geometry.
result Word embeddings can be connected to hyperbolic spaces through squashed shifted PMI matrix.
Paper proposes a matrix optimization model for reliable Euclidean embedding from noisy data.
problem Challenges in Euclidean embedding from noisy observations containing outliers.
method Matrix optimization based embedding model to detect and remove outliers.
result The model provides high accuracy estimators and successfully identifies outliers.
Sparse OSEs achieve optimal embedding dimension of O(d).
problem Achieving optimal embedding dimension for sparse OSEs.
method Random sparsified matrix with m≥(1+θ)d non-zeros per column. result Sparse OSEs can achieve embedding dimension m=O(d), improving on previous m=O(dlog(d)). Proposes DNN-based speaker embedding correlated with subjective inter-speaker similarity for speech synthesis.
problem Inadequate speaker representation for open speakers not in training data.
method Two training algorithms using inter-speaker similarity matrices: similarity vector embedding and similarity matrix embedding.
result Proposed algorithms learn speaker embedding highly correlated with subjective inter-speaker similarity.
Paper combines matrix factorization and random walk models for node embeddings.
problem Learning node representations in graphs for applications like link prediction and node classification.
method Proposes a weighted matrix factorization model using kernel functions to incorporate random walk-based information.
result The proposed model outperforms existing node embedding algorithms in link prediction and node classification tasks.
Saec compresses recommendation system embeddings by clustering similar features.
problem Large embedding matrix in recommendation systems consumes excessive memory.
method Saec clusters similar features within a field to reduce embedding matrix size.
result Saec reduces embedding size by ~27x with no performance loss.
As opposed to manual feature engineering which is tedious and difficult to scale, network representation learning has attracted a surge of research interests as it automates the process of feature learning on graphs. The learned low-dimensional node vector representation is generalizable and eases the knowledge discove…
Recent advances in the field of network embedding have shown the low-dimensional network representation is playing a critical role in network analysis. However, most of the existing principles of network embedding do not incorporate auxiliary information such as content and labels of nodes flexibly. In this paper, we t…
Unified framework for hyperbolic embeddings from mixed data types.
problem Computing hyperbolic embeddings from noisy metric and non-metric data.
method Semidefinite programming and spectral factorization methods.
result Efficient computation of hyperbolic embeddings from arbitrary data.
Two new methods improve graph embedding without needing a complete graph structure.
problem Graph autoencoders' performance depends on the adjacency matrix quality.
method BAGE and VBAGE: unsupervised graph embedding via adaptive graph learning.
result The methods expand GAEs' applicability to datasets without graph structure.
Improves scalability and robustness of dynamic graph clustering.
problem Scalability and robustness issues in matrix factorization methods for dynamic graphs.
method Temporal separated matrix factorization, bi-clustering regularization, selective embedding updating.
result Demonstrated scalability, robustness, and effectiveness on synthetic and real-world benchmarks.
Enhancing spectral embedding for low-dimensional embeddings in rare disease cohorts
problem Representing clinical concepts and patients in electronic health records
method Spectral-based unsupervised learning with flexible knowledge transfer
result Outperforms competing approaches in challenging scenarios
We prove optimal subspace embedding conjecture up to sub-polylogarithmic factors.
problem Optimal dimension and sparsity of subspace embeddings.
method Iterative decoupling technique to analyze higher-order trace moment bounds.
result Sub-polylogarithmic factors in dimension and sparsity of subspace embeddings.
This work optimizes induced correlation in joint graph embeddings.
problem Optimizing correlation across embedded networks in joint graph embeddings.
method Developed corr2Omni algorithm to estimate optimal Omnibus weights.
result corr2Omni algorithm improves inference fidelity compared to classical Omnibus construction.
Expected centre of mass for random embeddings is constant.
problem Understanding the expected centre of mass for random embeddings.
method Analyzing the Haar measure and Gaussian unitary ensemble on SL(N, C).
result The expectation of the centre of mass is a constant multiple of the identity matrix.
This work analyzes PPR-based node embeddings and their topological information.
problem Understanding and interpreting PPR-based node embeddings.
method Unified framework and two methods for topology recovery.
result PPR-based embeddings maintain more topological information than random walk-based embeddings.
New post-processing methods improve word embedding performance.
problem Boosting the performance of word embeddings for similarity and analogy tasks.
method Optimizing a semi-Riemannian manifold with Centralised Kernel Alignment (CKA) to shrink the covariance matrix towards a scaled identity matrix.
result Improved performance on downstream tasks after smoothing the spectrum of word vectors.
Enhanced EEG classification using augmented covariance matrix.
problem Improving motor imagery classification from EEG signals.
method Proposes a new framework based on the augmented covariance matrix derived from an autoregressive model.
result The augmented covariance matrix outperformed state-of-the-art methods.
Linear representations help embed manifolds into matrix spaces.
problem Embedding manifolds into matrix spaces with effective bounds.
method Defining linear representations of G-manifolds as maps into matrix spaces, encoding G-actions as matrix products. result Explicit bounds for Mostow-Palais G-equivariant embeddings of G-manifolds into G-modules V, showing dimV<∞ for compact G. The paper shows context vectors are half the dimensions of word vectors.
problem Understanding the relationship between word and context embeddings.
method Starting from probabilistic assumptions, the paper shows context vectors are reflections of word vectors in half the dimensions.
result Context vectors are reflections of word vectors in approximately half the dimensions.
Estimates low-rank distributional matrices from incomplete samples.
problem Matrix completion for distributional entries with limited observed data.
method Kernel mean embeddings, Tucker rank, functional unfolding operators.
result Effective estimator for distributional matrix completion established.
Since the invention of word2vec, the skip-gram model has significantly advanced the research of network embedding, such as the recent emergence of the DeepWalk, LINE, PTE, and node2vec approaches. In this work, we show that all of the aforementioned models with negative sampling can be unified into the matrix factoriza…
Method reduces categorical data to lower dimensions using density matrices.
problem Dimensionality reduction for categorical data.
method Density-matrix construction from class-conditional frequencies; spectral embedding.
result Low-dimensional spectral embeddings with controlled rank.
New GCNs solve graph embedding problems efficiently and interpretably.
problem Graph embedding for scalable and interpretable machine learning.
method Proposed two GCNs: CAFE-GCN and sphere-GCN, based on constrained optimization.
result Both GCNs yield good approximations of dominant eigenvectors and perform dimensionality reduction.
Two algorithms estimate Wasserstein distance matrices from few entries for manifold learning.
problem Estimating Wasserstein distance matrices from limited data for manifold learning.
method Proposes two algorithms: matrix completion and Nyström completion for square Wasserstein matrices.
result Nyström completion can outperform matrix completion with a fixed sample budget and improve classification stability.
MuRP embeds multi-relational graphs in hyperbolic space for better hierarchical representation.
problem Current hyperbolic models struggle with multi-relational knowledge graphs that exhibit multiple hierarchies.
method MuRP embeds multi-relational graph data in the Poincaré ball model of hyperbolic space, learning relation-specific parameters for entity embeddings.
result MuRP embeddings outperform Euclidean counterparts and other methods on link prediction tasks, especially at lower dimensions.
We prove a central limit theorem for the components of the eigenvectors corresponding to the d largest eigenvalues of the normalized Laplacian matrix of a finite dimensional random dot product graph. As a corollary, we show that for stochastic blockmodel graphs, the rows of the spectral embedding of the normalized La…
Optimal subspace embedding with near-optimal sparsity for high-dimensional data.
problem Efficiently preserving norms of vectors in high-dimensional subspaces.
method Near-optimal sparsity oblivious subspace embedding with decoupling argument and cumulant method.
result Achieved near-optimal sparsity of O~(1/ε) non-zeros per column. Survey of Laplacian-based methods for data dimensionality reduction and embedding.
problem Efficiently reducing high-dimensional data to lower dimensions while preserving important features and structures.
method Laplacian-based methods including spectral clustering, Laplacian eigenmap, locality preserving projection, graph embedding, and diffusion map.
result Comprehensive overview of various optimization variants and applications of Laplacian-based techniques.
We prove that many aspects of the differential geometry of embedded Riemannian manifolds can be formulated in terms of multi linear algebraic structures on the space of smooth functions. In particular, we find algebraic expressions for Weingarten's formula, the Ricci curvature and the Codazzi-Mainardi equations. For ma…
Manifold learning and dimensionality reduction techniques are ubiquitous in science and engineering, but can be computationally expensive procedures when applied to large data sets or when similarities are expensive to compute. To date, little work has been done to investigate the tradeoff between computational resourc…
The paper shows how to recover true node positions from a graph or similarity matrix.
problem Recovering true distances and positions from a graph or similarity matrix.
method Two steps: matrix factorisation followed by nonlinear dimension reduction.
result Nonlinear dimension reduction can recover latent positions close to a manifold where geodesic distance is encoded.
The paper analyzes DeepWalk and node2vec for community detection in stochastic blockmodels.
problem Theoretical understanding of community detection in stochastic blockmodels.
method Matrix factorization perspective on DeepWalk and node2vec.
result Node2vec and DeepWalk can perfectly recover community memberships in sparse graphs.
This paper addresses the problem of low-rank distance matrix completion. This problem amounts to recover the missing entries of a distance matrix when the dimension of the data embedding space is possibly unknown but small compared to the number of considered data points. The focus is on high-dimensional problems. We r…
Paper introduces S-SSE for stable sparse subspace embedding.
problem Inefficient sparse random projection matrices with uneven non-zero distribution.
method Uses uniform sampling without replacement to create a stable sparse subspace embedded matrix (S-SSE).
result S-SSE maintains Euclidean distance better after dimension reduction.
New method transfers word embeddings from large to small datasets efficiently.
problem Challenges in learning word embeddings from new domains with limited data.
method Group-sparse matrix factorization for transfer learning.
result Efficiently learns domain-specific word embeddings with less data.
A framework for stable dynamic network embeddings using static methods.
problem Dynamic network embedding in a nascent field.
method Using static network embedding methods on dilated unfolded adjacency matrices.
result Stable embeddings that preserve latent node behavior across time.
The paper tackles transfer learning for growing matrix representations, improving estimation accuracy.
problem Structured matrix estimation under growing ambient dimensions and latent representations.
method Proposes a general transfer framework decomposing target parameters into embedded source components, low-rank innovations, and sparse edits. Develops an anchored alternating projection estimator.
result Establishes deterministic error bounds that separate target noise, representation growth, and source estimation error, yielding improved rates.
Graph neural networks refine speaker embeddings for better session-level diarization.
problem Local speaker distinction in meeting sessions using deep embeddings.
method Graph Neural Networks (GNNs) refine speaker embeddings using session-level structural information.
result Spectral clustering on refined embeddings outperforms original embeddings significantly.
New method uses MHN for associative learning in network embedding.
problem Represent nodes in networks as low-dimensional vectors while incorporating topological and structural information.
method Introduces Modern Hopfield Networks (MHN) for associative learning between node content and neighbors.
result Competitive performance on node classification and linkage prediction tasks.
Unified model for signed networks separates balance and anomaly effects.
problem Ignoring sign information in signed networks leads to inaccurate analysis.
method Low rank plus sparse matrix decomposition with regularized formulation.
result The model accurately detects communities and anomalies in signed networks.
A new method uses matrix sketches for efficient graph clustering in dynamic environments.
problem Efficiently clustering large, dynamic graphs in distributed memory systems.
method Inspired by spectral clustering, the approach uses random dimension-reducing projections to derive matrix sketches.
result The method produces embeddings that yield performant clustering results in a fully-dynamic stochastic block model stream.
We introduce Gaussian Process Topic Models (GPTMs), a new family of topic models which can leverage a kernel among documents while extracting correlated topics. GPTMs can be considered a systematic generalization of the Correlated Topic Models (CTMs) using ideas from Gaussian Process (GP) based embedding. Since GPTMs w…