Embedding graph nodes into a vector space can allow the use of machine learning to e.g. predict node classes, but the study of node embedding algorithms is immature compared to the natural language processing field because of a diverse nature of graphs. We examine the performance of node embedding algorithms with respe…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “embedding algorithms”
Algorithm calculates genus of embedded graphs on surfaces.
problem Finding minimal and maximal genus for graph embeddings.
method Constructing special branched coverings of the 2-sphere.
result Algorithm calculates orientable genus and maximal genus of graphs.
For leveled spatial graphs, we find a surface embedding that allows cellular embedding.
problem Finding a surface embedding for general spatial graphs is not always possible.
method Define leveled property, decompose graph into subgraphs, and construct surface.
result For leveled spatial graphs with a small number of levels, a surface can always be found.
This work optimizes induced correlation in joint graph embeddings.
problem Optimizing correlation across embedded networks in joint graph embeddings.
method Developed corr2Omni algorithm to estimate optimal Omnibus weights.
result corr2Omni algorithm improves inference fidelity compared to classical Omnibus construction.
New algorithm B++&C improves hierarchical clustering on large deep embedding datasets.
problem Scaling up hierarchical clustering to massive datasets of deep embeddings.
method Proposes B++&C algorithm for practical hierarchical clustering, introduces B2SAT&C for theoretical approximation.
result Achieves 5%/20% improvement on MW/CKMM objectives compared to classic methods.
BIGUE algorithm provides credible intervals for hyperbolic network embeddings.
problem Uncertainty in hyperbolic network embeddings.
method Markov chain Monte Carlo (MCMC) algorithm for Bayesian hyperbolic random graph model.
result Samples from the posterior distribution provide credible intervals for hyperbolic coordinates and network properties.
Study finds significant instability in node embeddings due to randomness.
problem Stability of node embeddings under random variations.
method Evaluated five node embedding algorithms (HOPE, LINE, node2vec, SDNE, GraphSAGE) on synthetic and empirical graphs.
result Significant instability in embedding spaces and downstream task accuracy.
Simple non-convex methods outperform others in ordinal embedding.
problem Finding efficient Euclidean representations of abstract items using triplet comparisons.
method Comprehensive empirical evaluation of existing algorithms and a new neural network approach.
result Simple, non-convex methods consistently outperform other algorithms.
Local Procrustes for Manifold Embedding: A Measure of Embedding Quality and Embedding Algorithmsstat.ML
We present the Procrustes measure, a novel measure based on Procrustes rotation that enables quantitative comparison of the output of manifold-based embedding algorithms (such as LLE (Roweis and Saul, 2000) and Isomap (Tenenbaum et al, 2000)). The measure also serves as a natural tool when choosing dimension-reduction …
New algorithm for RL using mean embeddings of return distributions.
problem Improving reinforcement learning algorithms for dynamic programming.
method Mean embeddings of return distributions, novel algorithms for RL.
result Asymptotic convergence and improved performance in deep RL.
Machine learning algorithms are optimized to model statistical properties of the training data. If the input data reflects stereotypes and biases of the broader society, then the output of the learning algorithm also captures these stereotypes. In this paper, we initiate the study of gender stereotypes in {\em word emb…
The Whitney embedding theorem gives an upper bound on the smallest embedding dimension of a manifold. If a data set lies on a manifold, a random projection into this reduced dimension will retain the manifold structure. Here we present an algorithm to find a projection that distorts the data as little as possible.
Task Embedded Coordinate Update: A Realizable Framework for Multivariate Non-convex Optimizationcs.LG
We in this paper propose a realizable framework TECU, which embeds task-specific strategies into update schemes of coordinate descent, for optimizing multivariate non-convex problems with coupled objective functions. On one hand, TECU is capable of improving algorithm efficiencies through embedding productive numerical…
Effective Dimension Adaptive Sketching Methods for Faster Regularized Least-Squares Optimizationcs.LG
New algorithm reduces sketching dimension to effective problem size.
problem Solving L2-regularized least-squares problems efficiently.
method Randomized algorithm using Gaussian and SRHT embeddings.
result Preserves convergence guarantees with reduced embedding dimension.
We introduce a new multi-dimensional nonlinear embedding -- Piecewise Flat Embedding (PFE) -- for image segmentation. Based on the theory of sparse signal recovery, piecewise flat embedding with diverse channels attempts to recover a piecewise constant image representation with sparse region boundaries and sparse clust…
Stochastic neighbor embedding (SNE) and related nonlinear manifold learning algorithms achieve high-quality low-dimensional representations of similarity data, but are notoriously slow to train. We propose a generic formulation of embedding algorithms that includes SNE and other existing algorithms, and study their rel…
The paper analyzes and proposes an algorithm for multi-modal nonlinear embeddings with theoretical performance bounds.
problem Generalizability of multi-modal nonlinear embeddings to unseen data.
method Theoretical analysis and a multi-modal nonlinear representation learning algorithm motivated by performance bounds.
result The proposed algorithm yields promising performance in multi-modal image classification and cross-modal image-text retrieval applications.
Contrastive embeddings improve neural architecture search performance.
problem Improving performance of neural architecture search algorithms.
method Contrastive learning to identify networks based on data Jacobians and produce embeddings.
result Traditional black-box optimization algorithms can reach state-of-the-art performance with contrastive embeddings.
Improved spectral clustering algorithm for better performance.
problem Improving the performance of spectral clustering algorithms.
method Developed a new performance guarantee under a weaker assumption and evaluated using a different spectral embedding map.
result Better performance guarantee under a weaker assumption and evaluation of a new spectral embedding map.
In this work, we present a method for node embedding in temporal graphs. We propose an algorithm that learns the evolution of a temporal graph's nodes and edges over time and incorporates this dynamics in a temporal node embedding framework for different graph prediction tasks. We present a joint loss function that cre…
Continuous vector representations of words and objects appear to carry surprisingly rich semantic content. In this paper, we advance both the conceptual and theoretical understanding of word embeddings in three ways. First, we ground embeddings in semantic spaces studied in cognitive-psychometric literature and introdu…
This paper proposes an out-of-sample extension framework for a global manifold learning algorithm (Isomap) that uses temporal information in out-of-sample points in order to make the embedding more robust to noise and artifacts. Given a set of noise-free training data and its embedding, the proposed framework extends t…
The recent proliferation of publicly available graph-structured data has sparked an interest in machine learning algorithms for graph data. Since most traditional machine learning algorithms assume data to be tabular, embedding algorithms for mapping graph data to real-valued vector spaces has become an active area of …
Improves hierarchical clustering in Euclidean space using autoencoders.
problem Lack of unsupervised methods for learning hierarchical structure in Euclidean space.
method Variational autoencoder with Gaussian mixture prior, rescaling latent space, and Ward's linkage.
result Improved dendrogram purity and Moseley-Wang cost function results.
Word2Vec (W2V) and GloVe are popular, fast and efficient word embedding algorithms. Their embeddings are widely used and perform well on a variety of natural language processing tasks. Moreover, W2V has recently been adopted in the field of graph embedding, where it underpins several leading algorithms. However, despit…
Label embedding (LE) is an important family of multi-label classification algorithms that digest the label information jointly for better performance. Different real-world applications evaluate performance by different cost functions of interest. Current LE algorithms often aim to optimize one specific cost function, b…
Combines PCA and CEM for fast clustering and embedding.
problem Dimensionality and slow EM algorithm convergence for clustering.
method Combines PCA and CEM for simultaneous clustering and data embedding.
result Demonstrates improved clustering and embedding performance.
A new method speeds up SoftMax normalization for embedding learning.
problem Efficiently learning distributed representations with SoftMax normalization.
method Proposes a linear-time heuristic approximation for , optimizing cross entropy.
result Achieves higher or comparable accuracy to existing methods with lower computational time.
New method learns state embeddings from demonstrations for improved reinforcement learning.
problem Difficult relationship between observed state and useful policy actions in dynamic problems.
method Variational framework for learning state embeddings that optimize trajectory linearity.
result Learning embedding spaces improves policy gradient reinforcement learning performance.
Many manifold embedding algorithms fail apparently when the data manifold has a large aspect ratio (such as a long, thin strip). Here, we formulate success and failure in terms of finding a smooth embedding, showing also that the problem is pervasive and more complex than previously recognized. Mathematically, success …
Local Linear embedding (LLE) is a popular dimension reduction method. In this paper, we first show LLE with nonnegative constraint is equivalent to the widely used Laplacian embedding. We further propose to iterate the two steps in LLE repeatedly to improve the results. Thirdly, we relax the kNN constraint of LLE and p…
Neighbor embeddings balance attraction and repulsion to visualize data.
problem Visualizing high-dimensional datasets with trade-offs between continuous and discrete structures.
method Neighbor embeddings combine attractive and repulsive forces to visualize data.
result Changing the exaggeration parameter in t-SNE yields a spectrum of embeddings with a trade-off between continuous and discrete structures.
Robustly computes intrinsic coordinates on point clouds using resampling and averaging.
problem Computing intrinsic coordinates on noisy or outlier-prone point clouds.
method Subsample data, vary hyperparameters, cluster candidate embeddings, identify representative embeddings, and average them using Procrustes analysis.
result Robust to noise and outliers, validated on synthetic and real data.
DeepWalk embeddings converge on SBM graphs, recovering cluster structure.
problem Theoretical guarantees for DeepWalk embeddings on complex graphs.
method Solving a nonconvex optimization problem using random walks.
result DeepWalk embeddings on SBM graphs recover cluster structure with high probability.
It is shown that given any link-manifold, there is an algorithm to decide if the manifold contains an embedded, essential planar surface; if it does, the algorithm will construct one. If a slope on the boundary of the link-manifold is given, there is an algorithm to determine if the slope bounds an embedded punctured-d…
This work improves understanding of dimension reduction algorithms and their probabilistic embeddings.
problem Improving theoretical understanding of non-linear dimension reduction algorithms.
method Analytical investigation of a generalized multidimensional scaling optimization problem.
result Probabilistic formulation of the problem leads to deterministic embeddings, contrary to standard implementations.
New algorithm for MDS with quasi-polynomial dependency on aspect ratio.
problem Finding an embedding that minimizes a specific objective function for given dissimilarities.
method A novel geometry-aware analysis of a conditional rounding of the Sherali-Adams LP hierarchy.
result Achieved a solution with cost \(O(\log Δ) \cdot extrm{OPT}^{Ω(1)} + ε\) in quasi-polynomial time.
As a human choosing a supervised learning algorithm, it is natural to begin by reading a text description of the dataset and documentation for the algorithms you might use. We demonstrate that the same idea improves the performance of automated machine learning methods. We use language embeddings from modern NLP to imp…
This paper tackles efficient optimization for nonlinear embeddings in similarity learning.
problem Learning similarity with nonlinear embeddings is challenging due to the large number of pairs.
method Detailed derivations and efficient optimization methods for nonlinear embeddings are developed.
result Efficient optimization methods for nonlinear embeddings are shown to be highly effective.
A framework for network embedding using VDS principles.
problem Challenges in systematically studying network embedding algorithms.
method Veridical Data Science (VDS) framework applied to network embedding.
result Potential for new research directions in network embedding.
Efficient ANN search for sparse embeddings in ads targeting.
problem Efficiently searching near neighbors in sparse data for applications like ads targeting.
method Graph-based ANN algorithms (HNSW, chi-square two-tower model, Sign Cauchy Projections).
result Sparse embeddings and ANN algorithms improve efficiency in EBR applications.
This paper proposes novel algorithms for speaker embedding using subjective inter-speaker similarity based on deep neural networks (DNNs). Although conventional DNN-based speaker embedding such as a -vector can be applied to multi-speaker modeling in speech synthesis, it does not correlate with the subjective inter-…
Efficiently computes embeddings for large graphs using coarsening.
problem Inefficient computation of graph embeddings for large-scale graphs.
method Graph coarsening based on Schur complements and Gaussian elimination.
result Efficiently computed embeddings on coarsened graph match Schur complement embeddings in expectation.
Graph embedding is a popular algorithmic approach for creating vector representations for individual vertices in networks. Training these algorithms at scale is important for creating embeddings that can be used for classification, ranking, recommendation and other common applications in industry. While industrial syst…
We demonstrate an equivalence between reproducing kernel Hilbert space (RKHS) embeddings of conditional distributions and vector-valued regressors. This connection introduces a natural regularized loss function which the RKHS embeddings minimise, providing an intuitive understanding of the embeddings and a justificatio…
This work benchmarks neural embeddings for link prediction in evolving knowledge graphs.
problem Evaluating the robustness of neural embeddings in changing knowledge graphs.
method Proposes an open-source evaluation pipeline using relation-centric connectivity measures.
result Demonstrates the importance of simulating embedding accuracy for frequently updated knowledge graphs.
We will describe some results regarding the algorithmic nature of homeomorphism problems for manifolds; in particular, the following theorem. Theorem 1: Every PL or smooth simply connected manifold M^n of dimension n at least 5 can be recognized among simply connected manifolds. That is, there is an algorithm to decide…
A novel method extracts topological features from word embeddings for text classification.
problem High dimensional and noisy text representations in natural language processing.
method Persistent homology for topological data analysis on word embeddings.
result Topological features outperform conventional text mining features on long textual documents.