Recent work has demonstrated that embeddings of tree-like graphs in hyperbolic space surpass their Euclidean counterparts in performance by a large margin. Inspired by these results and scale-free structure in the word co-occurrence graph, we present an algorithm for learning word embeddings in hyperbolic space from fr…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,982 papers · 148 categories
Trend · papers per month
48 results for “hyperbolic Skip-gram”
Tag2Vec learns tag representations in hybrid networks with semantic and hierarchical information.
problem Lack of semantic and hierarchical information in tag networks.
method Tag2Vec model that combines nodes and tags into hybrid networks, using parameterized random walks and hyperbolic Skip-gram model.
result Tag2Vec outperforms other models in learning rich semantic tag representations.
Corrected CBOW performs similarly to Skip-gram.
problem CBOW embeddings underperform Skip-gram embeddings in word2vec.
method Fixed a bug in CBOW gradient update to improve performance.
result Corrected CBOW embeddings are competitive with Skip-gram on various tasks.
We simplify word embeddings by removing sigmoid in SGNS, revealing connections to hyperbolic spaces.
problem Improving word embeddings quality and understanding their relationship with hyperbolic spaces.
method Analyzing squashed shifted PMI matrix and its relation to graph properties and hyperbolic geometry.
result Word embeddings can be connected to hyperbolic spaces through squashed shifted PMI matrix.
We show that the skip-gram formulation of word2vec trained with negative sampling is equivalent to a weighted logistic PCA. This connection allows us to better understand the objective, compare it to other word embedding methods, and extend it to higher dimensional models.
Generates low-dimensional node vectors for graphs with privacy while preserving structural preferences.
problem Publishing graph node vectors can leak sensitive individual information.
method SE-PrivGEmb, a skip-gram based technique with a unified noise tolerance mechanism and negative sampling probabilities.
result Our method outperforms existing methods in structural equivalence and link prediction tasks.
A new method learns dynamic graph representations from time-varying data.
problem Learning dynamic graph representations from time-varying data.
method Higher-order skip-gram with negative sampling (HOSGNS) for tensor factorization.
result HOSGNS outperforms state-of-the-art methods in downstream tasks.
Proposes SNML for selecting word2vec Skip-gram dimensionality.
problem Selecting optimal dimensionality for word2vec Skip-gram models.
method Information criteria (AIC, BIC, SNML) applied to SG and SG Negative Sampling models.
result SNML outperforms AIC and BIC, selecting closer optimal dimensionality.
This work shows dimension regularization can replace skip-gram negative sampling for graph embeddings, improving efficiency and performance.
problem Efficiently enforcing dissimilarity among node embeddings in graph learning.
method Dimension regularization as an alternative to skip-gram negative sampling.
result Dimension regularization is a more efficient approach to enforcing dissimilarity in graph embeddings.
This research improves graph embeddings by optimizing node sampling with centrality weights.
problem Improving the accuracy and efficiency of graph embeddings using Skip-Gram methods.
method Implemented and analyzed four graph embedding techniques with different centrality-weighted sampling distributions.
result Centrality-weighted sampling leads to improved accuracy and faster learning times.
Eliciting semantic similarity between concepts in the biomedical domain remains a challenging task. Recent approaches founded on embedding vectors have gained in popularity as they risen to efficiently capture semantic relationships The underlying idea is that two words that have close meaning gather similar contexts. …
Efficient algorithm for dynamic network embedding using skip-gram with negative sampling.
problem Accurate and efficient embedding in dynamic networks.
method Incremental skip-gram with negative sampling, partitioning network into updated and retained parts.
result The proposed algorithm reduces training time while preserving performance.
Since the invention of word2vec, the skip-gram model has significantly advanced the research of network embedding, such as the recent emergence of the DeepWalk, LINE, PTE, and node2vec approaches. In this work, we show that all of the aforementioned models with negative sampling can be unified into the matrix factoriza…
We revisit skip-gram negative sampling (SGNS), one of the most popular neural-network based approaches to learning distributed word representation. We first point out the ambiguity issue undermining the SGNS model, in the sense that the word vectors can be entirely distorted without changing the objective value. To res…
Embed nodes with multi-scale attributes for robust network analysis.
problem Capturing complex node attributes across different scales.
method Multi-scale attributed node embedding (AE & MUSAE) using Skip-gram approach.
result Proves node-feature mutual information is implicitly factorized by embeddings.
InfiniteWalk connects deep network embeddings to spectral graph theory with a nonlinear transformation.
problem Learning node representations from networks with deep learning methods.
method Study of the DeepWalk objective in the limit as window size goes to infinity, linking to spectral graph embeddings with a nonlinear transformation.
result Simple binary thresholding of the Laplacian pseudoinverse can approximate DeepWalk embeddings.
Dynamic Word Embeddingsstat.ML
We present a probabilistic language model for time-stamped text data which tracks the semantic evolution of individual words over time. The model represents words and contexts by latent trajectories in an embedding space. At each moment in time, the embedding vectors are inferred from a probabilistic version of word2ve…
Proposes ML-GCN for multi-label graph node classification using GCN and relaxed skip-gram model.
problem Loss of label correlations in multi-label graph node classification.
method Uses a GCN to embed node features and graph topology, generates random label vectors, and detects correlations using a skip-gram model.
result Significantly outperforms state-of-the-art methods on graph classification datasets.
SENSE enhances node sequences in graphs using vector embeddings.
problem Efficiently capturing graph node sequences for applications.
method SENSE-S learns node embeddings and composes them for sequences, preserving node order.
result SENSE-S increases multi-label classification and link-prediction accuracy by up to 50% and 78% respectively.
Word embeddings provide point representations of words containing useful semantic information. We introduce multimodal word distributions formed from Gaussian mixtures, for multiple word meanings, entailment, and rich uncertainty information. To learn these distributions, we propose an energy-based max-margin objective…
Deep neural network predicts product returns before purchase.
problem High costs of handling returned fashion products.
method Bayesian Personalized Ranking (BPR) embeddings and skip-gram model for user and product features.
result Reduced overall returns through real-time return probability prediction.
A scalable graph embedding system for large networks.
problem Training graph embeddings on large datasets efficiently.
method Distributed infrastructure avoiding graph partitioning, dynamic graph creation, efficient indexing.
result System scales skip-gram algorithm to Friendster and internal networks.
The Global Vectors for word representation (GloVe), introduced by Jeffrey Pennington et al. is reported to be an efficient and effective method for learning vector representations of words. State-of-the-art performance is also provided by skip-gram with negative-sampling (SGNS) implemented in the word2vec tool. In this…
Paper builds LETOR models for e-commerce, segmenting queries and optimizing search results.
problem Optimizing search results for e-commerce platforms.
method Segmenting queries into broad and narrow, using denoising auto-encoders and skip-gram embeddings, employing various feature types.
result Specialized models for broad and narrow queries outperform a combined model.
We describe a method for learning word embeddings with data-dependent dimensionality. Our Stochastic Dimensionality Skip-Gram (SD-SG) and Stochastic Dimensionality Continuous Bag-of-Words (SD-CBOW) are nonparametric analogs of Mikolov et al.'s (2013) well-known 'word2vec' models. Vector dimensionality is made dynamic b…
This paper presents a new framework for distributed training of embeddings using graph analytics.
problem Training embeddings is computationally intensive and challenging to distribute.
method Formulated Any2Vec training algorithm as a graph application, leveraging D-Galois for dynamic graph generation and repartitioning, and incorporating novel communication optimizations.
result GraphAny2Vec matches state-of-the-art shared-memory implementations in accuracy and provides significant speedups.
There is rising interest in vector-space word embeddings and their use in NLP, especially given recent methods for their fast estimation at very large scale. Nearly all this work, however, assumes a single vector per word type ignoring polysemy and thus jeopardizing their usefulness for downstream tasks. We present an …
The recently introduced continuous Skip-gram model is an efficient method for learning high-quality distributed vector representations that capture a large number of precise syntactic and semantic word relationships. In this paper we present several extensions that improve both the quality of the vectors and the traini…
Skip-gram with negative sampling, a popular variant of Word2vec originally designed and tuned to create word embeddings for Natural Language Processing, has been used to create item embeddings with successful applications in recommendation. While these fields do not share the same type of data, neither evaluate on the …
The paper introduces a new method for graph embedding using exponential family distributions.
problem Representing networks in a low dimensional latent space for various applications.
method Introduces the exponential family graph embedding model, generalizing random walk-based techniques to exponential family conditional distributions.
result The proposed techniques outperform existing methods in link prediction and node classification tasks.
Although the word-popularity based negative sampler has shown superb performance in the skip-gram model, the theoretical motivation behind oversampling popular (non-observed) words as negative samples is still not well understood. In this paper, we start from an investigation of the gradient vanishing issue in the skip…
Enhancing Network Embedding with Auxiliary Information: An Explicit Matrix Factorization Perspectivecs.SI
Recent advances in the field of network embedding have shown the low-dimensional network representation is playing a critical role in network analysis. However, most of the existing principles of network embedding do not incorporate auxiliary information such as content and labels of nodes flexibly. In this paper, we t…
Zipfian Whiteningcs.CL
PCA whitening weighted by Zipfian word frequencies improves task performance.
problem Skewed word embedding spaces in neural models.
method PCA whitening weighted by empirical word frequencies following Zipf's law.
result Significantly improves task performance, surpassing baselines.
Network embedding aims to learn the low-dimensional representations of vertexes in a network, while structure and inherent properties of the network is preserved. Existing network embedding works primarily focus on preserving the microscopic structure, such as the first- and second-order proximity of vertexes, while th…
Analyzes word vectors and co-occurrence statistics in NLP models.
problem Understanding biases in NLP models through co-occurrence statistics.
method Developed an analytic model of statistics learned by Word2Vec and GloVe, derived the first solution to Word2Vec's algorithm, and analyzed independence in co-occurrence models.
result Demonstrated a universal property of word vectors that can reveal biases in data before they are absorbed by DL models.
New hyperbolicity concepts expand manifold study.
problem Studying hyperbolicity on complex manifolds.
method Introducing sG-hyperbolicity, weakly p-Kähler hyperbolic structures, and pluriclosed star split hyperbolic metrics.
result Expands the class of divisorially hyperbolic manifolds.
SCE improves network embedding using sparsest cut for negative samples only.
problem Efficient unsupervised network embedding using negative samples.
method Inspired by sparsest cut, SCE uses Laplacian smoothing for node representations.
result SCE outperforms GraphSAGE, G2G, and DGI in accuracy and scalability.
The paper defines two types of hyperbolicity for complex manifolds and proves related results.
problem Defining and studying hyperbolicity for a broader class of complex manifolds.
method Introducing SKT hyperbolicity and Gauduchon hyperbolicity, proving results using SKT and Gauduchon metrics.
result Every SKT hyperbolic manifold is also Kobayashi/Brody hyperbolic and every Gauduchon hyperbolic manifold is divisorially hyperbolic.
New criteria for relative hyperbolicity in hierarchically hyperbolic spaces.
problem Characterizing relative hyperbolicity in hierarchically hyperbolic spaces.
method New formulation of relative hyperbolicity in terms of hierarchy structures, applied to graphs associated to surfaces.
result The separating curve graph of a surface is relatively hyperbolic when the surface has zero or two punctures.
We explore the potential of a popular distributional semantics vector space model, word2vec, for capturing meaningful relationships in ecological (complex polyphonic) music. More precisely, the skip-gram version of word2vec is used to model slices of music from a large corpus spanning eight musical genres. In this newl…
Introduces Möbius structures and hyperbolic ends for -surfaces in hyperbolic space.
problem Understanding -surfaces in hyperbolic space.
method Studies Möbius structures and hyperbolic ends.
result Applications to -surfaces in hyperbolic space.
Introduces hyperbolic generalized framed surfaces and their properties.
problem None explicitly stated; focuses on introducing new geometric objects.
method Generalization of hyperbolic framed surfaces and curves.
result Established conditions for a surface to be a hyperbolic generalized framed base surface and explored their singularities.
Deformations of Non-Kähler Hyperbolicity Notions and Modifications of Degenerate Balanced Manifoldsmath.CV
Study on new hyperbolicity notions for non-Kähler manifolds and their deformations.
problem Analyzing new hyperbolicity notions for non-Kähler complex manifolds.
method Introducing and analyzing two new notions of hyperbolicity for compact complex non-Kähler manifolds, and studying their behavior under smooth modifications.
result Established openness results for -HS hyperbolicity and -Kähler hyperbolicity under holomorphic deformations.
New formulas for hyperbolic mass using horospheres.
problem Mass calculation of asymptotically hyperbolic manifolds.
method Geometric formulas derived using coordinate horospheres.
result Improved rigidity results of hyperbolic space.
Abstract: Non-residually finite hyperbolic groups imply non-residually finite rigid hyperbolic groups.
problem Existence of non-residually finite hyperbolic groups
method Direct implication
result Existence of non-residually finite rigid hyperbolic groups
Study complex hyperbolic lattices and their relation to strict hyperbolization.
problem Understanding the relationship between complex hyperbolic lattices and strict hyperbolization.
method Analyzing the fundamental groups of complex hyperbolic manifolds and spaces arising from strict hyperbolization.
result Uniform lattices in PU(n,1) cannot be fundamental groups of Charney-Davis strict hyperbolizations when n ≥ 2.
We study hyperbolic bongles and find their volumes.
problem Characterizing and quantifying hyperbolic bongles.
method Provided necessary and sufficient conditions for hyperbolicity, calculated volumes, and established upper bounds.
result All balanced hyperbolic -bongles have the same volume and this volume is an upper bound for any hyperbolic -bongle.
Abstract reviews hyperbolic positive energy theorems.
problem Analyzing positive energy theorems for hyperbolic spaces.
method Review of existing literature on asymptotically hyperbolic manifolds.
result Summarizes positive energy theorems for hyperbolic spaces.