We introduce Probabilistic FastText, a new model for word embeddings that can capture multiple word senses, sub-word structure, and uncertainty information. In particular, we represent each word with a Gaussian mixture density, where the mean of a mixture component is given by the sum of n-grams. This representation al…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “FastText Embedding”
Improved Naive Bayes classifier with neural network models.
problem Limited complexity handling and independence assumption in Naive Bayes.
method Introducing Neural Naive Bayes and Neural Pooled Markov Chain models.
result Error rate reduced by 4.5 on IMDB dataset.
This paper shows that a simple baseline based on a Bag-of-Words (BoW) representation learns surprisingly good knowledge graph embeddings. By casting knowledge base completion and question answering as supervised classification problems, we observe that modeling co-occurences of entities and relations leads to state-of-…
Key features of mental illnesses are reflected in speech. Our research focuses on designing a multimodal deep learning structure that automatically extracts salient features from recorded speech samples for predicting various mental disorders including depression, bipolar, and schizophrenia. We adopt a variety of pre-t…
Paper uses JIVE to decompose word embeddings, improving sentiment analysis performance.
problem Improving sentiment analysis performance on word embeddings.
method Joint and individual variance explained (JIVE) method for decomposition.
result Mapping word embeddings into joint components improves sentiment analysis performance.
Accurately learning from user data while providing quantifiable privacy guarantees provides an opportunity to build better ML models while maintaining user trust. This paper presents a formal approach to carrying out privacy preserving text perturbation using the notion of dx-privacy designed to achieve geo-indistingui…
qDKT improves KT models by considering individual question outcomes.
problem Ignoring question-specific outcomes leads to inaccurate learner skill tracking.
method qDKT incorporates graph Laplacian regularization and uses a fastText-inspired initialization scheme.
result qDKT achieves state-of-the-art performance in predicting learner outcomes.
Electronic health record (EHR) systems are used extensively throughout the healthcare domain. However, data interchangeability between EHR systems is limited due to the use of different coding standards across systems. Existing methods of mapping coding standards based on manual human experts mapping, dictionary mappin…
Paper presents a model for identifying informative COVID-19 tweets.
problem Identifying informative COVID-19 tweets on Twitter.
method Leveraged transformers (RoBERTa, XLNet, BERTweet) trained in Semi-Supervised Learning (SSL) setting.
result Achieved F1 score of 0.9011 on test set, ranking 7th on leaderboard.
Question-answering systems and voice assistants are becoming major part of client service departments of many organizations, helping them to reduce the labor costs of staff. In many such systems, there is always natural language understanding module that solves intent classification task. This task is complicated becau…
Extracts high-quality monolingual datasets from web crawl data.
problem Improving text representation quality through larger corpora.
method Automated pipeline using deduplication and language identification, augmented with filtering for high-quality documents.
result Extracted massive high-quality monolingual datasets from Common Crawl.
Paper tackles toxic comment detection using deep learning.
problem Automatic detection of toxic comments on the internet.
method Designs binary classification and regression-based approaches using DNN.
result BERT fine-tuning outperforms other methods.
Visual question answering is a recently proposed artificial intelligence task that requires a deep understanding of both images and texts. In deep learning, images are typically modeled through convolutional neural networks, and texts are typically modeled through recurrent neural networks. While the requirement for mo…
The paper uses geometry to assess how hard examples are for NLP models.
problem Challenges in NLP datasets and classifiers, especially with shallow features.
method Information geometry to quantify example difficulty, exploring BERT, CNN, and fasttext.
result Deep learning models are vulnerable to word substitutions in difficult examples.
Embeddings are ubiquitous in machine learning, appearing in recommender systems, NLP, and many other applications. Researchers and developers often need to explore the properties of a specific embedding, and one way to analyze embeddings is to visualize them. We present the Embedding Projector, a tool for interactive v…
Proposes QQE for transforming and embedding data distributions.
problem Transforming and embedding data distributions for better representation or visualization.
method Quantile-Quantile Embedding (QQE) using quantile-quantile plot concept.
result QQE allows for better discrimination of classes in some cases.
Maps can be embedded in higher dimensions if they lift to embeddings in product spaces.
problem Embedding maps in higher dimensions without self-intersections.
method Lifting maps to embeddings in product spaces.
result Maps can be embedded in higher dimensions if they lift to embeddings in product spaces.
New embeddings for manifolds using heat kernels.
problem Constructing canonical conformal embeddings for manifolds.
method Employing heat kernel embedding from Bérard-Besson-Gallot'94 to find canonical conformal embeddings.
result Intrinsic construction of canonical conformal embeddings with dimensions growing exponentially with .
Introduces PELP for graph-enhanced word embeddings.
problem Combining graph side-information into static word embeddings.
method Probabilistic embeddings using Laplacian priors.
result Unified and flexible approach to various embedding methods.
Curvature regularization prevents distortion in graph embeddings.
problem Graph topology patterns distort in Euclidean space, making detection difficult.
method Proposes curvature regularization to enforce flatness in embedding manifolds.
result Significant improvements in five embedding methods on open graph datasets.
Word embeddings are a powerful approach for unsupervised analysis of language. Recently, Rudolph et al. (2016) developed exponential family embeddings, which cast word embeddings in a probabilistic framework. Here, we develop dynamic embeddings, building on exponential family embeddings to capture how the meanings of w…
Proposes cone embedding for better graph hierarchical structure representation.
problem Lack of natural and interpretable hierarchical indicators in graph embeddings.
method Metric cone embedding method to capture hierarchical structure.
result Extracts hierarchical structure from other graph embedding outputs.
Classifies linear embeddings of grassmannians and ind-grassmannians.
problem Understanding linear embeddings of grassmannians and ind-grassmannians.
method Classification through isomorphism of Picard groups and direct limits.
result Most linear embeddings of grassmannians are equivariant.
BC-Aligner maintains backward compatibility of embeddings after frequent updates.
problem Updating embeddings without requiring consumer teams to retrain their models.
method Learning backward compatible embeddings through BC-Aligner.
result BC-Aligner maintains backward compatibility with existing unintended tasks after multiple model version updates.
Unified framework for word embedding models using noise examples.
problem Improving word embedding models with negative sampling.
method Formulated a Word-Context Classification (WCC) framework that generalizes SkipGram word embedding models.
result The best noise distribution is the data distribution, improving both performance and training speed.
Embedding calculus proves convergence for surfaces.
problem Proving convergence of embedding calculus for surfaces.
method Goodwillie-Weiss' embedding calculus for spaces of embeddings into a manifold of dimension at most two.
result Relates Johnson filtration of mapping class group to embedding calculus.
Models use embeddings and attention for better claim severity prediction.
problem Improving predictive modeling of claim severity with categorical data.
method Developed neural networks and attention-based architectures with embeddings.
result Attention-based models enhance predictive performance with contextual augmentation.
A fast graph embedding method for large graphs.
problem Efficiently embedding large graphs for various applications.
method One-hot graph encoder embedding with linear complexity.
result Graph encoder embedding is approximately normally distributed and converges to its mean.
Paper proves impossibility of three desirable properties in node embedding.
problem Understanding limitations of node embedding methods.
method Axiomatic approach to node embedding, proving impossibility of three properties.
result No node embedding method can satisfy all three desirable properties simultaneously.
The study characterizes and verifies equivariant embeddings of symmetric Kählerian manifolds.
problem Characterizing and verifying equivariant embeddings of symmetric Kählerian manifolds.
method Investigation motivated by Cartan and Wallach's theorem on symmetric spaces, focusing on and parallel plurimean curvature.
result If an equivariant embedding has parallel plurimean curvature, it is the extrinsically symmetric one.
Proves uniqueness of embedding complex manifold into infinite-dimensional space.
problem Balanced embedding of non-compact complex manifold into infinite-dimensional projective space.
method Fine estimates of asymptotics of a balanced embedding.
result Uniqueness of embedding proven.
The paper defines invariants for almost graph embeddings and explores their properties.
problem Understanding the properties and limitations of almost graph embeddings in the plane.
method Introducing and analyzing integer invariants (winding number, Wu numbers) for almost embeddings.
result Some values of invariants are realizable for almost embeddings but not for embeddings.
This paper proposes a new method for embedding sequences using Wasserstein distances.
problem Embedding sequences in a metric space for better pattern recognition.
method Develops a deep learning model that embeds sequences as distributions and uses Wasserstein distances for comparison.
result Distributional embeddings using Wasserstein distances outperform traditional vector embeddings.
Recently, click-through rate (CTR) prediction models have evolved from shallow methods to deep neural networks. Most deep CTR models follow an Embedding\&MLP paradigm, that is, first mapping discrete id features, e.g. user visited items, into low dimensional vectors with an embedding module, then learn a multi-layer pe…
This work analyzes PPR-based node embeddings and their topological information.
problem Understanding and interpreting PPR-based node embeddings.
method Unified framework and two methods for topology recovery.
result PPR-based embeddings maintain more topological information than random walk-based embeddings.
Network representation learning in low dimensional vector space has attracted considerable attention in both academic and industrial domains. Most real-world networks are dynamic with addition/deletion of nodes and edges. The existing graph embedding methods are designed for static networks and they cannot capture evol…
For leveled spatial graphs, we find a surface embedding that allows cellular embedding.
problem Finding a surface embedding for general spatial graphs is not always possible.
method Define leveled property, decompose graph into subgraphs, and construct surface.
result For leveled spatial graphs with a small number of levels, a surface can always be found.
Proposes spherical text embedding for better directional similarity.
problem Directional similarity is more effective but unsupervised text embeddings are typically learned in Euclidean space.
method Develops a spherical generative model and an efficient optimization algorithm for unsupervised word and paragraph embeddings.
result Achieves state-of-the-art performances on various text embedding tasks.
Long spacelike embeddings can be approximated by isometric ones.
problem Approximating long embeddings to isometric embeddings in Lorentzian spaces.
method Proving approximation by constructing isometric embeddings.
result Long spacelike embeddings can be -approximated by isometric embeddings.
Wrinkled Embeddingsmath.GT
A {\it wrinkled embedding} is a topological embedding which is a smooth embedding everywhere on except a set of -dimensional spheres, where has cuspidal corners. In this paper we prove that any rotation of the tangent plane field of a {\it smoothly embedded} submanifold $V\s…
Graph embeddings have become a key and widely used technique within the field of graph mining, proving to be successful across a broad range of domains including social, citation, transportation and biological. Graph embedding techniques aim to automatically create a low-dimensional representation of a given graph, whi…
Recent works reveal that network embedding techniques enable many machine learning models to handle diverse downstream tasks on graph structured data. However, as previous methods usually focus on learning embeddings for a single network, they can not learn representations transferable on multiple networks. Hence, it i…
The paper proves nonexistence and existence results for minimal surfaces in R^4.
problem Proving nonexistence and existence of minimal surfaces in R^4.
method Analyzing complete minimal immersions with finite total curvature and embedded planar ends.
result Existence of embedded minimal spheres in R^4 with 3 embedded planar ends.
Obtaining continuous representations of structural data such as directed acyclic graphs (DAGs) has gained attention in machine learning and artificial intelligence. However, embedding complex DAGs in which both ancestors and descendants of nodes are exponentially increasing is difficult. Tackling in this problem, we de…
Graphs embeddable on torus and linklessly in 3D can be embedded linklessly in standard torus.
problem Embedding linklessly in a standard torus for graphs embeddable on torus and in 3D.
method Analyzing graphs of order 9 and below, showing linkless embedding in standard torus.
result For graphs of order 9 and below, linkless embedding in standard torus is possible.
Geometrically transforms word embeddings into a common space for better comparison.
problem Comparing embeddings from different sources is challenging.
method Applies orthogonal rotations and Mahalanobis scaling to transform embeddings into a shared latent space.
result The method improves word similarity and analogy tasks.
MCE reduces embedding instability in nonlinear dimensionality reduction.
problem Embedding instability caused by random initialization.
method Median of multiple embeddings (MCE) based on large deviation theory.
result MCE achieves consistency at an exponential rate and effectively mitigates instability.
Natural language processing has improved tremendously after the success of word embedding techniques such as word2vec. Recently, the same idea has been applied on source code with encouraging results. In this survey, we aim to collect and discuss the usage of word embedding techniques on programs and source code. The a…