The recent advances in deep neural networks (DNNs) make them attractive for embedded systems. However, it can take a long time for DNNs to make an inference on resource-constrained computing devices. Model compression techniques can address the computation issue of deep inference on embedded devices. This technique is …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “embedding techniques”
Natural language processing has improved tremendously after the success of word embedding techniques such as word2vec. Recently, the same idea has been applied on source code with encouraging results. In this survey, we aim to collect and discuss the usage of word embedding techniques on programs and source code. The a…
While the celebrated Word2Vec technique yields semantically rich representations for individual words, there has been relatively less success in extending to generate unsupervised sentences or documents embeddings. Recent work has demonstrated that a distance measure between documents called \emph{Word Mover's Distance…
Bridging the Gap between Community and Node Representations: Graph Embedding via Community Detectioncs.SI
Graph embedding has become a key component of many data mining and analysis systems. Current graph embedding approaches either sample a large number of node pairs from a graph to learn node embeddings via stochastic optimization or factorize a high-order proximity/adjacency matrix of the graph via computationally expen…
Graph embeddings have become a key and widely used technique within the field of graph mining, proving to be successful across a broad range of domains including social, citation, transportation and biological. Graph embedding techniques aim to automatically create a low-dimensional representation of a given graph, whi…
New methods explain NE embeddings by identifying key variables.
problem Lack of interpretability in NE techniques.
method Combining PCA, Q-residuals, Hotelling's T2, and visualization.
result Identifies discriminatory features not seen in standard approaches.
Graph embedding aims at learning a vector-based representation of vertices that incorporates the structure of the graph. This representation then enables inference of graph properties. Existing graph embedding techniques, however, do not scale well to large graphs. We therefore propose a framework for parallel computat…
New research shows graph embeddings fail to capture key network properties.
problem Graph embeddings fail to capture salient properties of complex networks.
method Mathematical proof and empirical study of various embedding techniques.
result Any successful graph embedding must have a rank nearly linear in the number of vertices.
The paper explores theories behind graph and relational data vector embeddings.
problem Understanding the foundations of vector embeddings for graphs and relational structures.
method Proposes two theoretical approaches to understand vector embeddings.
result Draws connections between various embedding techniques and suggests future research directions.
We give several generalizations of the Kodaira vanishing and embedding theorems for Kähler manifolds to the case where the relevent line bundle has a small region of negative curvature. To prove the vanishing theorems we adapt techniques of Elworthy-Rosenberg for vanishing theorems in Riemannian geometry. For the embed…
Local Procrustes for Manifold Embedding: A Measure of Embedding Quality and Embedding Algorithmsstat.ML
We present the Procrustes measure, a novel measure based on Procrustes rotation that enables quantitative comparison of the output of manifold-based embedding algorithms (such as LLE (Roweis and Saul, 2000) and Isomap (Tenenbaum et al, 2000)). The measure also serves as a natural tool when choosing dimension-reduction …
New walk extraction strategies improve node embeddings in KGs.
problem Improving node embeddings in knowledge graphs.
method Proposed five different walk extraction strategies to complement basic random walks.
result The n-gram strategy performs best on average for node classification tasks.
Spectral embedding is a popular technique for the representation of graph data. Several regularization techniques have been proposed to improve the quality of the embedding with respect to downstream tasks like clustering. In this paper, we explain on a simple block model the impact of the complete graph regularization…
In this paper we construct a parametrization-free embedding technique for numerically evolving reaction-diffusion PDEs defined on algebraic curves that possess an isolated singularity. In our approach, we first desingularize the curve by appealing to techniques from algebraic geometry. We create a family of smooth curv…
Network embedding techniques inspired by word2vec represent an effective unsupervised relational learning model. Commonly, by means of a Skip-Gram procedure, these techniques learn low dimensional vector representations of the nodes in a graph by sampling node-context examples. Although many ways of sampling the contex…
This study improves sentence embeddings from BERT models.
problem Capturing the underlying meaning of sentences using BERT models.
method Comprehensive review and testing of various sentence embedding extraction and refinement methods.
result Representation-shaping techniques significantly improve sentence embeddings from BERT-based and simple baseline models.
FEET protocol evaluates foundation models across three scenarios.
problem Lack of standardized evaluation protocols for foundation models.
method Structured evaluation protocol across three use cases: frozen, few-shot, and fine-tuned embeddings.
result Demonstrates comprehensive assessment of foundation models' effectiveness.
ELM improves neural model embeddings for long-tail learning.
problem Learning skewed label distributions in neural models.
method Enforces margins in logit space and regularizes embedding distribution.
result ELM reduces generalization gap and tightens tail class embeddings.
(Bolukbasi et al., 2016) demonstrated that pretrained word embeddings can inherit gender bias from the data they were trained on. We investigate how this bias affects downstream classification tasks, using the case study of occupation classification (De-Arteaga et al.,2019). We show that traditional techniques for debi…
Embedding-reparameterization procedure for manifold-valued latent variables in generative modelscs.LG
Conventional prior for Variational Auto-Encoder (VAE) is a Gaussian distribution. Recent works demonstrated that choice of prior distribution affects learning capacity of VAE models. We propose a general technique (embedding-reparameterization procedure, or ER) for introducing arbitrary manifold-valued variables in VAE…
ALF reduces network parameters and operations by 70% and 61%, respectively, on embedded hardware.
problem Efficient deployment of deep learning models on resource-constrained hardware.
method Autoencoder-based low-rank filter-sharing technique.
result ALF achieves significant compression with minimal accuracy loss.
Isometric embeddingsmath.DG
Paper uses advanced math to embed complex shapes smoothly.
problem Embedding shapes smoothly in high dimensions.
method Convex integration technique with extra iteration.
result Achieved smooth embedding in class.
Task Embedded Coordinate Update: A Realizable Framework for Multivariate Non-convex Optimizationcs.LG
We in this paper propose a realizable framework TECU, which embeds task-specific strategies into update schemes of coordinate descent, for optimizing multivariate non-convex problems with coupled objective functions. On one hand, TECU is capable of improving algorithm efficiencies through embedding productive numerical…
Just as semantic hashing can accelerate information retrieval, binary valued embeddings can significantly reduce latency in the retrieval of graphical data. We introduce a simple but effective model for learning such binary vectors for nodes in a graph. By imagining the embeddings as independent coin flips of varying b…
In this paper we prove the existence of rational homology balls smoothly embedded in regular neighborhoods of certain linear chains of smooth -spheres by using techniques from minimal model program for 3-dimensional complex algebraic variety.
Surfaces in 4-Manifoldsmath.DG
In this paper we introduce a technique, called rim surgery, which can change a smooth embedding of an orientable surface of positive genus and nonnegative self-intersection in a smooth 4-manifold while leaving the topological embedding unchanged.
Optimizes embedding accuracy for data variance and error.
problem Efficiently embedding data while minimizing distortion.
method Uses Johnson-Lindenstrauss embeddings with orthogonal matrices and singular-value latent variables.
result Achieves best accuracy in variance, mean-squared error, and length distortion.
The power of machine learning systems not only promises great technical progress, but risks societal harm. As a recent example, researchers have shown that popular word embedding algorithms exhibit stereotypical biases, such as gender bias. The widespread use of these algorithms in machine learning systems, from automa…
Typical large-scale recommender systems use deep learning models that are stored on a large amount of DRAM. These models often rely on embeddings, which consume most of the required memory. We present Bandana, a storage system that reduces the DRAM footprint of embeddings, by using Non-volatile Memory (NVM) as the prim…
Node2vec embeddings are unstable and unstable with parameter choices.
problem Stability and robustness of node2vec embeddings for graph classification.
method Analysis of node2vec embeddings from multiple perspectives.
result Node2vec embeddings are unstable with respect to parameter choices.
Knowledge graphs store facts using relations between two entities. In this work, we address the question of link prediction in knowledge hypergraphs where relations are defined on any number of entities. While techniques exist (such as reification) that convert non-binary relations into binary ones, we show that curren…
New method learns state embeddings from demonstrations for improved reinforcement learning.
problem Difficult relationship between observed state and useful policy actions in dynamic problems.
method Variational framework for learning state embeddings that optimize trajectory linearity.
result Learning embedding spaces improves policy gradient reinforcement learning performance.
We propose and evaluate new techniques for compressing and speeding up dense matrix multiplications as found in the fully connected and recurrent layers of neural networks for embedded large vocabulary continuous speech recognition (LVCSR). For compression, we introduce and study a trace norm regularization technique f…
Paper enhances haptic signals distinguishability with boosted technique.
problem Lack of large datasets in haptics domain limits feature extraction.
method General framework for haptic signal analysis, using spectral features and boosted embedding.
result Framework needs less training data and outperforms state-of-the-art.
This paper evaluates debiasing methods on word embeddings to reduce religious bias.
problem Social biases persist in word embeddings, potentially amplifying them in AI applications.
method Investigates and evaluates three multiclass debiasing techniques on three word embeddings.
result ConceptorDebiasing is the most effective method, reducing religious bias by 82-96%.
Identifier names convey useful information about the intended semantics of code. Name-based program analyses use this information, e.g., to detect bugs, to predict types, and to improve the readability of code. At the core of name-based analyses are semantic representations of identifiers, e.g., in the form of learned …
Most popular word embedding techniques involve implicit or explicit factorization of a word co-occurrence based matrix into low rank factors. In this paper, we aim to generalize this trend by using numerical methods to factor higher-order word co-occurrence based arrays, or \textit{tensors}. We present four word embedd…
The paper proves isometric embeddings for smooth manifolds.
problem Embedding smooth manifolds isometrically into Euclidean spaces.
method Nash-Kuiper's convex integration construction and gluing technique.
result Infinitely many global isometric embeddings into 2n-dimensional Euclidean space.
A new clustering method improves recovery guarantees by re-embedding data.
problem Improving recovery guarantees in clustering algorithms.
method Chaining four techniques: leapfrog distances, multidimensional scaling, spectral methods, and sum-of-norms clustering.
result Re-embedding data improves recovery guarantees of clustering.
Motivated by manifold learning techniques, we give an explicit lower bound for how far a smoothly embedded compact submanifold in can move in a normal direction and remain an embedding. In addition, given a penalty function on the space of embeddi…
With the rising interest in graph representation learning, a variety of approaches have been proposed to effectively capture a graph's properties. While these approaches have improved performance in graph machine learning tasks compared to traditional graph techniques, they are still perceived as techniques with limite…
DSNE visualizes data velocity in lower dimensions.
problem Understanding movement patterns in high-dimensional data.
method DSNE is a variation of Stochastic Neighbor Embedding that learns velocity embeddings using Euclidean distances on a unit sphere.
result DSNE enables visualization of data movement in lower dimensions.
New families of embeddings in 4-manifolds, topologically trivial but smoothly non-trivial.
problem Constructing non-trivial smooth embeddings of 3-manifolds in 4-manifolds.
method Parameterized families of embeddings, using high-dimensional spheres.
result Embeddings of homology spheres and any 3-manifold in blown-up K3 surfaces.
Stochastic Neighbor Embedding and its variants are widely used dimensionality reduction techniques -- despite their popularity, no theoretical results are known. We prove that the optimal SNE embedding of well-separated clusters from high dimensions to any Euclidean space R^d manages to successfully separate the cluste…
Despite being very successful within the pattern recognition and machine learning community, graph-based methods are often unusable because of the lack of mathematical operations defined in graph domain. Graph embedding, which maps graphs to a vectorial space, has been proposed as a way to tackle these difficulties ena…
We propose a new embedding method which is particularly well-suited for settings where the sample size greatly exceeds the ambient dimension. Our technique consists of partitioning the space into simplices and then embedding the data points into features corresponding to the simplices' barycentric coordinates. We then …
New embedding method in function spaces improves expressiveness.
problem Enhancing expressiveness in knowledge graph embeddings.
method Employing polynomial functions and neural networks with varying layer complexities.
result Improved expressiveness and more degrees of freedom in entity representation.
LDLE embeds manifolds in lower dimensions with low distortion.
problem Embedding manifolds in lower dimensions with low distortion.
method Constructs local views using global eigenvectors of the graph Laplacian, registers them using Procrustes analysis, and tears manifolds apart for intrinsic dimension embedding.
result LDLE preserves distances up to a constant scale with low distortion.