Paper refines cross-lingual word embeddings using Manhattan norm.
problem Sensitivity of ℓ2 norm loss function to outliers in CLWEs. method Post-processing step using ℓ1 norm to improve CLWEs. result The ℓ1 refinement substantially outperforms state-of-the-art baselines. The paper proves norms on braid group commutator subgroup are unbounded.
problem Understanding norms on the commutator subgroup of infinite braid groups.
method Constructed stably unbounded norms and showed equivalence to biinvariant word norms.
result Found norms on commutator subgroup are equivalent to biinvariant word norms and are stably unbounded.
We study biinvariant word metrics on groups. We provide an efficient algorithm for computing the biinvariant word norm on a finitely generated free group and we construct an isometric embedding of a locally compact tree into the biinvariant Cayley graph of a nonabelian free group. We investigate the geometry of cyclic …
Sparse activations in neural models correlate with frequent words, suggesting sparsity is natural.
problem Interpretability and resource efficiency in neural language models.
method Used the Taxi-Euclidean norm to measure sparsity and analyzed gradients and activations of frequent words.
result Frequent input words are associated with sparse activations, while frequent target words are associated with dispersed activations.
Clarifies metric properties on group power sets.
problem Properties of metric on group power sets.
method Analysis of Hausdorff-like metric induced from word length norm.
result Results on quasi-isometries between subspaces of power sets.
PCA whitening weighted by Zipfian word frequencies improves task performance.
problem Skewed word embedding spaces in neural models.
method PCA whitening weighted by empirical word frequencies following Zipf's law.
result Significantly improves task performance, surpassing baselines.
Let G be a word-hyperbolic group, obtained as a graph of free groups amalgamated along cyclic subgroups. If H_2(G;Q) is nonzero, then G contains a closed hyperbolic surface subgroup. Moreover, the unit ball of the Gromov-Thurston norm on H_2(G;R) is a finite-sided rational polyhedron.
Random veering triangulations are not geometric in most cases.
problem Characterizing geometric properties of veering triangulations.
method Combining Teichmüller theory, Ending Lamination Theorem, Thurston norm study, and rigorous computation.
result Generically, veering triangulations are not geometric.
New ONMF model minimizes KL divergence for better sparse data modeling.
problem Clustering and data modeling with sparse vectors.
method Developed KL-ONMF algorithm based on alternating optimization.
result KL-ONMF outperforms Frobenius-norm ONMF for document classification and hyperspectral image unmixing.
New method improves interpretability of text adversarial examples.
problem Generating interpretable adversarial examples in text is challenging.
method Sparse Projected Gradient Descent (SPGD) with directional and sparsity constraints.
result Improves interpretability and likelihood of adversarial examples compared to state-of-the-art methods.
We prove a rigidity theorem for the geometry of the unit ball in random subspaces of the scl norm in B_1^H of a free group. In a free group F of rank k, a random word w of length n (conditioned to lie in [F,F]) has scl(w)=log(2k-1)n/6log(n) + o(n/log(n)) with high probability, and the unit ball in a subspace spanned by…
The paper offers generalization bounds for Transformers that ignore sequence length.
problem Developing generalization bounds for Transformers that are independent of sequence length.
method Covering number approach to upper bound Rademacher complexity of bounded linear transformations.
result Theoretical bounds for Transformer generalization are independent of sequence length.
Unified scalable equivalent formulations for Schatten quasi-norms improve efficiency.
problem Efficiently solving Schatten quasi-norm minimization problems for large-scale matrices.
method Proved equivalence between Schatten-p quasi-norm and product/sum of Schatten-p1 and p2 norms of factor matrices.
result Transformed SQNM problems into simpler, more efficient algorithms for p>1/2.
Quaternionic frames' admissibility and homotopy proven.
problem Existence and interpolation of quaternionic frames.
method Interpreting frames as adjoint orbits.
result Spaces of quaternionic frames are path-connected.
This paper studies ordered weighted L1 (OWL) norm regularization for sparse estimation problems with strongly correlated variables. We prove sufficient conditions for clustering based on the correlation/colinearity of variables using the OWL norm, of which the so-called OSCAR is a particular case. Our results extend pr…
Study on quantitative aspects of trace polynomials in free groups.
problem Understanding the exact formula and bounds for trace polynomials in free groups.
method Proved exact formula for leading homogeneous part, obtained sharp bounds, studied random words, and provided deterministic algorithm.
result Sharp bounds on the degree of trace polynomials and growth rates of polynomial sizes.
Classifies links with specific properties in 3-sphere.
problem Classifying links with Thurston norm minimizing surfaces.
method Uses sutured Floer homology and techniques from knot theory.
result Links with rank 1 in their Floer homology group are classified.
Transformers trained with three normalization changes outperform state-of-the-art on low-resource translation tasks.
problem Improving the training of Transformers, especially on low-resource datasets.
method Three normalization changes: PreNorm, ScaleNorm, and FixNorm.
result Significant improvements in BLEU scores on low-resource translation tasks.
Paper learns identity-sensitive word embeddings from text corpora.
problem Lack of context-aware word embeddings.
method Constructs a heterogeneous network of words and identities, then embeds into a low-dimensional space.
result Identity-sensitive word embeddings capture different meanings of words.
A new method selects anchor words for better topic discovery in text corpora.
problem Selecting anchor words for improved topic modeling in text corpora.
method Proposes a new greedy method to find a minimum edge-weight anchor clique in a word similarity graph.
result The proposed method outperforms existing methods on topic quality and is faster.
There are certain families of words and word sequences (words in the generators of a two-generator group) that arise frequently in the Teichm{ü}ller theory of hyperbolic three-manifolds and Kleinian and Fuchsian groups and in the discreteness problem for two generator matrix groups. We survey some of the families of su…
Probabilistic FastText captures multiple word senses and sub-word structures.
problem Capturing multiple word senses and sub-word structures in word embeddings.
method Probabilistic FastText uses Gaussian mixture densities to represent words, sharing statistical strength across sub-word structures and capturing different word senses.
result Probabilistic FastText outperforms existing models on word-similarity benchmarks and discerning different meanings.
We obtain sharp estimates on the growth rate of stable commutator length on random (geodesic) words, and on random walks, in hyperbolic groups and groups acting nondegenerately on hyperbolic spaces. In either case, we show that with high probability stable commutator length of an element of length n is of order $n/\l…
FRAGE learns word embeddings without frequency bias, improving performance across NLP tasks.
problem Word embeddings are biased towards word frequency, affecting performance for rare words.
method Adversarial training to learn Frequency-Agnostic word Embedding (FRAGE).
result FRAGE achieves higher performance than baselines in all four NLP tasks.
Geometrically transforms word embeddings into a common space for better comparison.
problem Comparing embeddings from different sources is challenging.
method Applies orthogonal rotations and Mahalanobis scaling to transform embeddings into a shared latent space.
result The method improves word similarity and analogy tasks.
We discuss a topological approach to words introduced by the author. Words on an arbitrary alphabet are approximated by Gauss words and then studied up to natural modifications inspired by the Reidemeister moves on knot diagrams. This leads us to a notion of homotopy for words. We introduce several homotopy invariants …
New word distributions capture multiple meanings and outperform existing methods.
problem Capturing semantic information for words with multiple meanings.
method Gaussian mixtures with an energy-based max-margin objective.
result Multimodal word distributions outperform word2vec and Gaussian embeddings.
Paper proposes a new word embedding method optimizing word similarity.
problem Optimizing word similarity in embedding space.
method Two-step random walks between words via topics to learn an optimal embedding simplex.
result Our method outperforms existing approaches in various queries.
The abstract explains how word and relation representations capture semantic meaning.
problem Understanding how word and relation representations capture semantic meaning.
method Theoretical justification and extension of geometric relationships between word embeddings and knowledge graph representations.
result The geometric relationships between word embeddings correspond to semantic relations between words and entities in knowledge graphs.
Paper finds better words for topic models by reranking top words.
problem Top words in topic models are not always representative.
method Reranking words by considering marginal probability over every topic.
result Reranked top words are more representative of topics.
This paper improves monaural source enhancement using SDR as an objective function.
problem Maximizing signal-to-distortion ratio (SDR) for better monaural source enhancement.
method Uses signal-to-distortion ratio (SDR) as an objective function to improve monaural source enhancement.
result The proposed method achieved better performance than conventional methods.
Approaches KL divergence for learning multi-sense word distributions.
problem Capturing the polysemy and uncertainty of words in word embeddings.
method Modeling words as multi-sense Gaussian mixtures and using KL divergence for learning.
result The proposed approach effectively captures word entailment and distribution similarity.
Paper proposes MMD-Sense-Analysis for detecting word sense shifts.
problem Detecting and interpreting shifts in word meanings over time.
method Leverages Maximum Mean Discrepancy (MMD) to identify and explain word sense changes.
result Demonstrates effectiveness of MMD-Sense-Analysis through empirical results.
Bayesian algorithm improves word representations using semantic taxonomy.
problem Improving word representations in semantic taxonomy.
method Bayesian Hierarchical Words Representation (BHWR) learning algorithm combining Variational Bayes and semantic taxonomy modeling.
result BHWR produces better representations for rare words.
New method improves ASR word confidence for diverse applications.
problem Mitigating ASR errors and improving word error rate.
method Heterogeneous Word Confusion Network (HWCN) with score calibration.
result Word sequence with best overall confidence is more accurate than 1-best result.
Word2vec improved but lacks multi-meaning words; ConEc creates new embeddings.
problem Lack of meaningful embeddings for words with multiple meanings and OOV words.
method Context encoders (ConEc) extend word2vec by multiplying embeddings with context vectors.
result ConEc creates embeddings for OOV words and words with multiple meanings based on local contexts.
New method uses word subspaces and term-frequency to improve text classification.
problem Lack of semantic meaning in bag-of-words features.
method Proposes word subspaces and term-frequency weighted word subspaces for text classification.
result Improved text classification performance compared to state-of-the-art algorithms.
Recent work on learning multilingual word representations usually relies on the use of word-level alignements (e.g. infered with the help of GIZA++) between translated sentences, in order to align the word embeddings in different languages. In this workshop paper, we investigate an autoencoder model for learning multil…
Proposes MorphMine for unsupervised morpheme segmentation to improve word embeddings.
problem Lack of semantic information in word-level analysis for infrequent and out-of-vocabulary words.
method MorphMine applies a parsimony criterion to hierarchically segment words into the fewest number of morphemes.
result MorphMine segments words into human-verified morphemes and improves word embedding quality.
Estimator Vectors learns OOV word embeddings using subword and context clues.
problem Lack of OOV word representations in neural network models.
method Jointly learns word, subword, and context clue representations.
result Strong estimates for OOV words via combined subword and context clue embeddings.
Word embeddings reveal multiple senses, which can be recovered using sparse coding.
problem Understanding word senses in polysemous words.
method Sparse coding of word embeddings to recover multiple senses.
result Sparse coding can approximate multiple word senses, with each sense associated with a discourse atom.
End-to-end ASR model combines word and character representation for improved performance.
problem Difficulty in training with word-level supervision due to sparsity of examples.
method Multi-task learning framework combining word and character representations.
result Improved word-error rate (WER) by interpolating between word-level and character-level models.
Paper analyzes word embedding composition using tensor decomposition.
problem Given vector representations of two words, compute a vector for the entire phrase.
method Generative model with low rank Tucker decomposition of word embedding correlations.
result Word embeddings and a core tensor can be derived from the Tucker decomposition.
SWESA learns word embeddings with document labels for sentiment analysis.
problem Sentiment analysis using limited text data.
method SWESA uses supervised learning to optimize word embeddings and classifier performance.
result SWESA outperforms existing methods in sentiment analysis.
The study examines how character and word-level representations improve sentence-level tasks.
problem Improving the quality of word and sentence representations using character-level information.
method Feature-wise sigmoid gating mechanism for combining character and word-level representations.
result Modeling characters improves final word and sentence representations, especially for less frequent words.
By defining combinatorial moves, we can define an equivalence relation on Gauss words called homotopy. In this paper we define a homotopy invariant of Gauss words. We use this to show that there exist Gauss words that are not homotopically equivalent to the empty Gauss word, disproving a conjecture by Turaev. In fact, …
We introduce a topological approach to words. Words are approximated by Gauss words and then studied up to natural modifications inspired by homotopy transformations of curves on the plane.
A longstanding question of Gromov asks whether every one-ended word-hyperbolic group contains a subgroup isomorphic to the fundamental group of a closed hyperbolic surface. An infinite family of word-hyperbolic groups can be obtained by taking doubles of free groups amalgamated along words that are not proper powers. W…