Paper proposes neural approach for Chinese named entity recognition.
problem Challenges in Chinese named entity recognition due to context-dependency and lack of word delimiters.
method Introduces a CNN-LSTM-CRF neural architecture and a unified framework for joint training with word segmentation.
result Improves Chinese named entity recognition performance, especially with limited training data.
DELIMIT is a framework extension for deep learning in diffusion imaging, which extends the basic framework PyTorch towards spherical signals. Based on several novel layers, deep learning can be applied to spherical diffusion imaging data in a very convenient way. First, two spherical harmonic interpolation layers are a…
New formula connects Loewner energy to moving frames' renormalised energy.
problem Calculating Loewner energy of Jordan curves.
method Using renormalised energy of moving frames.
result Loewner energy as Kähler potential for Weil-Petersson space.
New method optimizes tail dependence coefficient estimation.
problem Estimating tail dependence in nonparametric data.
method Optimal threshold selection combining mean squared error and copula estimation.
result Improved accuracy in tail dependence coefficient estimation.
The superfamily phenomenon of time series with different dynamics can be characterized by the motif rank patterns observed in the nearest-neighbor networks of the time series in phase space. However, the determinants of superfamily classification are unclear. We attack this problem by studying the influence of linear t…
In this work, an ensemble of economic interacting agents is considered. The agents are arranged in a linear array where only local couplings are allowed. The deterministic dynamics of each agent is given by a map. This map is expressed by two factors. The first one is a linear term that models the expansion of the agen…
Paper tackles counterfactual sentence detection and evaluation.
problem Detect and evaluate counterfactual sentences in natural language.
method Used a BERT base model for classification and a hybrid BERT Multi-Layer Perceptron for sequence identification. Introduced cascaded linear inputs to improve performance.
result Achieved an F1 score of 85.00% in Task 1 and 83.90% in Task 2.
IMeL turns RL into SL by interpolating improved experiences.
problem Improving reinforcement learning efficiency and scalability.
method IMeL uses a reservoir of experiences and a NN regressor for interpolation.
result IMeL achieves preliminary results and proposes itself as a baseline.
Integral invariants obtained from Principal Component Analysis on a small kernel domain of a submanifold encode important geometric information classically defined in differential-geometric terms. We generalize to hypersurfaces in any dimension major results known for surfaces in space, which in turn yield a method to …
A new method selects anchor words for better topic discovery in text corpora.
problem Selecting anchor words for improved topic modeling in text corpora.
method Proposes a new greedy method to find a minimum edge-weight anchor clique in a word similarity graph.
result The proposed method outperforms existing methods on topic quality and is faster.
The diagonal effect of orders is well documented in different markets, which states that orders are more likely to be followed by orders of the same aggressiveness and implies the presence of short-term correlations in order flows. Based on the order flow data of 43 Chinese stocks, we investigate if there are long-rang…
Most existing word embedding approaches do not distinguish the same words in different contexts, therefore ignoring their contextual meanings. As a result, the learned embeddings of these words are usually a mixture of multiple meanings. In this paper, we acknowledge multiple identities of the same word in different co…
There are certain families of words and word sequences (words in the generators of a two-generator group) that arise frequently in the Teichm{ü}ller theory of hyperbolic three-manifolds and Kleinian and Fuchsian groups and in the discreteness problem for two generator matrix groups. We survey some of the families of su…
Geometrically transforms word embeddings into a common space for better comparison.
problem Comparing embeddings from different sources is challenging.
method Applies orthogonal rotations and Mahalanobis scaling to transform embeddings into a shared latent space.
result The method improves word similarity and analogy tasks.
We discuss a topological approach to words introduced by the author. Words on an arbitrary alphabet are approximated by Gauss words and then studied up to natural modifications inspired by the Reidemeister moves on knot diagrams. This leads us to a notion of homotopy for words. We introduce several homotopy invariants …
Continuous word representation (aka word embedding) is a basic building block in many neural network-based models used in natural language processing tasks. Although it is widely accepted that words with similar semantics should be close to each other in the embedding space, we find that word embeddings learned in seve…
The abstract explains how word and relation representations capture semantic meaning.
problem Understanding how word and relation representations capture semantic meaning.
method Theoretical justification and extension of geometric relationships between word embeddings and knowledge graph representations.
result The geometric relationships between word embeddings correspond to semantic relations between words and entities in knowledge graphs.
Paper proposes MMD-Sense-Analysis for detecting word sense shifts.
problem Detecting and interpreting shifts in word meanings over time.
method Leverages Maximum Mean Discrepancy (MMD) to identify and explain word sense changes.
result Demonstrates effectiveness of MMD-Sense-Analysis through empirical results.
Approaches KL divergence for learning multi-sense word distributions.
problem Capturing the polysemy and uncertainty of words in word embeddings.
method Modeling words as multi-sense Gaussian mixtures and using KL divergence for learning.
result The proposed approach effectively captures word entailment and distribution similarity.
Bayesian algorithm improves word representations using semantic taxonomy.
problem Improving word representations in semantic taxonomy.
method Bayesian Hierarchical Words Representation (BHWR) learning algorithm combining Variational Bayes and semantic taxonomy modeling.
result BHWR produces better representations for rare words.
New method improves ASR word confidence for diverse applications.
problem Mitigating ASR errors and improving word error rate.
method Heterogeneous Word Confusion Network (HWCN) with score calibration.
result Word sequence with best overall confidence is more accurate than 1-best result.
We argue that in fully-connected networks a phase transition delimits the over- and under-parametrized regimes where fitting can or cannot be achieved. Under some general conditions, we show that this transition is sharp for the hinge loss. In the whole over-parametrized regime, poor minima of the loss are not encounte…
Recent work on learning multilingual word representations usually relies on the use of word-level alignements (e.g. infered with the help of GIZA++) between translated sentences, in order to align the word embeddings in different languages. In this workshop paper, we investigate an autoencoder model for learning multil…
Proposes MorphMine for unsupervised morpheme segmentation to improve word embeddings.
problem Lack of semantic information in word-level analysis for infrequent and out-of-vocabulary words.
method MorphMine applies a parsimony criterion to hierarchically segment words into the fewest number of morphemes.
result MorphMine segments words into human-verified morphemes and improves word embedding quality.
Estimator Vectors learns OOV word embeddings using subword and context clues.
problem Lack of OOV word representations in neural network models.
method Jointly learns word, subword, and context clue representations.
result Strong estimates for OOV words via combined subword and context clue embeddings.
The study examines how character and word-level representations improve sentence-level tasks.
problem Improving the quality of word and sentence representations using character-level information.
method Feature-wise sigmoid gating mechanism for combining character and word-level representations.
result Modeling characters improves final word and sentence representations, especially for less frequent words.
By defining combinatorial moves, we can define an equivalence relation on Gauss words called homotopy. In this paper we define a homotopy invariant of Gauss words. We use this to show that there exist Gauss words that are not homotopically equivalent to the empty Gauss word, disproving a conjecture by Turaev. In fact, …
We introduce a topological approach to words. Words are approximated by Gauss words and then studied up to natural modifications inspired by homotopy transformations of curves on the plane.
Word embeddings provide point representations of words containing useful semantic information. We introduce multimodal word distributions formed from Gaussian mixtures, for multiple word meanings, entailment, and rich uncertainty information. To learn these distributions, we propose an energy-based max-margin objective…
We introduce Probabilistic FastText, a new model for word embeddings that can capture multiple word senses, sub-word structure, and uncertainty information. In particular, we represent each word with a Gaussian mixture density, where the mean of a mixture component is given by the sum of n-grams. This representation al…
A longstanding question of Gromov asks whether every one-ended word-hyperbolic group contains a subgroup isomorphic to the fundamental group of a closed hyperbolic surface. An infinite family of word-hyperbolic groups can be obtained by taking doubles of free groups amalgamated along words that are not proper powers. W…
The top word list, i.e., the top-M words with highest marginal probability in a given topic, is the standard topic representation in topic models. Most of recent automatical topic labeling algorithms and popular topic quality metrics are based on it. However, we find, empirically, words in this type of top word list ar…
Enhanced word embeddings boost multiclass text classification accuracy.
problem Improving multiclass text classification accuracy using pre-trained embeddings.
method Proposed word-class embeddings (WCEs) to enhance pre-trained word embeddings.
result WCEs significantly improve multiclass text classification accuracy.
Corpus poisoning can manipulate word meanings in word embeddings, affecting natural language processing tasks.
problem Controlling word meanings via corpus modifications.
method Developed an explicit expression over corpus features to control word embeddings.
result Demonstrated the ability to manipulate word meanings in word embeddings, affecting various downstream tasks.
Word embeddings are representations of individual words of a text document in a vector space and they are often use- ful for performing natural language pro- cessing tasks. Current state of the art al- gorithms for learning word embeddings learn vector representations from large corpora of text documents in an unsu- pe…
Word embedding is a powerful tool in natural language processing. In this paper we consider the problem of word embedding composition \--- given vector representations of two words, compute a vector for the entire phrase. We give a generative model that can capture specific syntactic relations between words. Under our …
We create interpretable word embeddings through sparse coding.
problem Difficult to interpret word embeddings in natural language processing.
method Transform pretrained dense word embeddings into sparse embeddings through sparse coding.
result Sparse embeddings are more interpretable and achieve good performance.
Word embedding, which encodes words into vectors, is an important starting point in natural language processing and commonly used in many text-based machine learning tasks. However, in most current word embedding approaches, the similarity in embedding space is not optimized in the learning. In this paper we propose a …
With a simple architecture and the ability to learn meaningful word embeddings efficiently from texts containing billions of words, word2vec remains one of the most popular neural language models used today. However, as only a single embedding is learned for every word in the vocabulary, the model fails to optimally re…
Text classification has become indispensable due to the rapid increase of text in digital form. Over the past three decades, efforts have been made to approach this task using various learning algorithms and statistical models based on bag-of-words (BOW) features. Despite its simple implementation, BOW features lack se…
Deep learning has seen tremendous success over the past decade in computer vision, machine translation, and gameplay. This success rests in crucial ways on gradient-descent optimization and the ability to learn parameters of a neural network by backpropagating observed errors. However, neural network architectures are …
This work learns shared word embeddings for acoustic and phonetic sequences.
problem Mapping variable-length acoustic and phonetic sequences to fixed-dimensional vectors.
method Weak supervision and binary classification task to predict word similarity.
result Best model achieves an F1 score of 0.95 for binary classification.
End-to-end automatic speech recognition (ASR) commonly transcribes audio signals into sequences of characters while its performance is evaluated by measuring the word-error rate (WER). This suggests that predicting sequences of words directly may be helpful instead. However, training with word-level supervision can be …
PCA whitening weighted by Zipfian word frequencies improves task performance.
problem Skewed word embedding spaces in neural models.
method PCA whitening weighted by empirical word frequencies following Zipf's law.
result Significantly improves task performance, surpassing baselines.
Paper uses JIVE to decompose word embeddings, improving sentiment analysis performance.
problem Improving sentiment analysis performance on word embeddings.
method Joint and individual variance explained (JIVE) method for decomposition.
result Mapping word embeddings into joint components improves sentiment analysis performance.
Top2Vec finds topic vectors from documents and words without needing stop words or custom settings.
problem Topic modeling weaknesses, including needing known topics, stop words, and custom settings.
method Joint document and word semantic embedding to find topic vectors automatically.
result Top2Vec finds more informative and representative topics than probabilistic models.
3-manifold groups' word problem solved in nearly linear time.
problem Solving the word problem for 3-manifold groups efficiently.
method Proof for admissible graphs of groups, leveraging Croke and Kleiner's work.
result Word problem solved in O(nlogn) time for 3-manifold groups. Sparse activations in neural models correlate with frequent words, suggesting sparsity is natural.
problem Interpretability and resource efficiency in neural language models.
method Used the Taxi-Euclidean norm to measure sparsity and analyzed gradients and activations of frequent words.
result Frequent input words are associated with sparse activations, while frequent target words are associated with dispersed activations.