arXiv research

Feature extraction has gained increasing attention in the field of machine learning, as in order to detect patterns, extract information, or predict future observations from big data, the urge of informative features is crucial. The process of extracting features is highly linked to dimensionality reduction as it impli…

Word2vec (Mikolov et al., 2013) has proven to be successful in natural language processing by capturing the semantic relationships between different words. Built on top of single-word embeddings, paragraph vectors (Le and Mikolov, 2014) find fixed-length representations for pieces of text with arbitrary lengths, such a…

A classical link in 3-space can be represented by a Gauss paragraph encoding a link diagram in a combinatorial way. A Gauss paragraph may code not a classical link diagram, but a diagram with virtual crossings. We present a criterion and a linear algorithm detecting whether a Gauss paragraph encodes a classical link. W…

A Gauss paragraph is a combinatorial formulation of a generic closed curve with multiple components on some surface. A virtual string is a collection of circles with arrows that represent the crossings of such a curve. Every closed curve has an underlying virtual string and every virtual string has an underlying Gauss …

Gestalt combines two models to improve SQuAD2.0 performance.

Paper develops a framework for generating coherent image captions using visual features and hierarchical topics.

The paper identifies patterns in language model weights used for memorizing paragraphs.

Classic Topic Models are built under the Bag Of Words assumption, in which word position is ignored for simplicity. Besides, symmetric priors are typically used in most applications. In order to easily learn topics with different properties among the same corpus, we propose a new line of work in which the paragraph str…

Text embedding representing natural language documents in a semantic vector space can be used for document retrieval using nearest neighbor lookup. In order to study the feasibility of neural models specialized for retrieval in a semantically meaningful way, we suggest the use of the Stanford Question Answering Dataset…

In this paper we use theory of embedded graphs on oriented and compact $PL$-surfaces to construct minimal realizations of signed Gauss paragraphs. We prove that the genus of the ambient surface of these minimal realizations can be seen as a function of the maximum number of Carter's circles. For the case of signed Gaus…

We formulate coherence modeling as a regression task and propose two novel methods to combine techniques from our setup with pairwise approaches. The first of our methods is a model that we call "first-next," which operates similarly to selection sorting but conditions decision-making on information about already-sorte…

To simultaneously capture syntax and global semantics from a text corpus, we propose a new larger-context recurrent neural network (RNN) based language model, which extracts recurrent hierarchical semantic structure via a dynamic deep topic model to guide natural language generation. Moving beyond a conventional RNN-ba…

Benchmark evaluates financial misinformation detection models, revealing weaknesses without external context.

Recent years have seen remarkable progress of text generation in different contexts, such as the most common setting of generating text from scratch, and the emerging paradigm of retrieval-and-rewriting. Text infilling, which fills missing text portions of a sentence or paragraph, is also of numerous use in real life, …

Learning latent representations from long text sequences is an important first step in many natural language processing applications. Recurrent Neural Networks (RNNs) have become a cornerstone for this challenging task. However, the quality of sentences during RNN-based decoding (reconstruction) decreases with the leng…

Automatic measurement of semantic text similarity is an important task in natural language processing. In this paper, we evaluate the performance of different vector space models to perform this task. We address the real-world problem of modeling patent-to-patent similarity and compare TFIDF (and related extensions), t…

We present an attention-based ranking framework for learning to order sentences given a paragraph. Our framework is built on a bidirectional sentence encoder and a self-attention based transformer network to obtain an input order invariant representation of paragraphs. Moreover, it allows seamless training using a vari…

We present a new local entity disambiguation system. The key to our system is a novel approach for learning entity representations. In our approach we learn an entity aware extension of Embedding for Language Model (ELMo) which we call Entity-ELMo (E-ELMo). Given a paragraph containing one or more named entity mentions…

Transformer models improve query-document retrieval efficiency and accuracy.

Project aims to improve coherence in language generation models.

This work presents a novel objective function for the unsupervised training of neural network sentence encoders. It exploits signals from paragraph-level discourse coherence to train these models to understand text. Our objective is purely discriminative, allowing us to train models many times faster than was possible …

AutoML-GPT uses GPT to automate AI model training.

Automatic question generation is an important problem in natural language processing. In this paper we propose a novel adaptive copying recurrent neural network model to tackle the problem of question generation from sentences and paragraphs. The proposed model adds a copying mechanism component onto a bidirectional LS…

Transformer models are compared to curved spacetime in General Relativity.

Traditionally, text generation models take in a sequence of text as input, and iteratively generate the next most probable word using pre-trained parameters. In this work, we propose the architecture to use images instead of text as the input of the text generation model, called StoryGen. In the architecture, we design…

We apply text analysis approaches for a specialized search engine for 3D CAD models and associated products. The main goals are to distinguish between actual product descriptions and other text on a website, as well as to decide whether a given text is or contains a product name. For this we use paragraph vectors for t…

Unsupervised text embedding has shown great power in a wide range of NLP tasks. While text embeddings are typically learned in the Euclidean space, directional similarity is often more effective in tasks such as word similarity and document clustering, which creates a gap between the training stage and usage stage of t…

Quantum analysis tags news for sentiment and entities.

Most research in reading comprehension has focused on answering questions based on individual documents or even single paragraphs. We introduce a neural model which integrates and reasons relying on information spread within documents and across multiple documents. We frame it as an inference problem on a graph. Mentio…

A new model analyzes document structure and customer shopping patterns.

With the rapid increase of compound databases available in medicinal and material science, there is a growing need for learning representations of molecules in a semi-supervised manner. In this paper, we propose an unsupervised hierarchical feature extraction algorithm for molecules (or more generally, graph-structured…

In this paper, we obtain a new abstract formula relating eigenvalues of a self-adjoint operator to two families of symmetric and skew-symmetric operators and their commutators. This formula generalizes earlier ones obtained by Harrell, Stubbe, Hook, Ashbaugh, Hermi, Levitin and Parnovski. We also show how one can use t…

Study finds stocks with higher cyber risk scores outperform others, indicating a market-wide cyber risk premium.

We show that special cycles generate a large part of the cohomology of locally symmetric spaces associated to orthogonal groups. We prove in particular that classes of totally geodesic submanifolds generate the cohomology groups of degree $n$ of compact congruence $p$-dimensional hyperbolic manifolds "of simple type" a…

SCROLLS benchmarks long text NLP tasks, improving existing models.

Historical documents present many challenges for offline handwriting recognition systems, among them, the segmentation and labeling steps. Carefully annotated textlines are needed to train an HTR system. In some scenarios, transcripts are only available at the paragraph level with no text-line information. In this work…

Correctly detecting the semantic type of data columns is crucial for data science tasks such as automated data cleaning, schema matching, and data discovery. Existing data preparation and analysis systems rely on dictionary lookups and regular expression matching to detect semantic types. However, these matching-based …

Our work proves robustness of embedding schemes to discrete changes in text.

Recent progress in applying machine learning for jet physics has been built upon an analogy between calorimeters and images. In this work, we present a novel class of recursive neural networks built instead upon an analogy between QCD and natural languages. In the analogy, four-momenta are like words and the clustering…

The large volume of text in electronic healthcare records often remains underused due to a lack of methodologies to extract interpretable content. Here we present an unsupervised framework for the analysis of free text that combines text-embedding with paragraph vectors and graph-theoretical multiscale community detect…

Our study employs sentiment analysis to evaluate the compatibility of Amazon.com reviews with their corresponding ratings. Sentiment analysis is the task of identifying and classifying the sentiment expressed in a piece of text as being positive or negative. On e-commerce websites such as Amazon.com, consumers can subm…

New graph embedding method improves link prediction and node classification.

ECC Analyzer uses LLMs to predict stock volatility from ECCs.

The present article is the final part of a series on the classification of the totally geodesic submanifolds of the irreducible Riemannian symmetric spaces of rank 2. After this problem has been solved for the 2-Grassmannians in my previous papers cited in the present paper as [K1] and [K2], and for the space SU(3)/SO(…

We prove that for a relatively hyperbolic group G there is a sequence of relatively hyperbolic proper quotients such that their growth rates converge to the growth rate of G. Under natural assumptions, the same conclusion holds for the critical exponent of a cusp-uniform action of G on a hyperbolic metric space. As a c…

Automation engineering is the task of integrating, via software, various sensors, actuators, and controls for automating a real-world process. Today, automation engineering is supported by a suite of software tools including integrated development environments (IDE), hardware configurators, compilers, and runtimes. The…

NukeBERT improves performance on nuclear domain Q&A with less training data.

In this study, machine learning models were constructed to predict whether judgments made by the European Court of Human Rights (ECHR) would lead to a violation of an Article in the Convention on Human Rights. The problem is framed as a binary classification task where a judgment can lead to a "violation" or "non-viola…