We propose a flexible framework for spectral conversion (SC) that facilitates training with unaligned corpora. Many SC frameworks require parallel corpora, phonetic alignments, or explicit frame-wise correspondence for learning conversion functions or for synthesizing a target spectrum with the aid of alignments. Howev…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “limited corpora”
Speech emotion recognition plays an important role in building more intelligent and human-like agents. Due to the difficulty of collecting speech emotional data, an increasingly popular solution is leveraging a related and rich source corpus to help address the target corpus. However, domain shift between the corpora p…
Improved embeddings by topic-sensitive attention on large corpora.
problem Capturing sense of words in limited corpora using pretrained embeddings.
method Topic-sensitive attention on large topic-rich corpora to correct sense drift in pretrained embeddings.
result Limited corpus augmentation is more effective than adapting pretrained embeddings.
Unsupervised machine translation---i.e., not assuming any cross-lingual supervision signal, whether a dictionary, translations, or comparable corpora---seems impossible, but nevertheless, Lample et al. (2018) recently proposed a fully unsupervised machine translation (MT) model. The model relies heavily on an adversari…
Parallel sentences are a relatively scarce but extremely useful resource for many applications including cross-lingual retrieval and statistical machine translation. This research explores our new methodologies for mining such data from previously obtained comparable corpora. The task is highly practical since non-para…
Modeling lead-lag relationship between two text corpora for improved topic modeling.
problem Recognizing the relationship between multiple text corpora for better topic modeling.
method Proposed a jointly dynamic topic model and embedding extension for large-scale text corpus.
result The proposed model can well recognize the lead-lag relationship between two text corpora and improve topic learning.
Unlocking Noisy Real-World Corpora for Foundation Model Pre-Training via Quality-Aware Tokenizationcs.AI
QA-Token improves tokenization for noisy data, boosting model performance.
problem Tokenization ignores data quality, limiting model effectiveness on noisy corpora.
method QA-Token combines signal quality with vocabulary construction through bilevel optimization and reinforcement learning.
result QA-Token achieves state-of-the-art performance on genomic and financial datasets.
Unsupervised comparable corpora preparation and exploration for bi-lingual translation equivalentscs.CL
The multilingual nature of the world makes translation a crucial requirement today. Parallel dictionaries constructed by humans are a widely-available resource, but they are limited and do not provide enough coverage for good quality translation purposes, due to out-of-vocabulary words and neologisms. This motivates th…
Multi-domain machine translation enhancements by parallel data extraction from comparable corporacs.CL
Parallel texts are a relatively rare language resource, however, they constitute a very useful research material with a wide range of applications. This study presents and analyses new methodologies we developed for obtaining such data from previously built comparable corpora. The methodologies are automatic and unsupe…
We develop the multilingual topic model for unaligned text (MuTo), a probabilistic model of text that is designed to analyze corpora composed of documents in two languages. From these documents, MuTo uses stochastic EM to simultaneously discover both a matching between the languages and multilingual latent topics. We d…
Distributed representations of sentences have become ubiquitous in natural language processing tasks. In this paper, we consider a continual learning scenario for sentence representations: Given a sequence of corpora, we aim to optimize the sentence encoder with respect to the new corpus while maintaining its accuracy …
Ultra-fast search algorithm for trillion-scale corpora with semantic flexibility.
problem Efficiently searching over large natural language corpora with semantic variations.
method String matching based on suffix arrays, vector representation of words, dynamic corpus-aware pruning, fast exact lookup.
result Substantially lower search latency compared to existing methods on FineWeb-Edu corpus.
Parallel sentences are a relatively scarce but extremely useful resource for many applications including cross-lingual retrieval and statistical machine translation. This research explores our methodology for mining such data from previously obtained comparable corpora. The task is highly practical since non-parallel m…
Improves topic modeling for multi-collection corpora.
problem Challenges in text mining from multi-collection corpora.
method Compound Latent Dirichlet Allocation (cLDA) model and MCMC methods.
result cLDA model identifies topic proportions across multiple collections.
Grammatical Error Correction (GEC) has been recently modeled using the sequence-to-sequence framework. However, unlike sequence transduction problems such as machine translation, GEC suffers from the lack of plentiful parallel data. We describe two approaches for generating large parallel datasets for GEC using publicl…
In this paper, we attempt to improve Statistical Machine Translation (SMT) systems on a very diverse set of language pairs (in both directions): Czech - English, Vietnamese - English, French - English and German - English. To accomplish this, we performed translation model training, created adaptations of training sett…
Model learns from multiple data sources for D2T and T2D tasks.
problem Limited performance due to single-source corpora.
method Variational auto-encoder with disentangled style and content variables.
result Model outperforms single-source counterpart on multiple datasets.
ALBU improves LDA performance on small datasets.
problem Limited data impairs variational Bayes in LDA.
method Approximate Loopy Belief message passing algorithm.
result ALBU learns latent distributions more accurately than VB, especially for smaller data sets.
Neural topic models improve topic discovery in large corpora.
problem Topic discovery in large text corpora.
method Neural auto-encoder based document embedding with PLSA assumptions.
result Neural topic models outperform LDA in capturing relevant topics and perplexity.
The majority of medical documents and electronic health records (EHRs) are in text format that poses a challenge for data processing and finding relevant documents. Looking for ways to automatically retrieve the enormous amount of health and medical knowledge has always been an intriguing topic. Powerful methods have b…
In this paper, we try to explore the evolution of language through case calculations. First, we chose the novels of eleven British writers from 1400 to 2005 and found the corresponding works; Then, we use the natural language processing tool to construct the corresponding eleven corpora, and calculate the respective wo…
Word meaning changes over time, depending on linguistic and extra-linguistic factors. Associating a word's correct meaning in its historical context is a central challenge in diachronic research, and is relevant to a range of NLP tasks, including information retrieval and semantic search in historical texts. Bayesian m…
We study the problem of learning similarity functions over very large corpora using neural network embedding models. These models are typically trained using SGD with sampling of random observed and unobserved pairs, with a number of samples that grows quadratically with the corpus size, making it expensive to scale to…
Extracting common narratives from multi-author dynamic text corpora requires complex models, such as the Dynamic Author Persona (DAP) topic model. However, such models are complex and can struggle to scale to large corpora, often because of challenging non-conjugate terms. To overcome such challenges, in this paper we …
Non-negative matrix factorization (NMF) is a technique for finding latent representations of data. The method has been applied to corpora to construct topic models. However, NMF has likelihood assumptions which are often violated by real document corpora. We present a double parametric bootstrap test for evaluating the…
We study the problem of topic modeling in corpora whose documents are organized in a multi-level hierarchy. We explore a parametric approach to this problem, assuming that the number of topics is known or can be estimated by cross-validation. The models we consider can be viewed as special (finite-dimensional) instance…
Deep learning models improved sentence similarity in medical records.
problem Improving sentence similarity in electronic medical records.
method Developed models using traditional machine learning and deep learning approaches, pre-trained sentence embeddings on biomedical corpora.
result Ensembled model achieved a Person correlation coefficient of 0.8528.
Paper proposes set-valued prediction for historical POS tagging.
problem Difficult POS tagging in historical corpora due to lack of native speakers and sparse data.
method Set-valued prediction approach to allow uncertainty in tagging.
result Set-valued prediction improves POS tagging precision and robustness.
Bibliographic analysis considers the author's research areas, the citation network and the paper content among other things. In this paper, we combine these three in a topic model that produces a bibliographic model of authors, topics and documents, using a nonparametric extension of a combination of the Poisson mixed-…
Unsupervised segmentation learns features without labels, improving accuracy.
problem Discover and localize semantically meaningful categories in images without annotations.
method Separates feature learning from cluster compactification; distills unsupervised features into discrete semantic labels using a contrastive loss function.
result Significant improvement over prior state of the art on semantic segmentation challenges.
Word embeddings are a powerful approach for analyzing language and have been widely popular in numerous tasks in information retrieval and text mining. Training embeddings over huge corpora is computationally expensive because the input is typically sequentially processed and parameters are synchronously updated. Distr…
A single, stationary topic model such as latent Dirichlet allocation is inappropriate for modeling corpora that span long time periods, as the popularity of topics is likely to change over time. A number of models that incorporate time have been proposed, but in general they either exhibit limited forms of temporal var…
Study finds common poetic themes across languages over time.
problem Understanding thematic evolution in different poetic traditions.
method Applied Latent Dirichlet Allocation (LDA) to poetry corpora of four languages.
result Identified common themes and their temporal trends across poetic traditions.
ProxiModel extracts high-quality news events from news corpora.
problem Mining high-quality structured event knowledge from noisy news data.
method ProxiModel uses a proximity-network to model event correlation within and across news corpora.
result ProxiModel efficiently and effectively extracts high-quality event descriptors and attributes.
Extracts high-quality monolingual datasets from web crawl data.
problem Improving text representation quality through larger corpora.
method Automated pipeline using deduplication and language identification, augmented with filtering for high-quality documents.
result Extracted massive high-quality monolingual datasets from Common Crawl.
A new algorithm improves topic model accuracy in small data sets.
problem Limited data leads to inaccurate topic model identification.
method Developed a novel variational message passing algorithm (ALBU) for LDA.
result ALBU learns latent distributions more accurately than Variational Bayes (VB) in small data sets.
We describe a new method for visualizing topics, the distributions over terms that are automatically extracted from large text corpora using latent variable models. Our method finds significant -grams related to a topic, which are then used to help understand and interpret the underlying distribution. Compared with …
This study improves the performance of neural named entity recognition by a margin of up to 11% in F-score on the example of a low-resource language like German, thereby outperforming existing baselines and establishing a new state-of-the-art on each single open-source dataset. Rather than designing deeper and wider hy…
Proposes a multilingual email segmentation benchmark and model.
problem Lack of multilingual email zoning corpora and models.
method Analysis of existing corpora, development of multilingual benchmark, introduction of OKAPI model.
result OKAPI model achieves state-of-the-art performance in English and generalizes well to unseen languages.
Paper proposes neural networks for automatically naming assembly functions.
problem Automatically assigning names to assembly code functions.
method Formal definition of problem, baseline models (Seq2Seq, Transformer), fine-tuning neural networks.
result Neural networks can effectively predict function names in binaries, even outperforming state-of-the-art.
In this paper, we develop the continuous time dynamic topic model (cDTM). The cDTM is a dynamic topic model that uses Brownian motion to model the latent topics through a sequential collection of documents, where a "topic" is a pattern of word use that we expect to evolve over the course of the collection. We derive an…
Motivation: State-of-the-art biomedical named entity recognition (BioNER) systems often require handcrafted features specific to each entity type, such as genes, chemicals and diseases. Although recent studies explored using neural network models for BioNER to free experts from manual feature engineering, the performan…
In this paper, we present Russian language datasets in the digital humanities domain for the evaluation of word embedding techniques or similar language modeling and feature learning algorithms. The datasets are split into two task types, word intrusion and word analogy, and contain 31362 task units in total. The chara…
Irony and sarcasm are two complex linguistic phenomena that are widely used in everyday language and especially over the social media, but they represent two serious issues for automated text understanding. Many labeled corpora have been extracted from several sources to accomplish this task, and it seems that sarcasm …
The paper distinguishes between conditional and marginal processes in language models and discusses conditions for usefulness.
problem The conditional nature of language models trained on observed sequences and the need for marginal text-only processes.
method Distinguishing between full conditional language process, marginal text-only process, and model-induced distribution; analyzing assumptions of stationarity and ergodicity.
result The marginal text-only law is useful only when the observed prefix is an approximately sufficient statistic for the latent circumstances relevant to continuation.
We present a new approach to harmonic analysis that is trained to segment music into a sequence of chord spans tagged with chord labels. Formulated as a semi-Markov Conditional Random Field (semi-CRF), this joint segmentation and labeling approach enables the use of a rich set of segment-level features, such as segment…
The study analyzes convergence of random-walk embeddings in graph theory.
problem Understanding the convergence behavior of random-walk based vertex embeddings.
method Theoretical analysis of convergence in single and double limits of and .
result Proved convergence of vertex embeddings under weak assumptions and derived concentration bounds.
Deep Neural Networks for Automatic Speech Processing: A Survey from Large Corpora to Limited Dataeess.AS
Survey on DNNs for speech processing, focusing on limited data challenges.
problem Challenges in training DNNs for speech tasks with limited data.
method Overview of techniques for few-shot speech processing.
result Promising few-shot techniques for speech processing.