Parallel sentences are a relatively scarce but extremely useful resource for many applications including cross-lingual retrieval and statistical machine translation. This research explores our new methodologies for mining such data from previously obtained comparable corpora. The task is highly practical since non-para…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “parallel corpora”
We propose a flexible framework for spectral conversion (SC) that facilitates training with unaligned corpora. Many SC frameworks require parallel corpora, phonetic alignments, or explicit frame-wise correspondence for learning conversion functions or for synthesizing a target spectrum with the aid of alignments. Howev…
Multi-domain machine translation enhancements by parallel data extraction from comparable corporacs.CL
Parallel texts are a relatively rare language resource, however, they constitute a very useful research material with a wide range of applications. This study presents and analyses new methodologies we developed for obtaining such data from previously built comparable corpora. The methodologies are automatic and unsupe…
Parallel sentences are a relatively scarce but extremely useful resource for many applications including cross-lingual retrieval and statistical machine translation. This research explores our methodology for mining such data from previously obtained comparable corpora. The task is highly practical since non-parallel m…
Two methods generate parallel data for GEC, improving neural models' performance.
problem Lack of parallel data for GEC.
method Two approaches to generate large parallel datasets from Wikipedia data.
result Neural GEC models trained on generated corpora perform similarly and surpass state-of-the-art.
We develop the multilingual topic model for unaligned text (MuTo), a probabilistic model of text that is designed to analyze corpora composed of documents in two languages. From these documents, MuTo uses stochastic EM to simultaneously discover both a matching between the languages and multilingual latent topics. We d…
A new model improves machine translation by accounting for translation ambiguity.
problem Translation ambiguity in parallel corpora leads to inconsistent models.
method A deep generative model with latent variables to handle local variation.
result The model consistently outperforms existing baselines on multiple language pairs.
Topic models, and more specifically the class of Latent Dirichlet Allocation (LDA), are widely used for probabilistic modeling of text. MCMC sampling from the posterior distribution is typically performed using a collapsed Gibbs sampler. We propose a parallel sparse partially collapsed Gibbs sampler and compare its spe…
Extracts parallel sentences for machine translation.
problem Data sparsity in multilingual natural language processing.
method Bidirectional recurrent neural network approach.
result Significant improvements in machine translation performance.
In this paper, we attempt to improve Statistical Machine Translation (SMT) systems on a very diverse set of language pairs (in both directions): Czech - English, Vietnamese - English, French - English and German - English. To accomplish this, we performed translation model training, created adaptations of training sett…
Latent Dirichlet Allocation (LDA) is a topic model widely used in natural language processing and machine learning. Most approaches to training the model rely on iterative algorithms, which makes it difficult to run LDA on big corpora that are best analyzed in parallel and distributed computational environments. Indeed…
Framework converts singer identity and vocal technique from non-parallel corpora.
problem Converts singer identity and vocal technique from non-parallel corpora.
method Uses variational autoencoders with separate encoders for singer identity and vocal technique.
result Successfully disentangles and converts singer identity and vocal technique.
This research explores the effects of various training settings on a Polish to English Statistical Machine Translation system for spoken language. Various elements of the TED, Europarl, and OPUS parallel text corpora were used as the basis for training of language models, for development, tuning and testing of the tran…
Unsupervised comparable corpora preparation and exploration for bi-lingual translation equivalentscs.CL
The multilingual nature of the world makes translation a crucial requirement today. Parallel dictionaries constructed by humans are a widely-available resource, but they are limited and do not provide enough coverage for good quality translation purposes, due to out-of-vocabulary words and neologisms. This motivates th…
Modeling lead-lag relationship between two text corpora for improved topic modeling.
problem Recognizing the relationship between multiple text corpora for better topic modeling.
method Proposed a jointly dynamic topic model and embedding extension for large-scale text corpus.
result The proposed model can well recognize the lead-lag relationship between two text corpora and improve topic learning.
Automates API mapping across languages with minimal manual effort.
problem Manual effort required to identify parallel program corpora and API mappings.
method Combines domain adaptation and code embedding with GANs to align vector spaces of APIs.
result Automatically identifies cross-language API mappings with less prior knowledge.
This research explores the effects of various training settings from Polish to English Statistical Machine Translation system for spoken language. Various elements of the TED parallel text corpora for the IWSLT 2013 evaluation campaign were used as the basis for training of language models, and for development, tuning …
Efficiently trains large corpora models without sampling.
problem Training neural network embedding models on very large corpora using SGD is expensive.
method Proposes new methods to train models without sampling unobserved pairs, using Gramian estimation and variance reduction schemes.
result Significant improvement in training time and generalization quality compared to traditional methods.
Ultra-fast search algorithm for trillion-scale corpora with semantic flexibility.
problem Efficiently searching over large natural language corpora with semantic variations.
method String matching based on suffix arrays, vector representation of words, dynamic corpus-aware pruning, fast exact lookup.
result Substantially lower search latency compared to existing methods on FineWeb-Edu corpus.
Paper proposes continual learning for sentence encoders.
problem Optimize sentence encoders for new corpora while maintaining old corpus accuracy.
method Initialize encoders with corpus-independent features, update using Boolean operations of conceptor matrices.
result Proposed sentence encoder can continually learn features from new corpora.
New method uses sequential statistics for classifier parameter estimation without paired data.
problem Removing need for paired input-output data in classification problems.
method Introduces Caesar Cipher analogy and novel loss function for unsupervised learning.
result Estimates classifier parameters using sequential statistics without paired data.
Model learns from multiple data sources for D2T and T2D tasks.
problem Limited performance due to single-source corpora.
method Variational auto-encoder with disentangled style and content variables.
result Model outperforms single-source counterpart on multiple datasets.
DAPPER improves scalability of DAP topic model for large corpora.
problem Scaling complex models like DAP to large text corpora.
method Adapted approximate inference techniques for DAP, developing CVI-based EM.
result Significant improvements in model fit and training time without compromising structure.
This new research explores the effects of various training methods on a Polish to English Statistical Machine Translation system for medical texts. Various elements of the EMEA parallel text corpora from the OPUS project were used as the basis for training of phrase tables and language models and for development, tunin…
Improves topic modeling for multi-collection corpora.
problem Challenges in text mining from multi-collection corpora.
method Compound Latent Dirichlet Allocation (cLDA) model and MCMC methods.
result cLDA model identifies topic proportions across multiple collections.
Improved neural NER by optimizing large corpora for German.
problem Low-resource language named entity recognition.
method Optimized large corpora, lemmatization, part-of-speech tagging, and detailed optimization.
result Up to 11% improvement in F-score on German NER tasks.
Global anchor method detects language shifts and domain adaptation.
problem Detecting corpus-level language shifts and domain adaptation.
method Global anchor method for comparing word embeddings.
result Global anchor method is superior to alignment method in applicability and implementation.
This paper proposes an efficient method to train word embeddings for large corpora without synchronization.
problem Training word embeddings for large text corpora is computationally expensive and requires synchronization.
method Partition the input space instead of the vocabulary size, using asynchronous training without parameter synchronization.
result Comparable and up to 45% performance improvement in NLP benchmarks with 1/10 the training time.
BERT model improves cross-lingual document retrieval.
problem Improving cross-lingual information retrieval between English queries and foreign documents.
method Introduced a deep relevance matching model based on BERT, trained with weak supervision using CLIR data.
result Model outperforms competitive baseline approaches in retrieving Lithuanian documents against short English queries.
This research investigates the Statistical Machine Translation approaches to translate speech in real time automatically. Such systems can be used in a pipeline with speech recognition and synthesis software in order to produce a real-time voice communication system between foreigners. We obtained three main data sets …
Neural topic models improve topic discovery in large corpora.
problem Topic discovery in large text corpora.
method Neural auto-encoder based document embedding with PLSA assumptions.
result Neural topic models outperform LDA in capturing relevant topics and perplexity.
A new clustering method using autoencoders for improved data representation.
problem Improving clustering of complex data like images and text.
method DAMIC algorithm based on a mixture of deep autoencoders.
result Significant improvement over state-of-the-art methods on image and text corpora.
The majority of medical documents and electronic health records (EHRs) are in text format that poses a challenge for data processing and finding relevant documents. Looking for ways to automatically retrieve the enormous amount of health and medical knowledge has always been an intriguing topic. Powerful methods have b…
This work tackles domain shift in speech emotion recognition by proposing class-wise adversarial domain adaptation.
problem Domain shift between corpora poses a challenge for speech emotion recognition, especially for positive/negative emotions.
method Class-wise adversarial domain adaptation to reduce shift between different corpora.
result Our method is effective even with limited target labeled examples, as demonstrated on EMODB and Aibo corpora.
Non-negative matrix factorization (NMF) is a technique for finding latent representations of data. The method has been applied to corpora to construct topic models. However, NMF has likelihood assumptions which are often violated by real document corpora. We present a double parametric bootstrap test for evaluating the…
The paper creates a language evolution tree using word vectors from historical novels.
problem Exploring the evolution of language through historical texts.
method Constructed word vectors from novels, combined them, and used hierarchical clustering.
result Discovered a specific language evolution tree that reflects the year of the corpus.
We study the problem of topic modeling in corpora whose documents are organized in a multi-level hierarchy. We explore a parametric approach to this problem, assuming that the number of topics is known or can be estimated by cross-validation. The models we consider can be viewed as special (finite-dimensional) instance…
When building large-scale machine learning (ML) programs, such as big topic models or deep neural nets, one usually assumes such tasks can only be attempted with industrial-sized clusters with thousands of nodes, which are out of reach for most practitioners or academic researchers. We consider this challenge in the co…
Paper proposes set-valued prediction for historical POS tagging.
problem Difficult POS tagging in historical corpora due to lack of native speakers and sparse data.
method Set-valued prediction approach to allow uncertainty in tagging.
result Set-valued prediction improves POS tagging precision and robustness.
Utilizing Character and Word Embeddings for Text Normalization with Sequence-to-Sequence Modelscs.CL
Neural model improves text normalization for non-English languages.
problem Improving text normalization in non-English languages with limited data.
method Sequence-to-sequence model with character and word embeddings, using pre-trained word embeddings with subword information.
result Achieved state-of-the-art F1 score on Arabic language correction dataset.
Bibliographic analysis considers the author's research areas, the citation network and the paper content among other things. In this paper, we combine these three in a topic model that produces a bibliographic model of authors, topics and documents, using a nonparametric extension of a combination of the Poisson mixed-…
Unlocking Noisy Real-World Corpora for Foundation Model Pre-Training via Quality-Aware Tokenizationcs.AI
QA-Token improves tokenization for noisy data, boosting model performance.
problem Tokenization ignores data quality, limiting model effectiveness on noisy corpora.
method QA-Token combines signal quality with vocabulary construction through bilevel optimization and reinforcement learning.
result QA-Token achieves state-of-the-art performance on genomic and financial datasets.
Unsupervised segmentation learns features without labels, improving accuracy.
problem Discover and localize semantically meaningful categories in images without annotations.
method Separates feature learning from cluster compactification; distills unsupervised features into discrete semantic labels using a contrastive loss function.
result Significant improvement over prior state of the art on semantic segmentation challenges.
Method converts emotions in nonparallel speech data.
problem Lack of parallel data for speech emotion conversion.
method Unsupervised style transfer technique for nonparallel training.
result Effectiveness demonstrated on nonparallel corpora with four emotions.
Structural correspondence learning (SCL) is an effective method for cross-lingual sentiment classification. This approach uses unlabeled documents along with a word translation oracle to automatically induce task specific, cross-lingual correspondences. It transfers knowledge through identifying important features, i.e…
Study finds common poetic themes across languages over time.
problem Understanding thematic evolution in different poetic traditions.
method Applied Latent Dirichlet Allocation (LDA) to poetry corpora of four languages.
result Identified common themes and their temporal trends across poetic traditions.
Model learns multilingual word representations robust to noise.
problem Learning multilingual word representations in noisy environments.
method Fit a generative latent variable model to a multilingual dictionary.
result Competitive multilingual embeddings across various tasks.
CGMH uses Metropolis-Hastings sampling to generate sentences with complex constraints.
problem Generating sentences with specific constraints while maintaining fluency and naturalness.
method CGMH employs Metropolis-Hastings sampling for constrained sentence generation.
result CGMH outperforms previous methods in various constrained sentence generation tasks.