New ASR system handles multiple languages without needing language-specific encoding.
problem Joint training of data-rich and data-scarce languages in a single model.
method Transforms all languages to a single writing system through transliteration, separating modeling and rendering.
result Language-agnostic multilingual ASR system reduces WER up to 10% over language-dependent models.
Language models learn from training data and can leak private information.
problem Language models lack context understanding and can expose private data.
method Discussing the limitations of current privacy protection methods for language models.
result Existing privacy protection methods are insufficient for language models.
Neural language modeling (LM) has led to significant improvements in several applications, including Automatic Speech Recognition. However, they typically require large amounts of training data, which is not available for many domains and languages. In this study, we propose a multilingual neural language model archite…
Introduces challenges and techniques for creating machine translation for indigenous languages.
problem Limited data for machine translation of indigenous languages.
method Introduction to challenges, concepts, and techniques for creating MT systems.
result Discussion of recent advances and open questions in NLP for these languages.
iPrompt uses LLMs to generate natural-language explanations of data patterns.
problem Finding and explaining patterns in data using natural language.
method Interpretable autoprompting (iPrompt) that generates natural-language explanations based on LLMs.
result iPrompt can accurately find and explain data patterns, improving upon human-written prompts.
Chronologically consistent models maintain accuracy with time-restricted data.
problem Training data introduces lookahead bias and training leakage in large language models.
method ChronoBERT and ChronoGPT trained with only available data at each time point.
result Models achieve strong performance and competitive with larger models, mitigating lookahead bias.
LLMs translate natural language trading intents into correct option strategies using a domain-specific language.
problem Challenges in translating natural language trading intents into correct option strategies due to the complexity of option chain data.
method Introduce Option Query Language (OQL) as a domain-specific intermediate representation to abstract option markets into high-level primitives under grammatical rules. Use LLMs as semantic parsers and validate queries by an engine.
result Significantly improves execution accuracy and logical consistency over direct baselines.
Multilingual end-to-end (E2E) models have shown great promise in expansion of automatic speech recognition (ASR) coverage of the world's languages. They have shown improvement over monolingual systems, and have simplified training and serving by eliminating language-specific acoustic, pronunciation, and language models…
This research creates and classifies datasets for Setswana and Sepedi news headlines.
problem Challenges in creating and preparing datasets for low-resourced languages.
method Investigates an approach for data augmentation tailored to low resource languages.
result Improves classification performance on news topic classification task.
Paper reviews neurolinguistics and language technologies, emphasizing mutual enrichment.
problem Understanding brain activity during language processing.
method Brain imaging studies and natural language representations.
result Development of brain-aware natural language representations.
SentenceMIM learns rich latent representations for variable-length language data.
problem Challenges in learning VAEs for variable-length language data, especially posterior collapse.
method Probabilistic auto-encoder trained with Mutual Information Machine (MIM) learning.
result SentenceMIM learns informative latent representations with high mutual information.
Because it is not feasible to collect training data for every language, there is a growing interest in cross-lingual transfer learning. In this paper, we systematically explore zero-shot cross-lingual transfer learning on reading comprehension tasks with a language representation model pre-trained on multi-lingual corp…
Unified BERT model improves NER across multiple languages.
problem Language-specific NER models limit data extraction.
method Jointly trained multilingual BERT with regularization.
result Unified model outperforms monolingual models on various datasets.
The Widrow-Hoff rule simplifies language data simulation.
problem Simulating language phenomena computationally.
method Implementation and application of the Widrow-Hoff rule.
result The Widrow-Hoff rule offers new perspectives on language simulation.
Study examines bias in language models across multiple languages.
problem Assessing bias in language models across different languages.
method Semi-automatically translated data sets into multiple languages, analyzed mono- and multilingual models.
result Notable differences in bias across languages, with Turkish models showing least stereotypes.
StonkBERT predicts stock price movements using company text data.
problem Can language models predict medium-run stock price movements?
method Fine-tuning transformer-based language models (BERT) on company text data (news articles, blogs, annual reports) for stock price performance classification.
result StonkBERT shows substantial improvement in predictive accuracy compared to traditional models, with news articles providing the best results.
Extracts high-quality monolingual datasets from web crawl data.
problem Improving text representation quality through larger corpora.
method Automated pipeline using deduplication and language identification, augmented with filtering for high-quality documents.
result Extracted massive high-quality monolingual datasets from Common Crawl.
Paper introduces TrufLL for language model training without labeled data.
problem Training language models from scratch with reinforcement learning.
method Dynamic vocabulary truncation in RL for language generation.
result TrufLL successfully learns language generation policies from scratch.
New research shows larger language models improve data processing for diverse entries.
problem Optimizing data processing for tables with diverse string entries.
method Analytical tasks on tables with varying language model sizes and a fuzzy join benchmark.
result Larger language models improve data processing for diverse entries, but fine-tuning is necessary.
SSMBA generates synthetic data to improve robustness in natural language tasks.
problem Improving out-of-domain generalization of models trained on natural language data.
method SSMBA uses corruption and reconstruction functions to generate synthetic data points on the manifold assumption.
result SSMBA consistently outperforms existing methods on robustness benchmarks across multiple tasks and datasets.
Fine-tunes language models with captions for differential equation solving.
problem Overreliance on function data in operator learning.
method Integrates human knowledge through captions and fine-tunes language models.
result Significantly enhanced performance and reduced function data requirements.
This paper reviews sentiment analysis on Indian languages.
problem Understanding sentiment in multilingual web data.
method Reviews and discusses approaches for sentiment analysis on Indian languages.
result Challenges in sentiment analysis on indigenous languages.
Reduces gender bias in patient notes while maintaining medical classification accuracy.
problem Bias in natural language processing of patient notes.
method Identifying and removing gendered language using BERT-based classifiers, then augmenting data to maintain performance.
result Minimal degradation in health condition classification tasks with data augmentation.
System identifies language of transliterated text.
problem Users struggle to understand non-native language transliterated text.
method Feature extraction of phonetic syllables using LSTM network.
result System accurately identifies language of transliterated text.
OT domain adaptation improves aphasia detection across languages.
problem Detecting aphasia in low-resource languages with limited data.
method Utilized OT domain adaptation to map linguistic features across multiple languages.
result OT domain adaptation significantly improved F1 scores for French and Mandarin aphasia detection.
Proposes a method to align language and image data.
problem Aligning language and image data for better understanding.
method Uses triplet loss to learn consistent embeddings of language and images.
result Outperforms four baselines across multiple metrics.
This is a lecture note for the course DS-GA 3001 <Natural Language Understanding with Distributed Representation> at the Center for Data Science , New York University in Fall, 2015. As the name of the course suggests, this lecture note introduces readers to a neural network based approach to natural language understand…
We present a TTS neural network that is able to produce speech in multiple languages. The proposed network is able to transfer a voice, which was presented as a sample in a source language, into one of several target languages. Training is done without using matching or parallel data, i.e., without samples of the same …
This research explores discrete diffusion models for natural language generation.
problem Challenges in applying diffusion models to discrete data, especially natural language.
method Investigates Discrete Denoising Diffusion Probabilistic Model (D3PM) and compares it with autoregressive models.
result Discrete diffusion models achieve better processing speed than autoregressive models.
POUF fine-tunes large models without labeled data.
problem Lack of labeled data for fine-tuning large pre-trained models.
method Prompt-oriented unsupervised fine-tuning.
result Consistent improvements across various tasks.
Proposes using Wasserstein barycenter for better multilingual alignment.
problem Finding word-to-word translations between multiple languages without parallel data.
method Uses Wasserstein barycenter as a more informative pivot language, minimizing pairwise transportation costs.
result Demonstrates state-of-the-art performances on standard benchmarks.
StockTime predicts stock prices more accurately using LLMs and time series data.
problem Challenges in integrating time series data and natural language for stock price prediction.
method StockTime is a specialized LLM architecture that integrates textual and time series data to predict stock prices.
result StockTime outperforms recent LLMs in predicting stock prices with more accuracy.
This study applies neural models to automatically recognize medical entities from natural language.
problem Automated recognition of medical entities from natural language is complex and time-consuming.
method Utilizes deep neural sequence models trained on a large dataset of death certificates.
result Deep neural models can efficiently recognize medical entities from natural language.
New bounds show large language models can generalize beyond training data.
problem Generalization of large language models beyond training data.
method Compression bound derivation using prediction smoothing and SubLoRA.
result Large language models can discover generalizable regularities.
Improved ASR for English-isiZulu code-switched speech with semi-supervised training.
problem Improving ASR for code-switched speech between English and isiZulu.
method Semi-supervised training using automatic transcription of multilingual speech data.
result Semi-supervised training achieved significant WER reduction in ASR performance.
Neural network framework for language recognition considers sequence information and improves accuracy.
problem Challenging task of automatic language identification in noisy conditions.
method Proposes a neural network framework with bidirectional LSTM and attention modeling for relevance weighting.
result Significant improvements over conventional methods in noisy conditions and multi-speaker speech.
Paper shows pretrained models can categorize Thai social media content effectively.
problem Lack of labeled data for Thai social media content classification.
method Pretrained language models on a large noisy Thai social media corpus, fine-tuned for downstream tasks.
result State-of-the-art results achieved on Thai social text categorization tasks.
A framework for flagging content with limited data.
problem Content flagging with scarce target-language data.
method Nearest-neighbor architecture using Transformer representations.
result Significant performance improvements over prior work.
DeepSubQE estimates translation quality for subtitles, improving on existing methods.
problem Hard quality estimation for subtitle translations due to language variations.
method Proposes DeepSubQE, a hybrid network combining semantic and syntactic features.
result DeepSubQE outperforms existing methods by significant margin.
Large language models predict human sensory judgments across multiple modalities.
problem Determining the extent of perceptual information in language.
method State-of-the-art large language models were used to predict sensory judgments across six psychophysical datasets.
result Large language models can predict human sensory judgments across multiple modalities with significant correlation to human data.
Language models trained on chess board states outperform those on moves, even with causal masking.
problem Applying causal masking to spatial data for training unimodal language models.
method Trained bidirectional and causal self-attention models on both spatial (board-based) and sequential (move-based) chess data.
result Models trained on spatial board states achieve stronger playing strength than those trained on sequential data, even with causal masking.
New method boosts performance of diffusion models on discrete data like natural language.
problem Performance of diffusion models on discrete data like natural language is poor.
method Proposes score entropy, a novel loss that extends score matching to discrete spaces.
result Significantly boosts performance on language modeling tasks.
We present a novel language adaptable spell checking system which detects spelling errors and suggests context sensitive corrections in real-time. We show that our system can be extended to new languages with minimal language-specific processing. Available literature majorly discusses spell checkers for English but the…
Analysis of model updates reveals sensitive data leaks.
problem Information leakage during model updates.
method Differential analysis of language model snapshots.
result New metrics (differential score, differential rank) reveal sensitive data leaks.
Code-switching, the alternation of languages within a conversation or utterance, is a common communicative phenomenon that occurs in multilingual communities across the world. This survey reviews computational approaches for code-switched Speech and Natural Language Processing. We motivate why processing code-switched …
Transformer models can solve complex math problems with less data.
problem Solving complex symbolic mathematics problems with limited data.
method Pretrain transformer models on language translation tasks and fine-tune for symbolic math.
result Pretrained transformer models achieve comparable accuracy to state-of-the-art models with less data.
Improved language generation with faster sampling speed.
problem Speed and coherence issues in autoregressive language models.
method Introduces Neural Flow Diffusion Models (NFDM) for discrete state spaces.
result Substantially reduces likelihood gap with autoregressive models.
Forward translation improves neural machine translation for sentences originally in source language.
problem Improving neural machine translation quality using synthetic data.
method Case study with French-English news translation, separating test sets by original language, analyzing domains, translationese, and noise.
result Forward translation delivers superior gains on sentences originally in source language, complementing back-translation on target language sentences.