Paper explores using memory models to improve idiomatic language translation.
problem Translation of idiomatic language poses challenges to machine translation systems.
method Uses memory augmented models to enhance neural machine translation of idioms.
result Improves translation of idiomatic language by bridging long-distance dependencies.
We describe DyNet, a toolkit for implementing neural network models based on dynamic declaration of network structure. In the static declaration strategy that is used in toolkits like Theano, CNTK, and TensorFlow, the user first defines a computation graph (a symbolic representation of the computation), and then exampl…
The recently introduced continuous Skip-gram model is an efficient method for learning high-quality distributed vector representations that capture a large number of precise syntactic and semantic word relationships. In this paper we present several extensions that improve both the quality of the vectors and the traini…
New ASR system handles multiple languages without needing language-specific encoding.
problem Joint training of data-rich and data-scarce languages in a single model.
method Transforms all languages to a single writing system through transliteration, separating modeling and rendering.
result Language-agnostic multilingual ASR system reduces WER up to 10% over language-dependent models.
Study quantifies gender bias in language models across 7 languages.
problem Measuring gender bias in language models across multiple languages.
method Curated dataset of politicians, multilingual language models, probing language models.
result Larger language models do not show significant gender bias compared to smaller ones.
Paper reviews neurolinguistics and language technologies, emphasizing mutual enrichment.
problem Understanding brain activity during language processing.
method Brain imaging studies and natural language representations.
result Development of brain-aware natural language representations.
New technique reduces language biases in large language models.
problem Gap between language and thought in large language models.
method Proposes Language-of-Thoughts (LoT) technique to adjust order and tokens.
result Significantly reduces language biases and improves reasoning tasks.
Improved multilingual speech recognition with low latency for nine Indic languages.
problem Imbalance in training data across languages and low latency in interactive applications.
method Conditioning on a language vector and training language-specific adapter layers.
result Lower word error rate than monolingual E2E models and conventional systems.
When a bilingual student learns to solve word problems in math, we expect the student to be able to solve these problem in both languages the student is fluent in,even if the math lessons were only taught in one language. However, current representations in machine learning are language dependent. In this work, we pres…
Neural network converts speech from one language to multiple languages.
problem Creating speech in multiple languages without parallel data.
method Polyglot neural network with multiple per-language sub-networks and loss terms to preserve speaker identity.
result Convincing conversion capabilities demonstrated across three languages.
Paper introduces TrufLL for language model training without labeled data.
problem Training language models from scratch with reinforcement learning.
method Dynamic vocabulary truncation in RL for language generation.
result TrufLL successfully learns language generation policies from scratch.
Multilingual LM improves low-resource language modeling.
problem Lack of data for many languages and domains.
method Jointly trained multilingual neural language model with shared parameters.
result Significant improvements in conversational data domain with limited training data.
System identifies language of transliterated text.
problem Users struggle to understand non-native language transliterated text.
method Feature extraction of phonetic syllables using LSTM network.
result System accurately identifies language of transliterated text.
LLMs translate natural language trading intents into correct option strategies using a domain-specific language.
problem Challenges in translating natural language trading intents into correct option strategies due to the complexity of option chain data.
method Introduce Option Query Language (OQL) as a domain-specific intermediate representation to abstract option markets into high-level primitives under grammatical rules. Use LLMs as semantic parsers and validate queries by an engine.
result Significantly improves execution accuracy and logical consistency over direct baselines.
New findings show language models can't simultaneously avoid hallucinations and capture all language richness.
problem Achieving both valid output and full language richness in language generation.
method Investigates language generation within a statistical setting, focusing on consistency and breadth.
result For most collections of candidate languages, a language model cannot simultaneously avoid hallucinations and capture all language richness.
This paper proposes a new method to connect language and physical actions in reinforcement learning.
problem Connecting linguistic representations to the physical world in embodied agents.
method Language-conditioned goal generators to decouple sensorimotor learning from language acquisition.
result Agents can demonstrate a diversity of behaviors for any given instruction.
Julia accelerates machine learning in various fields with balance of efficiency and simplicity.
problem Efficiency and simplicity in machine learning algorithms.
method Developed and applied Julia language in machine learning.
result Julia balances efficiency and simplicity for machine learning.
Proposes using Wasserstein barycenter for better multilingual alignment.
problem Finding word-to-word translations between multiple languages without parallel data.
method Uses Wasserstein barycenter as a more informative pivot language, minimizing pairwise transportation costs.
result Demonstrates state-of-the-art performances on standard benchmarks.
Survey reviews code-switched speech and language processing.
problem Processing code-switched text and speech for multilingual communities.
method Reviews computational approaches and lists available resources.
result Essential for building intelligent agents that interact in multilingual settings.
The paper applies math and physics to language models, introducing entropy and geometric concepts.
problem Understanding and improving language models to approximate intelligent language.
method Formal definitions, functional analysis, topology, thermodynamics, and set theory.
result Entropy function reveals key obstacles for LLMs and offers insights into language models.
New method shows cultural transmission improves language compositionality.
problem How cultural transmission affects language emergence and compositionality.
method Introduced cultural evolutionary dynamics into language emergence by periodically replacing agents in a population.
result Languages emerge with better compositional generalization due to implicit cultural transmission.
Benchmark tests spoken language models for infant language learning.
problem Understanding how infants learn language from speech.
method Developed a language-acquisition-friendly benchmark.
result Benchmarking shows models' strengths and weaknesses.
Paper examines why LSTMs outperform GRUs in language modeling.
problem Why LSTMs outperform GRUs in language modeling and machine translation.
method Used simplified language models (Context-Free and Context-Sensitive) to analyze LSTM and GRU performance.
result LSTMs perform better due to their ability to count using cell states, while GRUs struggle.
Introduces challenges and techniques for creating machine translation for indigenous languages.
problem Limited data for machine translation of indigenous languages.
method Introduction to challenges, concepts, and techniques for creating MT systems.
result Discussion of recent advances and open questions in NLP for these languages.
Optimus pre-trains sentences in a latent space for various NLP tasks.
problem Training large-scale language models for diverse NLP tasks.
method Pre-trained Variational Autoencoder (VAE) on large text corpus, fine-tuned for various tasks.
result Optimus achieves state-of-the-art on VAE language modeling benchmarks.
ScratchGAN trains language GANs without pre-training, achieving comparable results.
problem Training GANs for natural language is challenging due to gradient estimation, optimization instability, and mode collapse.
method Used large batch sizes, dense rewards, and discriminator regularization to stabilize and improve language GANs.
result ScratchGAN performs comparably to maximum likelihood training on quality and diversity metrics.
Work shows hallucination detection by LLMs is impossible without expert feedback.
problem Detecting hallucinations in LLMs is theoretically impossible without expert-labeled feedback.
method Investigated hallucination detection using a theoretical framework inspired by language identification.
result Automated hallucination detection is impossible for most language collections without expert-labeled feedback.
Real-time spell checker adapts to new languages.
problem No real-time, language-adaptable spell checkers for non-English languages.
method Used Wikipedia and subtitles data to generate dictionaries, created noisy channel datasets, compared with industry tools.
result System performs well across 24 languages, outperforming existing tools.
Model captures author language diffusion over time.
problem Lack of author identity and temporal context in language models.
method Temporal language model conditioning on author and temporal vectors.
result Beat temporal and non-temporal baselines, learns time-varying author representations.
Improved language identification with tuplemax loss.
problem Language identification with user-specified small sets of languages.
method Replaced softmax loss with tuplemax loss.
result 2.33% error rate improvement over softmax loss.
RPP prunes large language models without sacrificing accuracy.
problem How to compress large-scale language representation models without losing performance.
method Reweighted Proximal Pruning (RPP) for large-scale language models.
result RPP maintains high accuracy for both pre-training and downstream tasks at high prune ratios.
New approach uses graphs for sign language recognition.
problem Challenges in recognizing sign language for deaf individuals.
method Spatial-Temporal Graph Convolutional Network.
result Improved sign language recognition using human skeletal movements.
Trains neural machine translation models for 5 Southern African languages.
problem Lack of shared resources and reproducible results for African languages.
method Trained neural machine translation models on publicly-available datasets.
result Provides code for training and evaluating models, aiming to spur future research.
Study explores various methods for detecting offensive language.
problem Detecting offensive language in text.
method Examined custom and off-the-shelf architectures, feature vector generation approaches.
result These methods perform well on small, noisy datasets.
Improved LID for multilingual speakers using context-aware models.
problem Low accuracy for languages spoken by multilingual speakers, especially with accented speech.
method Coarser-grained acoustic model and integration with interaction context signals.
result Average 97% accuracy across all language combinations, 60% improvement in worst-case accuracy.
Language models learn from training data and can leak private information.
problem Language models lack context understanding and can expose private data.
method Discussing the limitations of current privacy protection methods for language models.
result Existing privacy protection methods are insufficient for language models.
Large language models predict human sensory judgments across multiple modalities.
problem Determining the extent of perceptual information in language.
method State-of-the-art large language models were used to predict sensory judgments across six psychophysical datasets.
result Large language models can predict human sensory judgments across multiple modalities with significant correlation to human data.
New model shows natural language exhibits phase transition similar to physics.
problem Understanding critical properties in natural language models.
method Created a context-sensitive random language model.
result Demonstrated a Berezinskii--Kosterlitz--Thouless phase transition.
This is a lecture note for the course DS-GA 3001 <Natural Language Understanding with Distributed Representation> at the Center for Data Science , New York University in Fall, 2015. As the name of the course suggests, this lecture note introduces readers to a neural network based approach to natural language understand…
New approach decouples skill learning and language grounding for autonomous agents.
problem Autonomous acquisition of skills without external instructions and feedback.
method Language-Goal-Behavior (LGB) architecture with semantic representation.
result Decouples skill learning and language grounding, enabling diversity and strategy switching.
lamBERT learns language and actions using multimodal BERT.
problem Learning language and actions in complex environments.
method Extending BERT to multimodal representation and integrating with reinforcement learning.
result lamBERT model achieved higher rewards in multitask and transfer settings.
The paper fine-tunes language models using human preferences for natural language tasks.
problem Applying reinforcement learning to natural language tasks with human-defined rewards.
method Fine-tuning language models using reward learning techniques, leveraging generative pretraining.
result The approach achieves good results in natural language tasks like text continuation and summarization.
LORL learns object-centric representations from vision and language.
problem Learning disentangled, object-centric scene representations from vision and language.
method LORL integrates unsupervised object discovery and segmentation with language input to learn object-centric concepts.
result LORL improves unsupervised object discovery methods and aids downstream tasks.
Paper uses language models to predict MBTI personality types with high accuracy.
problem Predicting Myers-Briggs personality types from text.
method Fine-tuned BERT model for predicting MBTI types and generating personality-specific language.
result BERT model achieves high accuracy in predicting MBTI types and personality-specific language generation.
New method converts natural language commands into reward functions for robots.
problem Creating effective reward functions for autonomous machines.
method Language-conditioned reward learning (LC-RL) using inverse reinforcement learning.
result Model learns transferable rewards from natural language commands.
Transformers improve Finnish language modeling, achieving lower perplexity scores.
problem Improving language modeling for Finnish using deep learning models.
method Used BERT and Transformer-XL models in a sub-word setting, compared to LSTM.
result Transformer-XL outperforms LSTM, achieving a 27% better perplexity score.
Automates API mapping across languages with minimal manual effort.
problem Manual effort required to identify parallel program corpora and API mappings.
method Combines domain adaptation and code embedding with GANs to align vector spaces of APIs.
result Automatically identifies cross-language API mappings with less prior knowledge.
VALC provides concept-level interpretations of FLMs, overcoming word-level limitations.
problem Lack of higher-level structure interpretation in FLMs' attention weights.
method Formal definition of conceptual interpretation, variational Bayesian framework (VALC).
result VALC finds optimal language concepts for FLM predictions, providing concept-level interpretations.