Introduces challenges and techniques for creating machine translation for indigenous languages.
problem Limited data for machine translation of indigenous languages.
method Introduction to challenges, concepts, and techniques for creating MT systems.
result Discussion of recent advances and open questions in NLP for these languages.
An increase in the use of smartphones has laid to the use of the internet and social media platforms. The most commonly used social media platforms are Twitter, Facebook, WhatsApp and Instagram. People are sharing their personal experiences, reviews, feedbacks on the web. The information which is available on the web i…
Amharic is the official language of the Federal Democratic Republic of Ethiopia. There are lots of historic Amharic and Ethiopic handwritten documents addressing various relevant issues including governance, science, religious, social rules, cultures and art works which are very reach indigenous knowledge. The Amharic …
The paper develops a theory of Ehresmann structures in positive characteristic.
problem Developing a theory for Ehresmann structures in positive characteristic.
method Comparing Frobenius-Ehresmann structures with Cartan geometries and studying their equivalence.
result Formulating and proving the Ehresmann-Weil-Thurston principle for Frobenius-Ehresmann structures.
We investigate the Brazilian personal income distribution using data from National Household Sample Survey (PNAD), an annual research available by the Brazilian Institute of Geography and Statistics (IBGE). It provides general characteristics of the country's population. Using PNAD data background we also confirm the e…
Constructs a map from stable extensions to irreducible metrics on Riemann surfaces.
problem Understanding and constructing cone spherical metrics on Riemann surfaces.
method Using the theory of indigenous bundles, the construction involves developing maps and stable extensions of two line bundles.
result Generically injective map from stable extensions to irreducible metrics, with properties about effective divisors.
Footfall based biometric system is perhaps the only person identification technique which does not hinder the natural movement of an individual. This is a clear edge over all other biometric systems which require a formidable amount of human intervention and encroach upon an individual's privacy to some extent or the o…
In many developing countries intellectual property infringement and the commerce of pirate goods is an entrepreneurial activity. Digital piracy is very often the only media for having access to music, cinema, books and software. At the same time, bio-prospecting and infringement of indigenous knowledge rights by intern…
Decision tool helps manage biofouling risks for ships in the Baltic Sea.
problem Biofouling of ships causes environmental and economic issues.
method Bayesian networks to identify biofouling management strategies.
result Optimal biofouling management includes biocidal-free coating and in-water cleaning.
Deep learning automates biofouling detection in ship hull images.
problem Automating biofouling assessment from ship hull images.
method Deep learning model trained on expert-annotated images.
result Deep learning model agrees with expert annotations.
New ASR system handles multiple languages without needing language-specific encoding.
problem Joint training of data-rich and data-scarce languages in a single model.
method Transforms all languages to a single writing system through transliteration, separating modeling and rendering.
result Language-agnostic multilingual ASR system reduces WER up to 10% over language-dependent models.
Study quantifies gender bias in language models across 7 languages.
problem Measuring gender bias in language models across multiple languages.
method Curated dataset of politicians, multilingual language models, probing language models.
result Larger language models do not show significant gender bias compared to smaller ones.
Paper reviews neurolinguistics and language technologies, emphasizing mutual enrichment.
problem Understanding brain activity during language processing.
method Brain imaging studies and natural language representations.
result Development of brain-aware natural language representations.
New technique reduces language biases in large language models.
problem Gap between language and thought in large language models.
method Proposes Language-of-Thoughts (LoT) technique to adjust order and tokens.
result Significantly reduces language biases and improves reasoning tasks.
When a bilingual student learns to solve word problems in math, we expect the student to be able to solve these problem in both languages the student is fluent in,even if the math lessons were only taught in one language. However, current representations in machine learning are language dependent. In this work, we pres…
Paper introduces TrufLL for language model training without labeled data.
problem Training language models from scratch with reinforcement learning.
method Dynamic vocabulary truncation in RL for language generation.
result TrufLL successfully learns language generation policies from scratch.
System identifies language of transliterated text.
problem Users struggle to understand non-native language transliterated text.
method Feature extraction of phonetic syllables using LSTM network.
result System accurately identifies language of transliterated text.
LLMs translate natural language trading intents into correct option strategies using a domain-specific language.
problem Challenges in translating natural language trading intents into correct option strategies due to the complexity of option chain data.
method Introduce Option Query Language (OQL) as a domain-specific intermediate representation to abstract option markets into high-level primitives under grammatical rules. Use LLMs as semantic parsers and validate queries by an engine.
result Significantly improves execution accuracy and logical consistency over direct baselines.
New findings show language models can't simultaneously avoid hallucinations and capture all language richness.
problem Achieving both valid output and full language richness in language generation.
method Investigates language generation within a statistical setting, focusing on consistency and breadth.
result For most collections of candidate languages, a language model cannot simultaneously avoid hallucinations and capture all language richness.
This paper proposes a new method to connect language and physical actions in reinforcement learning.
problem Connecting linguistic representations to the physical world in embodied agents.
method Language-conditioned goal generators to decouple sensorimotor learning from language acquisition.
result Agents can demonstrate a diversity of behaviors for any given instruction.
Julia accelerates machine learning in various fields with balance of efficiency and simplicity.
problem Efficiency and simplicity in machine learning algorithms.
method Developed and applied Julia language in machine learning.
result Julia balances efficiency and simplicity for machine learning.
Neural language modeling (LM) has led to significant improvements in several applications, including Automatic Speech Recognition. However, they typically require large amounts of training data, which is not available for many domains and languages. In this study, we propose a multilingual neural language model archite…
Proposes using Wasserstein barycenter for better multilingual alignment.
problem Finding word-to-word translations between multiple languages without parallel data.
method Uses Wasserstein barycenter as a more informative pivot language, minimizing pairwise transportation costs.
result Demonstrates state-of-the-art performances on standard benchmarks.
The paper applies math and physics to language models, introducing entropy and geometric concepts.
problem Understanding and improving language models to approximate intelligent language.
method Formal definitions, functional analysis, topology, thermodynamics, and set theory.
result Entropy function reveals key obstacles for LLMs and offers insights into language models.
Multilingual end-to-end (E2E) models have shown great promise in expansion of automatic speech recognition (ASR) coverage of the world's languages. They have shown improvement over monolingual systems, and have simplified training and serving by eliminating language-specific acoustic, pronunciation, and language models…
Benchmark tests spoken language models for infant language learning.
problem Understanding how infants learn language from speech.
method Developed a language-acquisition-friendly benchmark.
result Benchmarking shows models' strengths and weaknesses.
Optimus pre-trains sentences in a latent space for various NLP tasks.
problem Training large-scale language models for diverse NLP tasks.
method Pre-trained Variational Autoencoder (VAE) on large text corpus, fine-tuned for various tasks.
result Optimus achieves state-of-the-art on VAE language modeling benchmarks.
Work shows hallucination detection by LLMs is impossible without expert feedback.
problem Detecting hallucinations in LLMs is theoretically impossible without expert-labeled feedback.
method Investigated hallucination detection using a theoretical framework inspired by language identification.
result Automated hallucination detection is impossible for most language collections without expert-labeled feedback.
Improved LID for multilingual speakers using context-aware models.
problem Low accuracy for languages spoken by multilingual speakers, especially with accented speech.
method Coarser-grained acoustic model and integration with interaction context signals.
result Average 97% accuracy across all language combinations, 60% improvement in worst-case accuracy.
Language models learn from training data and can leak private information.
problem Language models lack context understanding and can expose private data.
method Discussing the limitations of current privacy protection methods for language models.
result Existing privacy protection methods are insufficient for language models.
Large language models predict human sensory judgments across multiple modalities.
problem Determining the extent of perceptual information in language.
method State-of-the-art large language models were used to predict sensory judgments across six psychophysical datasets.
result Large language models can predict human sensory judgments across multiple modalities with significant correlation to human data.
New model shows natural language exhibits phase transition similar to physics.
problem Understanding critical properties in natural language models.
method Created a context-sensitive random language model.
result Demonstrated a Berezinskii--Kosterlitz--Thouless phase transition.
This is a lecture note for the course DS-GA 3001 <Natural Language Understanding with Distributed Representation> at the Center for Data Science , New York University in Fall, 2015. As the name of the course suggests, this lecture note introduces readers to a neural network based approach to natural language understand…
We present a TTS neural network that is able to produce speech in multiple languages. The proposed network is able to transfer a voice, which was presented as a sample in a source language, into one of several target languages. Training is done without using matching or parallel data, i.e., without samples of the same …
New approach decouples skill learning and language grounding for autonomous agents.
problem Autonomous acquisition of skills without external instructions and feedback.
method Language-Goal-Behavior (LGB) architecture with semantic representation.
result Decouples skill learning and language grounding, enabling diversity and strategy switching.
lamBERT learns language and actions using multimodal BERT.
problem Learning language and actions in complex environments.
method Extending BERT to multimodal representation and integrating with reinforcement learning.
result lamBERT model achieved higher rewards in multitask and transfer settings.
LORL learns object-centric representations from vision and language.
problem Learning disentangled, object-centric scene representations from vision and language.
method LORL integrates unsupervised object discovery and segmentation with language input to learn object-centric concepts.
result LORL improves unsupervised object discovery methods and aids downstream tasks.
Recent work has studied the emergence of language among deep reinforcement learning agents that must collaborate to solve a task. Of particular interest are the factors that cause language to be compositional -- i.e., express meaning by combining words which themselves have meaning. Evolutionary linguists have found th…
Transformers improve Finnish language modeling, achieving lower perplexity scores.
problem Improving language modeling for Finnish using deep learning models.
method Used BERT and Transformer-XL models in a sub-word setting, compared to LSTM.
result Transformer-XL outperforms LSTM, achieving a 27% better perplexity score.
VALC provides concept-level interpretations of FLMs, overcoming word-level limitations.
problem Lack of higher-level structure interpretation in FLMs' attention weights.
method Formal definition of conceptual interpretation, variational Bayesian framework (VALC).
result VALC finds optimal language concepts for FLM predictions, providing concept-level interpretations.
The task of translating between programming languages differs from the challenge of translating natural languages in that programming languages are designed with a far more rigid set of structural and grammatical rules. Previous work has used a tree-to-tree encoder/decoder model to take advantage of the inherent tree s…
Study finds phase transition in context-sensitive language model with short-range interactions.
problem Understanding phase transitions in language models with short-range interactions.
method Constructed a random language model with short-range interactions and investigated its statistical properties.
result Phase transition occurs in context-sensitive language models with constant context length.
We propose multi-way, multilingual neural machine translation. The proposed approach enables a single neural translation model to translate between multiple languages, with a number of parameters that grows only linearly with the number of languages. This is made possible by having a single attention mechanism that is …
We present a novel language adaptable spell checking system which detects spelling errors and suggests context sensitive corrections in real-time. We show that our system can be extended to new languages with minimal language-specific processing. Available literature majorly discusses spell checkers for English but the…
Improved robot navigation using multi-head attention for natural language instructions.
problem Improving robot navigation in unfamiliar environments.
method Proposes a multi-head attention mechanism blending layer in a neural network model.
result Significant performance gains in translating instructions for unseen environments.
Text documents are structured on multiple levels of detail: individual words are related by syntax, but larger units of text are related by discourse structure. Existing language models generally fail to account for discourse structure, but it is crucial if we are to have language models that reward coherence and gener…
Language helps RL agents learn complex relational and causal structures.
problem Learning relational and causal structure in complex environments.
method Training RL agents to predict language descriptions and explanations.
result Language aids agents in learning challenging relational and causal tasks.
The task of determining a speaker's native language based only on his speeches in a second language is known as Native Language Identification or NLI. Due to its increasing applications in various domains of speech signal processing, this has emerged as an important research area in recent times. In this paper we have …