Improved ASR-free keyword spotting in under-resourced languages.
problem Dynamic time warping for keyword spotting in languages with limited resources.
method Multilingual bottleneck extractor and correspondence autoencoder integration.
result More than 11% absolute improvement in ROC AUC over MFCCs.
Paper benchmarks Bengali language classification tasks using MConv-LSTM network.
problem Lack of computational resources for NLP tasks in under-resourced languages like Bengali.
method Built three datasets, BengFastText word embeddings, and MConv-LSTM network for hate speech detection, document classification, and sentiment analysis.
result BengFastText yields up to 92.30%, 82.25%, and 90.45% F1-scores in document classification, sentiment analysis, and hate speech detection respectively.
We present our first efforts in building an automatic speech recognition system for Somali, an under-resourced language, using 1.57 hrs of annotated speech for acoustic model training. The system is part of an ongoing effort by the United Nations (UN) to implement keyword spotting systems supporting humanitarian relief…
Paper tackles Arabic question similarity, outperforming state-of-the-art.
problem Detecting semantically similar questions in Arabic is challenging.
method Utilizes contextualized word representations (ELMo embeddings) trained on MSA and dialectic sentences, combined with a pairwise similarity layer.
result Achieves 93% F1-score on Modern Standard Arabic benchmark and 82% on dialectical benchmark.
Survey on DNNs for speech processing, focusing on limited data challenges.
problem Challenges in training DNNs for speech tasks with limited data.
method Overview of techniques for few-shot speech processing.
result Promising few-shot techniques for speech processing.
We consider multilingual bottleneck features (BNFs) for nearly zero-resource keyword spotting. This forms part of a United Nations effort using keyword spotting to support humanitarian relief programmes in parts of Africa where languages are severely under-resourced. We use 1920 isolated keywords (40 types, 34 minutes)…
Paper evaluates Arabic question similarity, 9 teams participated.
problem Predicting semantic similarity between Arabic questions.
method 9 teams participated in a shared task to predict semantic similarity.
result 9 teams participated in the task, results made publicly available.
LinearAPT optimizes decision-making under resource constraints for a linear threshold problem.
problem Optimizing sequential decisions with a linear threshold under resource limitations.
method LinearAPT, an adaptive algorithm for fixed-budget TLB problem.
result LinearAPT achieves theoretical upper bounds and robust performance on various datasets.
New method creates coresets for deep neural networks efficiently.
problem Efficiently handling large data streams with limited resources.
method Cardinality-constrained bilevel optimization for deep neural networks.
result Demonstrated efficient generation of coresets for deep neural networks.
Subjective expected utility theory assumes that decision-makers possess unlimited computational resources to reason about their choices; however, virtually all decisions in everyday life are made under resource constraints - i.e. decision-makers are bounded in their rationality. Here we experimentally tested the predic…
The robustness and integrity of IP networks require efficient tools for traffic monitoring and analysis, which scale well with traffic volume and network size. We address the problem of optimal large-scale flow monitoring of computer networks under resource constraints. We propose a stochastic optimization framework wh…
Bounded rationality, that is, decision-making and planning under resource limitations, is widely regarded as an important open problem in artificial intelligence, reinforcement learning, computational neuroscience and economics. This paper offers a consolidated presentation of a theory of bounded rationality based on i…
Neural Architecture Search aims at automatically finding neural architectures that are competitive with architectures designed by human experts. While recent approaches have achieved state-of-the-art predictive performance for image recognition, they are problematic under resource constraints for two reasons: (1)the ne…
DSA efficiently allocates sparsity across layers for budgeted pruning.
problem Efficiently distributing resources (sparsity) across layers in pruning under resource constraints.
method DSA uses differentiable pruning to find continuous layer-wise pruning ratios via gradient-based optimization.
result DSA achieves superior performance and significantly reduces the time cost of pruning.
We consider the noisy power method algorithm, which has wide applications in machine learning and statistics, especially those related to principal component analysis (PCA) under resource (communication, memory or privacy) constraints. Existing analysis of the noisy power method shows an unsatisfactory dependency over …
New algorithm tackles dynamic assortment optimization with knapsack constraints.
problem Optimizing retailer's assortment decisions under resource constraints with multi-nomial choice modeling.
method Epoch-based re-solving algorithm that transforms MNL's fractional structure into a linear program with slack variables.
result Regret scales logarithmically with time horizon and resource capacities.
New framework guides resource usage to achieve sublinear regret in adversarial settings.
problem Achieving sublinear regret in online decision making with changing reward and cost distributions.
method General primal-dual methods guided by spending plans that ensure balanced resource usage.
result Achieves sublinear regret with respect to spending plans that balance resource usage.
Improved inference efficiency for complex simulations.
problem Challenges in performing inference under resource-intensive stochastic simulators.
method Active sequential neural posterior estimation (ASNPE) integrating active learning into posterior estimation.
result Improved sample efficiency with low computational overhead.
System predicts HIV patients at risk of dropping out of care.
problem High non-adherence and dropout rates among HIV patients.
method Predictive machine learning model based on resource constraints, stability, and fairness.
result Model performs 3x better than baseline for clinical use and 2.3x better for city-wide use.
New ASR system handles multiple languages without needing language-specific encoding.
problem Joint training of data-rich and data-scarce languages in a single model.
method Transforms all languages to a single writing system through transliteration, separating modeling and rendering.
result Language-agnostic multilingual ASR system reduces WER up to 10% over language-dependent models.
Study quantifies gender bias in language models across 7 languages.
problem Measuring gender bias in language models across multiple languages.
method Curated dataset of politicians, multilingual language models, probing language models.
result Larger language models do not show significant gender bias compared to smaller ones.
Paper reviews neurolinguistics and language technologies, emphasizing mutual enrichment.
problem Understanding brain activity during language processing.
method Brain imaging studies and natural language representations.
result Development of brain-aware natural language representations.
New technique reduces language biases in large language models.
problem Gap between language and thought in large language models.
method Proposes Language-of-Thoughts (LoT) technique to adjust order and tokens.
result Significantly reduces language biases and improves reasoning tasks.
SAMS-VAE models cellular perturbations using sparse additive mechanisms.
problem Modeling effects of diverse interventions on cells.
method Sparse Additive Mechanism Shift Variational Autoencoder (SAMS-VAE).
result SAMS-VAE identifies disentangled, perturbation-specific latent subspaces.
Improved multilingual speech recognition with low latency for nine Indic languages.
problem Imbalance in training data across languages and low latency in interactive applications.
method Conditioning on a language vector and training language-specific adapter layers.
result Lower word error rate than monolingual E2E models and conventional systems.
When a bilingual student learns to solve word problems in math, we expect the student to be able to solve these problem in both languages the student is fluent in,even if the math lessons were only taught in one language. However, current representations in machine learning are language dependent. In this work, we pres…
Neural network converts speech from one language to multiple languages.
problem Creating speech in multiple languages without parallel data.
method Polyglot neural network with multiple per-language sub-networks and loss terms to preserve speaker identity.
result Convincing conversion capabilities demonstrated across three languages.
Paper introduces TrufLL for language model training without labeled data.
problem Training language models from scratch with reinforcement learning.
method Dynamic vocabulary truncation in RL for language generation.
result TrufLL successfully learns language generation policies from scratch.
Multilingual LM improves low-resource language modeling.
problem Lack of data for many languages and domains.
method Jointly trained multilingual neural language model with shared parameters.
result Significant improvements in conversational data domain with limited training data.
System identifies language of transliterated text.
problem Users struggle to understand non-native language transliterated text.
method Feature extraction of phonetic syllables using LSTM network.
result System accurately identifies language of transliterated text.
LLMs translate natural language trading intents into correct option strategies using a domain-specific language.
problem Challenges in translating natural language trading intents into correct option strategies due to the complexity of option chain data.
method Introduce Option Query Language (OQL) as a domain-specific intermediate representation to abstract option markets into high-level primitives under grammatical rules. Use LLMs as semantic parsers and validate queries by an engine.
result Significantly improves execution accuracy and logical consistency over direct baselines.
New findings show language models can't simultaneously avoid hallucinations and capture all language richness.
problem Achieving both valid output and full language richness in language generation.
method Investigates language generation within a statistical setting, focusing on consistency and breadth.
result For most collections of candidate languages, a language model cannot simultaneously avoid hallucinations and capture all language richness.
This paper proposes a new method to connect language and physical actions in reinforcement learning.
problem Connecting linguistic representations to the physical world in embodied agents.
method Language-conditioned goal generators to decouple sensorimotor learning from language acquisition.
result Agents can demonstrate a diversity of behaviors for any given instruction.
Julia accelerates machine learning in various fields with balance of efficiency and simplicity.
problem Efficiency and simplicity in machine learning algorithms.
method Developed and applied Julia language in machine learning.
result Julia balances efficiency and simplicity for machine learning.
Proposes using Wasserstein barycenter for better multilingual alignment.
problem Finding word-to-word translations between multiple languages without parallel data.
method Uses Wasserstein barycenter as a more informative pivot language, minimizing pairwise transportation costs.
result Demonstrates state-of-the-art performances on standard benchmarks.
Survey reviews code-switched speech and language processing.
problem Processing code-switched text and speech for multilingual communities.
method Reviews computational approaches and lists available resources.
result Essential for building intelligent agents that interact in multilingual settings.
The paper applies math and physics to language models, introducing entropy and geometric concepts.
problem Understanding and improving language models to approximate intelligent language.
method Formal definitions, functional analysis, topology, thermodynamics, and set theory.
result Entropy function reveals key obstacles for LLMs and offers insights into language models.
New method shows cultural transmission improves language compositionality.
problem How cultural transmission affects language emergence and compositionality.
method Introduced cultural evolutionary dynamics into language emergence by periodically replacing agents in a population.
result Languages emerge with better compositional generalization due to implicit cultural transmission.
Benchmark tests spoken language models for infant language learning.
problem Understanding how infants learn language from speech.
method Developed a language-acquisition-friendly benchmark.
result Benchmarking shows models' strengths and weaknesses.
Paper examines why LSTMs outperform GRUs in language modeling.
problem Why LSTMs outperform GRUs in language modeling and machine translation.
method Used simplified language models (Context-Free and Context-Sensitive) to analyze LSTM and GRU performance.
result LSTMs perform better due to their ability to count using cell states, while GRUs struggle.
Introduces challenges and techniques for creating machine translation for indigenous languages.
problem Limited data for machine translation of indigenous languages.
method Introduction to challenges, concepts, and techniques for creating MT systems.
result Discussion of recent advances and open questions in NLP for these languages.
Optimus pre-trains sentences in a latent space for various NLP tasks.
problem Training large-scale language models for diverse NLP tasks.
method Pre-trained Variational Autoencoder (VAE) on large text corpus, fine-tuned for various tasks.
result Optimus achieves state-of-the-art on VAE language modeling benchmarks.
ScratchGAN trains language GANs without pre-training, achieving comparable results.
problem Training GANs for natural language is challenging due to gradient estimation, optimization instability, and mode collapse.
method Used large batch sizes, dense rewards, and discriminator regularization to stabilize and improve language GANs.
result ScratchGAN performs comparably to maximum likelihood training on quality and diversity metrics.
Work shows hallucination detection by LLMs is impossible without expert feedback.
problem Detecting hallucinations in LLMs is theoretically impossible without expert-labeled feedback.
method Investigated hallucination detection using a theoretical framework inspired by language identification.
result Automated hallucination detection is impossible for most language collections without expert-labeled feedback.
Real-time spell checker adapts to new languages.
problem No real-time, language-adaptable spell checkers for non-English languages.
method Used Wikipedia and subtitles data to generate dictionaries, created noisy channel datasets, compared with industry tools.
result System performs well across 24 languages, outperforming existing tools.
Model captures author language diffusion over time.
problem Lack of author identity and temporal context in language models.
method Temporal language model conditioning on author and temporal vectors.
result Beat temporal and non-temporal baselines, learns time-varying author representations.
Improved language identification with tuplemax loss.
problem Language identification with user-specified small sets of languages.
method Replaced softmax loss with tuplemax loss.
result 2.33% error rate improvement over softmax loss.
RPP prunes large language models without sacrificing accuracy.
problem How to compress large-scale language representation models without losing performance.
method Reweighted Proximal Pruning (RPP) for large-scale language models.
result RPP maintains high accuracy for both pre-training and downstream tasks at high prune ratios.