A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Study finds phase transition in context-sensitive language model with short-range interactions.
New model shows natural language exhibits phase transition similar to physics.
Convolutional neural networks (CNNs) have recently emerged as a popular building block for natural language processing (NLP). Despite their success, most existing CNN models employed in NLP share the same learned (and static) set of filters for all input sentences. In this paper, we consider an approach of using a smal…
We present a novel language adaptable spell checking system which detects spelling errors and suggests context sensitive corrections in real-time. We show that our system can be extended to new languages with minimal language-specific processing. Available literature majorly discusses spell checkers for English but the…
In this paper we examine a possible reason for the LSTM outperforming the GRU on language modeling and more specifically machine translation. We hypothesize that this has to do with counting. This is a consistent theme across the literature of long term dependence, counting, and language modeling for RNNs. Using the si…
GOAT learns multiple node representations from graph structure alone.
FinLlama uses a fine-tuned Llama 2 model for financial sentiment analysis.
Network representation learning (NRL) is a powerful technique for learning low-dimensional vector representation of high-dimensional and sparse graphs. Most studies explore the structure and metadata associated with the graph using random walks and employ an unsupervised or semi-supervised learning schemes. Learning in…
We propose a principle for exploring context in machine learning models. Starting with a simple assumption that each observation may or may not depend on its context, a conditional probability distribution is decomposed into two parts: context-free and context-sensitive. Then by employing the log-linear word production…
Existing machine reading comprehension (MRC) models do not scale effectively to real-world applications like web-level information retrieval and question answering (QA). We argue that this stems from the nature of MRC datasets: most of these are static environments wherein the supporting documents and all necessary inf…
As global greenhouse gas emissions continue to rise, the use of stratospheric aerosol injection (SAI), a form of solar geoengineering, is increasingly considered in order to artificially mitigate climate change effects. However, initial research in simulation suggests that naive SAI can have catastrophic regional conse…
The quality of data representation in deep learning methods is directly related to the prior model imposed on the representations; however, generally used fixed priors are not capable of adjusting to the context in the data. To address this issue, we propose deep predictive coding networks, a hierarchical generative mo…
A new method for efficient neural network fine-tuning using queryable low-rank update atoms.
Complex architectures of biological neural circuits, such as parallel processing pathways, has been behaviorally implicated in many cognitive studies. However, the theoretical consequences of circuit complexity on neural computation have only been explored in limited cases. Here, we introduce a mechanism by which direc…
New framework enforces demographic parity on distribution tails.
A framework uses preprocessing to improve psychiatric questionnaire predictions while maintaining interpretability.
We consider Markov models of stochastic processes where the next-step conditional distribution is defined by a kernel density estimator (KDE), similar to Markov forecast densities and certain time-series bootstrap schemes. The KDE Markov models (KDE-MMs) we discuss are nonlinear, nonparametric, fully probabilistic repr…
Study quantifies gender bias in language models across 7 languages.
New ASR system handles multiple languages without needing language-specific encoding.
New technique reduces language biases in large language models.
The paper applies math and physics to language models, introducing entropy and geometric concepts.
Paper introduces TrufLL for language model training without labeled data.
Multilingual end-to-end (E2E) models have shown great promise in expansion of automatic speech recognition (ASR) coverage of the world's languages. They have shown improvement over monolingual systems, and have simplified training and serving by eliminating language-specific acoustic, pronunciation, and language models…
Paper reviews neurolinguistics and language technologies, emphasizing mutual enrichment.
New findings show language models can't simultaneously avoid hallucinations and capture all language richness.
Transformers improve Finnish language modeling, achieving lower perplexity scores.
Neural language modeling (LM) has led to significant improvements in several applications, including Automatic Speech Recognition. However, they typically require large amounts of training data, which is not available for many domains and languages. In this study, we propose a multilingual neural language model archite…
lamBERT learns language and actions using multimodal BERT.
Optimus pre-trains sentences in a latent space for various NLP tasks.
Language models learn from training data and can leak private information.
Large language models predict human sensory judgments across multiple modalities.
Benchmark tests spoken language models for infant language learning.
Chronologically consistent models maintain accuracy with time-restricted data.
Text documents are structured on multiple levels of detail: individual words are related by syntax, but larger units of text are related by discourse structure. Existing language models generally fail to account for discourse structure, but it is crucial if we are to have language models that reward coherence and gener…
LLMs translate natural language trading intents into correct option strategies using a domain-specific language.
Study reveals AI skin cancer classifiers underperform for darker skin phototypes, advocating for fairness auditing.
When a bilingual student learns to solve word problems in math, we expect the student to be able to solve these problem in both languages the student is fluent in,even if the math lessons were only taught in one language. However, current representations in machine learning are language dependent. In this work, we pres…
Paper reduces vocabulary losslessly for language model cooperation.
Introduces challenges and techniques for creating machine translation for indigenous languages.
AudioPaLM combines text and speech models to improve speech processing and translation.
We propose multi-way, multilingual neural machine translation. The proposed approach enables a single neural translation model to translate between multiple languages, with a number of parameters that grows only linearly with the number of languages. This is made possible by having a single attention mechanism that is …
Language models are essential for natural language processing (NLP) tasks, such as machine translation and text summarization. Remarkable performance has been demonstrated recently across many NLP domains via a Transformer-based language model with over a billion parameters, verifying the benefits of model size. Model …
The task of translating between programming languages differs from the challenge of translating natural languages in that programming languages are designed with a far more rigid set of structural and grammatical rules. Previous work has used a tree-to-tree encoder/decoder model to take advantage of the inherent tree s…
Neural network framework for language recognition considers sequence information and improves accuracy.
Improved language generation with faster sampling speed.
Loops in surfaces and chord diagrams are studied with graph factorizations and grammars.
Language models are at the heart of numerous works, notably in the text mining and information retrieval communities. These statistical models aim at extracting word distributions, from simple unigram models to recurrent approaches with latent variables that capture subtle dependencies in texts. However, those models a…
GPT-f uses language models to find new proofs in formal math.