Groups with specific curvature have a regular language of geodesics.
problem Understanding the language of geodesics in non-positively curved triangle groups.
method Proving finitely many cone types and regularity of geodesic languages.
result The language of lexicographically first geodesics is regular and satisfies the fellow traveller property.
NLP techniques improve drug discovery by analyzing chemical and protein text.
problem Improving drug discovery through better analysis of chemical and protein text.
method Natural language processing techniques applied to biochemical entities.
result Enhanced prediction of molecular properties and design of novel molecules.
New model shows natural language exhibits phase transition similar to physics.
problem Understanding critical properties in natural language models.
method Created a context-sensitive random language model.
result Demonstrated a Berezinskii--Kosterlitz--Thouless phase transition.
Develops a learning algorithm for PSL formulas from examples.
problem Learning human-interpretable descriptions of complex systems from examples.
method Reduces learning to propositional logic constraint satisfaction and uses SAT solver.
result Proposed method provides succinct human-interpretable descriptions from examples.
Recent advancements in language representation models such as BERT have led to a rapid improvement in numerous natural language processing tasks. However, language models usually consist of a few hundred million trainable parameters with embedding space distributed across multiple layers, thus making them challenging t…
New approach decouples skill learning and language grounding for autonomous agents.
problem Autonomous acquisition of skills without external instructions and feedback.
method Language-Goal-Behavior (LGB) architecture with semantic representation.
result Decouples skill learning and language grounding, enabling diversity and strategy switching.
Many recent works have discussed the propensity, or lack thereof, for emergent languages to exhibit properties of natural languages. A favorite in the literature is learning compositionality. We note that most of those works have focused on communicative bandwidth as being of primary importance. While important, it is …
LLMs translate natural language trading intents into correct option strategies using a domain-specific language.
problem Challenges in translating natural language trading intents into correct option strategies due to the complexity of option chain data.
method Introduce Option Query Language (OQL) as a domain-specific intermediate representation to abstract option markets into high-level primitives under grammatical rules. Use LLMs as semantic parsers and validate queries by an engine.
result Significantly improves execution accuracy and logical consistency over direct baselines.
Neural language modeling (LM) has led to significant improvements in several applications, including Automatic Speech Recognition. However, they typically require large amounts of training data, which is not available for many domains and languages. In this study, we propose a multilingual neural language model archite…
Agents learn to communicate and solve navigation tasks efficiently.
problem Understanding and developing communication in artificial agents.
method Multi-agent navigation tasks in gridworld environments.
result Agents develop an interpretable communication protocol that optimally solves tasks.
Large language models predict human sensory judgments across multiple modalities.
problem Determining the extent of perceptual information in language.
method State-of-the-art large language models were used to predict sensory judgments across six psychophysical datasets.
result Large language models can predict human sensory judgments across multiple modalities with significant correlation to human data.
Enhances reward specification in RL with a novel language-based approach.
problem Reward specification in RL can lead to unintended, potentially harmful behaviours.
method Developed a novel class of language-based Reward Machines using RML's built-in memory.
result Can specify non-regular, non-Markovian reward functions for complex tasks.
Multilingual end-to-end (E2E) models have shown great promise in expansion of automatic speech recognition (ASR) coverage of the world's languages. They have shown improvement over monolingual systems, and have simplified training and serving by eliminating language-specific acoustic, pronunciation, and language models…
This paper investigates how large language models achieve neural collapse, a phenomenon linked to generalization.
problem Neural collapse in large language models under imbalanced and token-rich conditions.
method Empirical investigation of scaling and regularization effects on CLMs' progression towards neural collapse.
result Neural collapse properties develop with scale and regularization, linked to generalization in language modeling.
The Myers-Briggs Type Indicator (MBTI) is a popular personality metric that uses four dichotomies as indicators of personality traits. This paper examines the use of pre-trained language models to predict MBTI personality types based on scraped labeled texts. The proposed model reaches an accuracy of 0.47 for correct…
Linear properties are either universal or absent across language models.
problem Explaining the prevalence of linear properties in language models.
method Proved identifiability of distribution-equivalent next-token predictors and analyzed various notions of linearity.
result Linear properties either hold in all or none distribution-equivalent next-token predictors.
CondMTL improves toxicity detection by learning group-specific representations.
problem Algorithmic bias in toxic language detection across demographic groups.
method Conditional Multi-Task Learning (CondMTL) for demographic-specific tasks.
result CondMTL improves predictive recall for minority demographic groups.
This paper introduces Sigma, a domain-specific computational representation for collaboration in large-scale for the field of economics. A computational representation is not a programming language or a software platform. A computational representation is a domain-specific representation system based on three specific …
The goal of homomorphic encryption is to encrypt data such that another party can operate on it without being explicitly exposed to the content of the original data. We introduce an idea for a privacy-preserving transformation on natural language data, inspired by homomorphic encryption. Our primary tool is {\em obfusc…
Study introduces KorFinMTEB for Korean financial texts, revealing model limitations.
problem Limited evaluation benchmarks for low-resource domains, especially Korean.
method Developed KorFinMTEB, a tailored benchmark for Korean financial texts.
result Models perform better on translated benchmarks than on domain-specific ones.
A new deep model generates molecules by fragments, improving validity and uniqueness.
problem Generating valid and unique molecules using deep learning.
method Develops a language model for molecular fragments, using frequency-based masking.
result Significantly outperforms other language model-based competitors in molecule generation.
In this paper we examine a possible reason for the LSTM outperforming the GRU on language modeling and more specifically machine translation. We hypothesize that this has to do with counting. This is a consistent theme across the literature of long term dependence, counting, and language modeling for RNNs. Using the si…
BDH model learns like the brain, rivaling Transformer performance.
problem Leveraging brain-like properties for machine learning.
method Scale-free biologically inspired network of neuron particles.
result BDH model achieves Transformer-like performance with interpretability.
The goal of program synthesis is to automatically generate programs in a particular language from corresponding specifications, e.g. input-output behavior. Many current approaches achieve impressive results after training on randomly generated I/O examples in limited domain-specific languages (DSLs), as with string tra…
Paper uses NLP and IRT to score ESG factors from news articles.
problem Lack of precise ESG metrics in finance.
method Combines NLP and IRT models on ESG-related news data.
result Method offers more precise ESG metrics with temporal dynamics.
A language for specifying complex reinforcement learning tasks.
problem Challenges in specifying and shaping reward functions for complex reinforcement learning tasks.
method Proposes a new language and algorithm for automatically generating and shaping reward functions.
result SPECTRL tool outperforms state-of-the-art baselines.
Paper introduces a specialized text classification system for French Open Banking transactions.
problem Classifying specialized banking text data with high accuracy and efficiency.
method Data collection, labeling, preprocessing, modeling, and evaluation stages with language-specific techniques.
result Enhanced performance and efficiency compared to generic approaches.
In this paper, we present Russian language datasets in the digital humanities domain for the evaluation of word embedding techniques or similar language modeling and feature learning algorithms. The datasets are split into two task types, word intrusion and word analogy, and contain 31362 task units in total. The chara…
Study finds phase transition in context-sensitive language model with short-range interactions.
problem Understanding phase transitions in language models with short-range interactions.
method Constructed a random language model with short-range interactions and investigated its statistical properties.
result Phase transition occurs in context-sensitive language models with constant context length.
The paper tackles data misappropriation in LLMs by embedding watermarks and testing for their presence.
problem Detecting data misappropriation in LLMs trained on copyrighted data.
method Embedding watermarks, formulating as hypothesis testing, developing statistical framework, constructing test statistics, determining optimal thresholds, controlling errors, establishing asymptotic optimality.
result The proposed statistical testing framework effectively detects data misappropriation in LLMs.
Study evaluates if LLMs have company-specific biases in financial sentiment analysis.
problem Evaluating if large language models exhibit company-specific biases in financial sentiment analysis.
method Comparing sentiment scores with and without company names, constructing economic models, and empirical analysis.
result LLMs show company-specific biases in sentiment analysis, impacting investor behavior and stock prices.
Paper introduces TrufLL for language model training without labeled data.
problem Training language models from scratch with reinforcement learning.
method Dynamic vocabulary truncation in RL for language generation.
result TrufLL successfully learns language generation policies from scratch.
Study quantifies gender bias in language models across 7 languages.
problem Measuring gender bias in language models across multiple languages.
method Curated dataset of politicians, multilingual language models, probing language models.
result Larger language models do not show significant gender bias compared to smaller ones.
This paper describes a language representation model which combines the Bidirectional Encoder Representations from Transformers (BERT) learning mechanism described in Devlin et al. (2018) with a generalization of the Universal Transformer model described in Dehghani et al. (2018). We further improve this model by addin…
New ASR system handles multiple languages without needing language-specific encoding.
problem Joint training of data-rich and data-scarce languages in a single model.
method Transforms all languages to a single writing system through transliteration, separating modeling and rendering.
result Language-agnostic multilingual ASR system reduces WER up to 10% over language-dependent models.
Improves drug properties using a novel LLM and reinforcement learning.
problem Optimizing drug properties while retaining chemical stability.
method Structured Policy Optimization (SPO) for fine-tuning a large language model.
result Enhanced drug properties across multiple target objectives.
German FinBERT improves financial text analysis performance.
problem Capturing domain-specific nuances in financial text.
method German FinBERT is a pre-trained German language model trained on a large corpus of financial data.
result German FinBERT outperforms standard models on finance-specific tasks.
The study enhances financial rule matching using NLP without datasets.
problem Performing semantic matching between financial rules and policies.
method Outperforming pre-trained models with NLP techniques using free resources.
result Improved semantic matching between financial rules and policies.
Let G be a finitely generated group. We show that for any finite generating set A, the language consisting of all geodesics in Cay(G,A) with a contracting property is a regular language. As an application, we show that any finitely generated group containing an infinite contracting geodesic must be either virtual…
ERP improves drug discovery by balancing molecule generation quality and efficiency.
problem Generating valid and optimal molecules from large language models.
method Entropy-Reinforced Planning (ERP) for Transformer Decoding.
result ERP outperforms current state-of-the-art algorithms by 1-5 percent on SARS-CoV-2 and human cancer cell targets.
ContextBench benchmarks methods for generating linguistically fluent inputs that activate specific latent features in language models.
problem Identifying inputs that trigger specific behaviours or latent features in language models.
method Context modification and benchmarking methods like Evolutionary Prompt Optimisation (EPO) with LLM-assistance and diffusion model inpainting.
result Enhanced methods achieve state-of-the-art performance in balancing elicitation effectiveness and fluency.
Gradient descent biases linear models in next-token prediction towards data entropy.
problem Optimization bias in next-token prediction models.
method Analysis of gradient descent on linear models with sparse conditional distributions.
result Gradient descent selects parameters that equate token logits differences to log-odds in the data subspace.
LLMs predict SGD convergence without training on specific data.
problem Predicting SGD convergence without labeled data.
method Link between SGD and Markov chains, leveraging LLMs' understanding of dynamical systems.
result Zero-shot prediction of SGD convergence for new starting points.
Twitter provides an open and rich source of data for studying human behaviour at scale and is widely used in social and network sciences. However, a major criticism of Twitter data is that demographic information is largely absent. Enhancing Twitter data with user ages would advance our ability to study social network …
The presence of Long Distance Dependencies (LDDs) in sequential data poses significant challenges for computational models. Various recurrent neural architectures have been designed to mitigate this issue. In order to test these state-of-the-art architectures, there is growing need for rich benchmarking datasets. Howev…
AFTER technique improves NLP models by preventing overfitting to task-specific domains.
problem Standard fine-tuning degrades pretraining domain representations.
method Complements task-specific loss with adversarial objective.
result AFTER leads to improved performance on various NLP tasks.
We present a novel language adaptable spell checking system which detects spelling errors and suggests context sensitive corrections in real-time. We show that our system can be extended to new languages with minimal language-specific processing. Available literature majorly discusses spell checkers for English but the…
Automates translating natural language to Verilog for digital design.
problem Manual translation of natural language specifications to Verilog is time-consuming and error-prone.
method Fine-tuned GPT-2 to derive Verilog from English, using a dataset of design tasks.
result GPT-2 achieved 94.8% correct translation across simple and abstract design tasks.