Word evolution refers to the changing meanings and associations of words throughout time, as a byproduct of human language evolution. By studying word evolution, we can infer social trends and language constructs over different periods of human history. However, traditional techniques such as word representation learni…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,786 papers · 148 categories
Trend · papers per month
48 results for “language evolution”
Study finds common poetic themes across languages over time.
problem Understanding thematic evolution in different poetic traditions.
method Applied Latent Dirichlet Allocation (LDA) to poetry corpora of four languages.
result Identified common themes and their temporal trends across poetic traditions.
Aligns text representations over time for better performance in sequential tasks.
problem Language evolution causes data drift in sequential tasks.
method Sequentially aligns learned representations to combat data drift.
result Sequential alignment outperforms strong baselines on various tasks.
SLED improves factuality in LLMs without external knowledge.
problem Unreliable or factually incorrect outputs from large language models.
method Contrasts final layer logits with early layers' logits, uses approximate gradient to refine outputs.
result Consistently improves factual accuracy over existing methods.
Language is dynamic, constantly evolving and adapting with respect to time, domain or topic. The adaptability of language is an active research area, where researchers discover social, cultural and domain-specific changes in language using distributional tools such as word embeddings. In this paper, we introduce the gl…
In this paper, we try to explore the evolution of language through case calculations. First, we chose the novels of eleven British writers from 1400 to 2005 and found the corresponding works; Then, we use the natural language processing tool to construct the corresponding eleven corpora, and calculate the respective wo…
CausalEvolve improves efficiency and discovery in open-ended scientific tasks.
problem Lack of targeted guidance and knowledge organization in evolve-based agents.
method Causal scratchpad that identifies and reasons about guiding factors for evolution.
result Effective improvement in evolutionary efficiency and better solutions.
In this work, sequence-to-sequence (seq2seq) models, originally developed for language translation, are used to predict the temporal evolution of complex, multi-physics computer simulations. The predictive performance of seq2seq models is compared to state transition models for datasets generated with multi-physics cod…
EFS: Evolutionary Factor Searching for Sparse Portfolio Optimization Using Large Language Modelsq-fin.PM
EFS uses LLMs to optimize sparse portfolios by evolving alpha factors.
problem Sparse portfolio optimization in dynamic market regimes.
method Evolutionary feedback loop with LLM-generated alpha factors.
result Significantly outperforms baselines in diverse datasets.
LR-Robot automates SLRs with AI, expert oversight, and multidimensional analysis.
problem Efficient but contextually limited outputs from existing SLR frameworks.
method Human-in-the-loop process, structured knowledge sources, retrieval-augmented generation.
result Empirical demonstration of AI-driven literature synthesis in option pricing.
A framework uses free probability to analyze Transformer models.
problem Understanding the dynamics and complexity of Transformer-based language models.
method Formal operator-theoretic analysis using free probability theory.
result Entropy-based generalization bounds derived under freeness assumptions.
Word embeddings are a powerful approach for unsupervised analysis of language. Recently, Rudolph et al. (2016) developed exponential family embeddings, which cast word embeddings in a probabilistic framework. Here, we develop dynamic embeddings, building on exponential family embeddings to capture how the meanings of w…
Online review communities are dynamic as users join and leave, adopt new vocabulary, and adapt to evolving trends. Recent work has shown that recommender systems benefit from explicit consideration of user experience. However, prior work assumes a fixed number of discrete experience levels, whereas in reality users gai…
Agents learn to communicate and solve navigation tasks efficiently.
problem Understanding and developing communication in artificial agents.
method Multi-agent navigation tasks in gridworld environments.
result Agents develop an interpretable communication protocol that optimally solves tasks.
INFUSER improves reasoning by co-evolving a generator and solver with adaptive curriculum.
problem Improving reasoning in language models with minimal external supervision.
method INFUSER uses a Generator and Solver to co-evolve iteratively, rewarding the Generator with an influence score and the Solver with correctness rewards.
result INFUSER outperforms self-evolution baselines by over 20% on Olympiad and SuperGPQA benchmarks.
Recurrent neural networks trained on regular languages exhibit stable states that can recover from noise.
problem Stability of internal states in recurrent neural networks trained on regular languages.
method Empirical study with analysis of network activation and transitions between states.
result Recurrent neural networks trained on regular languages can recover from random perturbations and maintain stable states.
Survey of diffusion and optimal transport methods in machine learning.
problem Design and analysis of time-evolving probability distributions in machine learning.
method Switch from Eulerian to Lagrangian representation through vector fields.
result Both diffusion methods and optimal transport offer computational advantages.
This study uses AI to analyze financial market coverage from YouTube videos.
problem Challenges in analyzing a large number of financial market videos.
method Used Whisper model to generate text from videos, applied natural language processing.
result Highlights dynamics of financial market coverage and identifies trending topics.
LLMs improve financial sentiment analysis in finance.
problem Defining and measuring financial sentiment.
method Investigation of sentiment measurement methods and LLMs.
result LLMs enhance financial sentiment analysis.
Study reveals neural scaling laws in random graphs and natural language models.
problem Understanding the origin of neural scaling laws in complex systems.
method Examined scaling laws in transformers trained on random walks and simplified natural language models.
result Neural scaling laws emerge in the absence of power law structure in data correlations.
We propose a population-based Evolutionary Stochastic Gradient Descent (ESGD) framework for optimizing deep neural networks. ESGD combines SGD and gradient-free evolutionary algorithms as complementary algorithms in one framework in which the optimization alternates between the SGD step and evolution step to improve th…
Graphs help agents learn emergent communication.
problem Learning robust communication protocols in multi-agent systems.
method Graph convolutional networks to model emergent language and cooperation.
result Agents learn to generalize beyond training samples, revealing true factors of variation.
The study examines how language models learn to represent the world, identifying conditions for ecological veridicality.
problem Understanding when language models learn to represent the world accurately and how this learning process can fail.
method Analyzes the Bayes-optimal next-token cross-entropy decomposition and the role of training ecology in shaping model representations.
result The minimum-complexity zero-excess solution is the quotient partition by training equivalence, and this solution is not preserved in in-context learning or per-task adaptation.
The study develops a machine learning system for credit scoring and default prediction.
problem Developing a robust credit rating and default prediction system.
method Combines NLP, AE, GBM, DE, and SHAP/LIME for model interpretability.
result Obtained excellent out-of-sample performance in credit rating and default prediction.
Deep learning improves conversational recommender systems.
problem Creating effective goal-oriented dialogue systems through natural language interactions.
method Apply deep learning techniques to conversational recommender systems.
result Deep learning models enhance user preference learning in CRS.
Indirect competition emerged from the complex organization of human societies, and knowledge of the existing network topology may aid in developing effective strategies for success. Here, we propose an agent-based model of competition with systems co-existing in a `small-world' social network. We show that within the r…
Paper reviews the evolution of alpha from human insight to AI-powered systems.
problem Exceeding market benchmarks in finance.
method Five-stage taxonomy integrating representation learning, multimodal data fusion, and LLM agents.
result Unified framework for evaluating and developing next-gen alpha systems.
Characterizing temporal dependence patterns is a critical step in understanding the statistical properties of sequential data. Long Range Dependence (LRD) --- referring to long-range correlations decaying as a power law rather than exponentially w.r.t. distance --- demands a different set of tools for modeling the unde…
INFUSER improves reasoning by self-evolving with a generator and solver that co-learn from unstructured documents.
problem Improving reasoning through self-evolution
method INFUSER uses a generator and solver co-evolving in a document pool to improve reasoning.
result INFUSER outperforms strong self-evolution baselines on Olympiad and SuperGPQA benchmarks.
This paper uses LLMs to streamline industrial data-centric R&D cycles.
problem High costs in human, computational, and time resources in data-centric R&D.
method Explores how large language models can understand domain-specific requirements and automate R&D tasks.
result Promising results show potential for automating industrial data-centric R&D cycles.
Flow-based models use ODEs to generate complex data distributions.
problem Generating high-dimensional data with complex probability distributions.
method Flow-based models use invertible mappings governed by ODEs to capture these distributions.
result Flow-based models provide exact likelihood estimation and efficient sampling.
We consider probabilistic programming for birth-death models of evolution and introduce a new widely-applicable inference method that combines an extension of the alive particle filter (APF) with automatic Rao-Blackwellization via delayed sampling. Birth-death models of evolution are an important family of phylogenetic…
This paper addresses the scalability challenge of architecture search by formulating the task in a differentiable manner. Unlike conventional approaches of applying evolution or reinforcement learning over a discrete and non-differentiable search space, our method is based on the continuous relaxation of the architectu…
This paper explores NLP techniques for insurance, detailing methods and applications.
problem Extracting value from insurance reports using complex text data.
method Detailed explanation of NLP methods and their implementation in insurance.
result Enhanced risk monitoring and policyholder benefits through NLP.
Token-adaptive FFN design improves LLM expressivity.
problem Fixed activation functions limit FFN expressivity in LLMs.
method Mixture of Activations (MoA) and learnable activations (LA).
result MoA achieves lower terminal loss and better scaling than baselines.
In human cognition, the expansion of perceived between-category distances and compression of within-category distances is known as categorical perception (CP). There are several hypotheses about the causes of CP (e.g., language, learning, evolution) but no functional model. Whether CP is essential to categorisation or …
We introduce a scalable measure of curvature for analyzing training dynamics of large language models.
problem Analyzing the training dynamics of large language models due to high computational cost of measuring Hessian sharpness.
method We introduce critical sharpness and relative critical sharpness as computationally efficient measures capturing Hessian sharpness phenomena.
result We provide the first demonstration of sharpness phenomena at scale up to 7B parameters.
CLARITY compares dissimilar datasets, identifying structural and relationship inconsistencies.
problem Integrating qualitatively different datasets from various disciplines.
method Non-parametric approach decomposing similarities into structural and relationship components.
result Identifies and interprets inconsistencies between datasets.
eva method improves RL models by adaptively creating prompts, boosting performance.
problem Fixed prompt distribution limits scalability of RL models post-training.
method Evolving Alignment via Asymmetric Self-Play (eva) approach.
result Significant performance boost on benchmarks, e.g. 51.6% to 60.1% win-rate.
The calculus correspondence has been known to exist between generic pedal evolutions and generic wave front evolutions. In this paper, we first extend the known results on the calculus correspondence to evolutions with multi-parameters, and then give applications of calculus correspondence. Moreover, we discuss the pos…
LR-Robot accelerates SLRs by combining expert oversight and AI, revealing trends and patterns in financial research.
problem Manual SLRs are impractical due to the scale and complexity of modern financial research.
method Domain experts define taxonomies and constraints, LLMs execute classification, and human evaluation ensures reliability.
result AI can understand and synthesize literature, revealing trends and core research directions.
Word meaning changes over time, depending on linguistic and extra-linguistic factors. Associating a word's correct meaning in its historical context is a central challenge in diachronic research, and is relevant to a range of NLP tasks, including information retrieval and semantic search in historical texts. Bayesian m…
Study material evolution using groupoids to track intrinsic properties.
problem Tracking material evolution without considering the whole body.
method Construct a groupoid encoding intrinsic properties and characteristic foliations.
result Define the evolution equation for material points.
The paper studies circular evolutes and involutes of framed curves in Euclidean space.
problem Investigating properties of framed curves and their evolutes and involutes.
method Definition and analysis of circular evolutes and involutes of framed curves, properties of normal surfaces, and their relations.
result Circular evolutes and involutes of framed curves are opposite operations under suitable assumptions, similar to fronts in the Euclidean plane.
Dynamic Word Embeddingsstat.ML
We present a probabilistic language model for time-stamped text data which tracks the semantic evolution of individual words over time. The model represents words and contexts by latent trajectories in an embedding space. At each moment in time, the embedding vectors are inferred from a probabilistic version of word2ve…
The paper studies curve evolution using the PLR equation and its solutions.
problem Investigating the evolution of space curves governed by the PLR equation.
method Examined the Lund-Regge evolution and derived its representation in the Frenet frame, aligning with the Lax system of the PLR equation. Developed a construction method for curve families via the Sym formula.
result Described the Lund-Regge evolution corresponding to Date multi-soliton solutions to the PLR equation.
Predict human lives using sequences of life events.
problem Predicting human life outcomes based on detailed event sequences.
method Adapting natural language processing techniques to analyze life events.
result Models predict diverse outcomes with high accuracy, outperforming state-of-the-art models.
Serendipity and strategy in rapid innovationphysics.soc-ph
Innovation is to organizations what evolution is to organisms: it is how organisations adapt to changes in the environment and improve. Governments, institutions and firms that innovate are more likely to prosper and stand the test of time; those that fail to do so fall behind their competitors and succumb to market an…