A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Traditional centralized energy systems have the disadvantages of difficult management and insufficient incentives. Blockchain is an emerging technology, which can be utilized in energy systems to enhance their management and control. Integrating token economy and blockchain technology, token economic systems in energy …
LLM-as-a-service prices vary arbitrarily due to tokenization multiplicity.
problem Arbitrary price variation in LLM-as-a-service due to multiple tokenizations of the same output.
method Introduce canonical generation to restrict LLMs to unique tokenizations and develop an efficient sampling algorithm.
result Our sampling algorithm for canonical generation solves tokenization multiplicity and maintains comparable performance and runtime to standard sampling.
Currently, engineers at substation service providers match customer data with the corresponding internally used signal names manually. This paper proposes a machine learning method to automate this process based on substation signal mapping data from a repository of executed projects. To this end, a bagged token classi…
Shampoo achieves higher token efficiency than Muon in language models.
problem Understanding the relationship and relative data efficiency of Shampoo and Muon compared to Adam and Signum.
method Extensive experiments on language models, demonstrating Shampoo's higher efficiency and decomposing its updates.
result Shampoo's benefits are attributed to its application to weight matrices, challenging interpretations based on variance adaptation and whitening.
Attention mechanism learns to focus on sparse tokens efficiently.
problem Detecting weak, rare, and sparsely located features in long sequences.
method Theoretical analysis and training of a single-layer attention classifier in a sparse-token classification model.
result A single-layer attention classifier can achieve vanishing test error with logarithmic signal strength growth, unlike linear classifiers requiring linear growth.
Recurrent Neural Networks (RNNs) are among the most popular models in sequential data analysis. Yet, in the foundational PAC learning language, what concept class can it learn? Moreover, how can the same recurrent unit simultaneously learn functions from different input tokens to different output tokens, without affect…
Reinforcement learning after next-token prediction aids in learning from diverse sequence lengths.
problem Learning from sequences of varying lengths and complexity.
method Introducing a framework to study reinforcement learning with autoregressive transformers, focusing on next-token prediction and mixture distributions of short and long sequences.
result Reinforcement learning after next-token prediction enables autoregressive transformers to generalize from long sequences, even when they are rare.
Developing efficient and scalable algorithms for Latent Dirichlet Allocation (LDA) is of wide interest for many applications. Previous work has developed an O(1) Metropolis-Hastings sampling method for each token. However, the performance is far from being optimal due to random accesses to the parameter matrices and fr…
Neural sequence generation is typically performed token-by-token and left-to-right. Whenever a token is generated only previously produced tokens are taken into consideration. In contrast, for problems such as sequence classification, bidirectional attention, which takes both past and future tokens into consideration, …
This study examines whether tokenized assets improve liquidity and finds significant differences across categories.
problem Improving liquidity for real-world assets through tokenization.
method Examined tokenized real-world assets using Ethereum-based data, measuring liquidity through turnover, active addresses, and active-month indicator.
result Gold-backed tokens show more persistent on-chain activity than Treasury and private-credit-related products, but asset value alone does not reliably predict liquidity.
Universal Language Model for Fine-tuning [arXiv:1801.06146] (ULMFiT) is one of the first NLP methods for efficient inductive transfer learning. Unsupervised pretraining results in improvements on many NLP tasks for English. In this paper, we describe a new method that uses subword tokenization to adapt ULMFiT to langua…