A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Unified framework for complex, split-complex, and dual numbers.
This paper finds a linear relationship between t-SNE perplexity and data set size.
Perplexity fails to distinguish correct predictions from incorrect ones in some cases.
Proposes a parametric t-SNE without perplexity tuning.
t-Distributed Stochastic Neighbor Embedding (t-SNE) is one of the most widely used dimensionality reduction methods for data visualization, but it has a perplexity hyperparameter that requires manual selection. In practice, proper tuning of t-SNE perplexity requires users to understand the inner working of the method a…
Theoretical analysis shows MDMs can be efficient but not for all metrics.
New method reduces discrete flow transitions, improving perplexity estimation.
Transformers improve Finnish language modeling, achieving lower perplexity scores.
We present Meena, a multi-turn open-domain chatbot trained end-to-end on data mined and filtered from public domain social media conversations. This 2.6B parameter neural network is simply trained to minimize perplexity of the next token. We also propose a human evaluation metric called Sensibleness and Specificity Ave…
Improved language models using ratio-matching and KL divergence.
In order to find hyperparameters for a machine learning model, algorithms such as grid search or random search are used over the space of possible values of the models hyperparameters. These search algorithms opt the solution that minimizes a specific cost function. In language models, perplexity is one of the most pop…
Telescope detects LLM generated text by measuring token repetition probability.
We study large-scale kernel methods for acoustic modeling and compare to DNNs on performance metrics related to both acoustic modeling and recognition. Measuring perplexity and frame-level classification accuracy, kernel-based acoustic models are as effective as their DNN counterparts. However, on token-error-rates DNN…
Improved language models with talking-heads attention.
Enhances LLM quantization with MDBF, improving perplexity and accuracy.
Streaming variational Bayes (SVB) is successful in learning LDA models in an online manner. However previous attempts toward developing online Monte-Carlo methods for LDA have little success, often by having much worse perplexity than their batch counterparts. We present a streaming Gibbs sampling (SGS) method, an onli…
Improved GEC models use scored data from large pretraining to outperform.
Neural language models (LMs) based on recurrent neural networks (RNN) are some of the most successful word and character-level LMs. Why do they work so well, in particular better than linear neural LMs? Possible explanations are that RNNs have an implicitly better regularization or that RNNs have a higher capacity for …
Kronecker Products (KP) have been used to compress IoT RNN Applications by 15-38x compression factors, achieving better results than traditional compression methods. However when KP is applied to large Natural Language Processing tasks, it leads to significant accuracy loss (approx 26%). This paper proposes a way to re…
Aioli unifies language model data mixing methods and improves performance.
Many challenges in natural language processing require generating text, including language translation, dialogue generation, and speech recognition. For all of these problems, text generation becomes more difficult as the text becomes longer. Current language models often struggle to keep track of coherence for long pi…
New attack reveals memorization patterns in pre-trained LLMs.
Recurrent Neural Networks (RNN), Long Short-Term Memory Networks (LSTM), and Memory Networks which contain memory are popularly used to learn patterns in sequential data. Sequential data has long sequences that hold relationships. RNN can handle long sequences but suffers from the vanishing and exploding gradient probl…
Neural text generation models are often autoregressive language models or seq2seq models. These models generate text by sampling words sequentially, with each word conditioned on the previous word, and are state-of-the-art for several machine translation and summarization benchmarks. These benchmarks are often defined …
AlphaPruning optimizes LLM pruning using HT-SR theory for better performance.
AutoScale improves LLM pre-training by adjusting data mixtures at different scales.
ECLIPSE detects AI hallucinations in finance with high accuracy.
Paper proposes a new neural machine translation method for wave data.
New framework selects high-quality pretraining data without training LLMs.
Numeracy is the ability to understand and work with numbers. It is a necessary skill for composing and understanding documents in clinical, scientific, and other technical domains. In this paper, we explore different strategies for modelling numerals with language models, such as memorisation and digit-by-digit composi…
Unified SVD compression fails in practical tasks, highlighting the importance of per layer activation reconstruction.
QuantEase optimizes LLMs with CD-based quantization, achieving state-of-the-art performance.
DatedGPT prevents lookahead bias in financial forecasting models.
t-SNE algorithm's points remain bounded under gradient flow.
Proof of existence of a complex structure on the six-sphere, followed by an explicit computation of its underlying integrable almost complex tensor by the aid of inner automorphisms of the octonions, is exhibited. Both are elementary and self-contained however the size and complexity of the emerging almost complex tens…
A neural network method for topic modeling from few documents.
Graph signal processing detects hallucinations in large language models.
Recurrent neural networks (RNNs) stand at the forefront of many recent developments in deep learning. Yet a major difficulty with these models is their tendency to overfit, with dropout shown to fail when applied to recurrent layers. Recent results at the intersection of Bayesian modelling and deep learning offer a Bay…
Simplified masked diffusion models improve discrete data generation.
This work compares and evaluates various sampling methods for neural language models.
New method boosts performance of diffusion models on discrete data like natural language.
We present two simple ways of reducing the number of parameters and accelerating the training of large Long Short-Term Memory (LSTM) networks: the first one is "matrix factorization by design" of LSTM matrix into the product of two smaller matrices, and the second one is partitioning of LSTM matrix, its inputs and stat…
Paper proposes a neural network method for fast, interpretable AR model estimation.
One-Shot Neural architecture search (NAS) attracts broad attention recently due to its capacity to reduce the computational hours through weight sharing. However, extensive experiments on several recent works show that there is no positive correlation between the validation accuracy with inherited weights from the supe…
We propose a new generative model of sentences that first samples a prototype sentence from the training corpus and then edits it into a new sentence. Compared to traditional models that generate from scratch either left-to-right or by first sampling a latent sentence vector, our prototype-then-edit model improves perp…
Diffusion LLMs can efficiently generate harmful prompts for adversarial testing.
In this paper we present a model for unsupervised topic discovery in texts corpora. The proposed model uses documents, words, and topics lookup table embedding as neural network model parameters to build probabilities of words given topics, and probabilities of topics given documents. These probabilities are used to re…
We develop dependent hierarchical normalized random measures and apply them to dynamic topic modeling. The dependency arises via superposition, subsampling and point transition on the underlying Poisson processes of these measures. The measures used include normalised generalised Gamma processes that demonstrate power …