Study categorizes mutual funds using natural language processing from unstructured data.
problem Categorizing mutual funds using unstructured data for financial analysis.
method Used natural language processing models to classify mutual funds from their investment strategy descriptions.
result High accuracy in categorizing mutual funds using NLP from unstructured data.
Extends linear representation hypothesis to categorical and hierarchical concepts in LLMs.
problem Representing concepts without natural contrasts in large language models.
method Formalizes linear representation hypothesis for categorical and hierarchical concepts, proving relationships between concept hierarchy and representation geometry.
result Validated theoretical results on large language models, estimating representations for 900+ concepts.
Differentiable language model attacks improve adversarial examples for categorical sequence classifiers.
problem Challenges in adversarial attacks for categorical sequence models due to non-differentiability.
method Fine-tuning a language model as a generator of adversarial examples, using a differentiable loss function that combines a surrogate classifier score and approximate edit distance.
result Semantically better and more resistant adversarial examples.
Probabilistic learning for binary classification with categorical variables.
problem Binary classification with categorical covariates.
method Probabilistic analysis and two algorithms for learning boolean functions.
result Effective learning of boolean functions from binary data.
Natural Language Processing models help encode categorical process inputs.
problem Encoding categorical variables in industrial process modeling.
method Using NLP models for categorical variable encoding, combined with dimensionality reduction.
result Meaningful embeddings of categorical variables improve feature importance.
Developed text classification system for Azerbaijani language.
problem Text clustering problem in Azerbaijani language.
method Machine learning and embedding techniques.
result System successfully categorizes news, product reviews, and more.
This study compares the effectiveness and robustness of multi-class categorization of Amazon product data using transfer learning on pre-trained contextualized language models. Specifically, we fine-tuned BERT and XLNet, two bidirectional models that have achieved state-of-the-art performance on many natural language t…
The ever-growing volume of data of user-generated content on social media provides a nearly unlimited corpus of unlabeled data even in languages where resources are scarce. In this paper, we demonstrate that state-of-the-art results on two Thai social text categorization tasks can be realized by pretraining a language …
Paper introduces new methods for modeling categorical data.
problem Training generative models on categorical data like text and segmentation.
method Argmax Flows and Multinomial Diffusion models.
result Models outperform existing methods in log-likelihood.
New model uses attention for in-context learning of categorical data.
problem Learning from categorical data in context.
method Attention-based network with self-attention and cross-attention layers, using functional gradient descent.
result Model can perform multi-step inference for categorical observations.
The paper proposes a method to analyze categorical feature interactions in large datasets using graph covariance and LLMs.
problem Analyzing complex datasets with numerous categorical features and timestamps.
method Binarization of categorical features using one-hot encoding, computation of graph covariance, identifying significant feature pairs, and using LLMs to generate explanations.
result The method identifies meaningful feature pairs and potential data stories underlying categorical feature interactions.
Deep learning models outperform classical methods in text classification.
problem Improving text classification accuracy using deep learning.
method Comprehensive review of deep learning models and datasets for text classification.
result Deep learning models outperform classical methods on various text classification tasks.
Categorical distributions are ubiquitous in machine learning, e.g., in classification, language models, and recommendation systems. However, when the number of possible outcomes is very large, using categorical distributions becomes computationally expensive, as the complexity scales linearly with the number of outcome…
In human cognition, the expansion of perceived between-category distances and compression of within-category distances is known as categorical perception (CP). There are several hypotheses about the causes of CP (e.g., language, learning, evolution) but no functional model. Whether CP is essential to categorisation or …
The study categorizes reward errors in reinforcement learning, finding some can be beneficial.
problem Training language models with imperfect proxy rewards.
method Theoretical analysis of policy gradient optimization and categorization of reward errors.
result Reward errors can be benign or even beneficial, preventing policy from stalling.
Analyzes the concept of fields in classical and quantum physics.
problem Challenges in defining fields in classical and quantum physics.
method Uses groupoid description of quantum mechanics and categorical language.
result Fields as functors among groupoids of test particles and intrinsic system nature.
Project classifies Hinglish social content on platforms like Twitter, Reddit.
problem Classifying abusive and hate-inducing content in Hinglish on social media.
method Used deep learning with bi-directional sequence models and text augmentation techniques.
result Produced a state-of-the-art classifier that outperforms previous work.
This work concludes a series of four papers on the foundational theory of orbifolds and stacks. We apply the abstract theory, developed in its predecessors, to orbifolds derived from manifolds. Specifically, we show how the very concrete topological base spaces associated to such orbifolds can be described and manipula…
LR-Robot automates SLRs with AI, expert oversight, and multidimensional analysis.
problem Efficient but contextually limited outputs from existing SLR frameworks.
method Human-in-the-loop process, structured knowledge sources, retrieval-augmented generation.
result Empirical demonstration of AI-driven literature synthesis in option pricing.
A new model tackles language generation issues by using discrete variational attention.
problem Information under-representation and posterior collapse in variational autoencoders.
method Proposes a discrete variational attention model with categorical distribution over attention mechanism.
result Enhances latent space for language generation and avoids posterior collapse.
Survey on using large models to train smaller datasets in NLP.
problem Lack of large datasets and computing resources for NLP tasks.
method Analysis of recent transfer learning approaches in NLP.
result Increased demand for transfer learning in NLP due to large models.
Develops a machine learning framework for identifying authorship in texts.
problem Identifying the author of texts, especially when authors are unknown or multiple.
method Formulates authorship identification as a text categorization problem, uses supervised machine learning with stylometric features.
result A model accurately predicts authorship with high accuracy, especially with linguistic stylometric features.
Efficiently models categorical data with low to medium class overlap, improving accuracy over standard distributions.
problem Poor parameter estimates and accuracy in multinomial and Dirichlet multinomial distributions when assumptions are violated.
method Introduces Beta-Liouville multinomial distribution and efficient estimation methods.
result Beta-Liouville multinomial outperforms standard distributions on two out of four datasets.
This work defines a categorical notion of principal bundles.
problem Different definitions of principal bundles in various categories.
method Formulated in join-restriction categories, which generalize partial maps.
result Shows the tangent bundle as the product of tangent space and group object.
This paper surveys NLP techniques for predicting stock market movements.
problem Predicting stock market movements with volatility, seasonality, and time dependence.
method Survey of recent literature in NLP and ML for stock market prediction.
result Illustrates recent trends in stock market prediction research.
New research shows larger language models improve data processing for diverse entries.
problem Optimizing data processing for tables with diverse string entries.
method Analytical tasks on tables with varying language model sizes and a fuzzy join benchmark.
result Larger language models improve data processing for diverse entries, but fine-tuning is necessary.
This paper surveys UQ methods for deep learning.
problem Overconfident predictions in deep learning models.
method Categorizes UQ methods by uncertainty sources.
result Identifies strengths and limitations of each category.
Floer field theory is a construction principle for e.g. 3-manifold invariants via decomposition in a bordism category and a functor to the symplectic category, and is conjectured to have natural 4-dimensional extensions. This survey provides an introduction to the categorical language for the construction and extension…
Unified discrete diffusion for categorical data simplifies training and sampling.
problem Training and sampling in discrete diffusion models for categorical data.
method Mathematical simplifications and elegant unification of discrete-time and continuous-time discrete diffusion.
result Unified Simplified Discrete Denoising Diffusion (USD3) outperforms SOTA baselines.
Softmax is an output activation function for modeling categorical probability distributions in many applications of deep learning. However, a recent study revealed that softmax can be a bottleneck of representational capacity of neural networks in language modeling (the softmax bottleneck). In this paper, we propose an…
This research improves emotion detection from speech, enhancing CCC by 30%.
problem Improving emotion detection from speech for categorical emotions.
method Used LSTM and TC-LSTM networks, trained with multiple datasets and robust features.
result Improved CCC for valence by 30% compared to baseline.
LLMs improve financial analysis by processing large data sets.
problem Traditional financial analysis methods struggle with large data volumes.
method Integrating LLMs for enhanced data processing and analysis.
result LLMs offer new capabilities for real-time financial decision-making.
The output scores of a neural network classifier are converted to probabilities via normalizing over the scores of all competing categories. Computing this partition function, Z, is then linear in the number of categories, which is problematic as real-world problem sets continue to grow in categorical types, such as …
Improved language models using ratio-matching and KL divergence.
problem Efficiently modeling discrete data with diffusion models.
method Introduced new theorems and a novel CTMC transition-rate matrix for ratio-matching and KL divergence.
result 10-15% improvement in perplexity and faster training steps.
Medical applications challenge today's text categorization techniques by demanding both high accuracy and ease-of-interpretation. Although deep learning has provided a leap ahead in accuracy, this leap comes at the sacrifice of interpretability. To address this accuracy-interpretability challenge, we here introduce, fo…
Attention is an increasingly popular mechanism used in a wide range of neural architectures. The mechanism itself has been realized in a variety of formats. However, because of the fast-paced advances in this domain, a systematic overview of attention is still missing. In this article, we define a unified model for att…
Multilayer switch networks are proposed as artificial generators of high-dimensional discrete data (e.g., binary vectors, categorical data, natural language, network log files, and discrete-valued time series). Unlike deconvolution networks which generate continuous-valued data and which consist of upsampling filters a…
We present 27 problems encountered in automating the translation of movie/TV show subtitles. We categorize each problem in one of the three categories viz. problems directly related to textual translation, problems related to subtitle creation guidelines, and problems due to adaptability of machine translation (MT) eng…
This study compresses BERT to make it suitable for low-capability devices.
problem Large Transformer models are resource-intensive.
method Survey and analysis of model compression techniques for BERT.
result Insights into best practices for compressing large-scale Transformer models.
Natural language processing has improved tremendously after the success of word embedding techniques such as word2vec. Recently, the same idea has been applied on source code with encouraging results. In this survey, we aim to collect and discuss the usage of word embedding techniques on programs and source code. The a…
StructureBoost improves gradient boosting for complex categorical variables efficiently.
problem Efficiently handling complex categorical variables with known structure.
method Two methods to overcome computational obstacles in SCDT enumeration for structured categorical variables.
result StructureBoost outperforms existing packages on complex categorical problems.
Categorical bundles provide a natural framework for gauge theories involving multiple gauge groups. Unlike the case of traditional bundles there are distinct notions of triviality, and hence also of local triviality, for categorical bundles. We study categorical principal bundles that are product bundles in the categor…
This study uses NLP to predict stock performance based on analyst reports.
problem Predicting stock performance using textual information from analyst reports.
method Natural language processing (NLP) and a customized BERT deep learning model for Chinese text.
result Strong positive sentiment in analyst reports increases excess return and intraday volatility, while strong negative sentiment increases volatility and trading volume but decreases excess return.
This paper surveys scalable automated alignment methods for LLMs.
problem Scalability issues in traditional human-annotated alignment methods for LLMs.
method Categorizes and discusses various automated alignment methods.
result Emerging automated alignment methods are effective and scalable.
This paper shows using classification instead of regression improves deep RL scalability.
problem Challenges in training value functions for large networks in deep RL.
method Used categorical cross-entropy loss instead of mean squared error regression.
result Significant improvements in performance and scalability across various domains.
Bayesian model improves categorization of explosions from sparse data.
problem Challenges in categorizing explosions from limited data.
method Bayesian update to Event Categorization Matrix model with Bayesian Decision Theory.
result Consistent gains in overall accuracy and lower false negative rates.
A new LLM-based method enhances diversity in oversampling for imbalanced classification.
problem Limited diversity in synthetic minority samples generated by current LLM-based approaches reduces robustness and generalizability.
method Condition synthetic sample generation on minority labels and features, use permutation strategy for fine-tuning, fine-tune on minority and interpolated samples.
result Significantly outperforms eight SOTA baselines in diverse synthetic sample generation and downstream classification tasks.
Human infants can discover words directly from unsegmented speech signals without any explicitly labeled data. In this paper, we develop a novel machine learning method called nonparametric Bayesian double articulation analyzer (NPB-DAA) that can directly acquire language and acoustic models from observed continuous sp…