This study uses NLP to detect financial risks from documents.
problem Detecting and predicting financial risks in documents.
method NLP model design, text preprocessing, feature extraction, machine learning.
result NLP model effectively identifies and predicts financial risks.
Introduces challenges and techniques for creating machine translation for indigenous languages.
problem Limited data for machine translation of indigenous languages.
method Introduction to challenges, concepts, and techniques for creating MT systems.
result Discussion of recent advances and open questions in NLP for these languages.
Recent Transformer-based contextual word representations, including BERT and XLNet, have shown state-of-the-art performance in multiple disciplines within NLP. Fine-tuning the trained contextual models on task-specific datasets has been the key to achieving superior performance downstream. While fine-tuning these pre-t…
A benchmark for NLP models trained on text datasets.
problem Limited access to high-performance clusters for NAS experiments.
method Created a search space for recurrent neural networks on text datasets and trained 14k architectures.
result Demonstrated the potential of precomputed NAS results for NLP.
Code-switching, the alternation of languages within a conversation or utterance, is a common communicative phenomenon that occurs in multilingual communities across the world. This survey reviews computational approaches for code-switched Speech and Natural Language Processing. We motivate why processing code-switched …
Recent successes and advances in Deep Neural Networks (DNN) in machine vision and Natural Language Processing (NLP) have motivated their use in traditional signal processing and communications systems. In this paper, we present results of such applications to the problem of automatic modulation recognition. Variations …
Humor is a unique and creative communicative behavior displayed during social interactions. It is produced in a multimodal manner, through the usage of words (text), gestures (vision) and prosodic cues (acoustic). Understanding humor from these three modalities falls within boundaries of multimodal language; a recent r…
Anti-phishing aims to detect phishing content/documents in a pool of textual data. This is an important problem in cybersecurity that can help to guard users from fraudulent information. Natural language processing (NLP) offers a natural solution for this problem as it is capable of analyzing the textual content to per…
Word2Vec is a widely used algorithm for extracting low-dimensional vector representations of words. It generated considerable excitement in the machine learning and natural language processing (NLP) communities recently due to its exceptional performance in many NLP applications such as named entity recognition, sentim…
QUACKIE creates a new benchmark for NLP interpretability.
problem Evaluating NLP interpretability methods is challenging due to biased ground truths.
method Formulated a custom classification task from question-answering datasets, generating unbiased ground truths.
result Demonstrated the effectiveness of current interpretability methods on the new benchmark.
Deep models for financial transactions are vulnerable to adversarial attacks, especially when adding transaction tokens.
problem Vulnerability of deep models to adversarial attacks on financial transaction records.
method Examine adversarial attacks and defenses on transaction records data, considering black-box attacks and adding transaction tokens.
result A few generated transactions can fool a deep-learning model, highlighting the need for robustness improvements.
Study uses few-shot learning to analyze claims and arguments in German debate on arms deliveries.
problem Limited data and computational resources for automated content analysis.
method Multilingual transformer model with adapter extension and few-shot learning.
result Parameter-efficient approach performs well on varying training set sizes.
Optimus pre-trains sentences in a latent space for various NLP tasks.
problem Training large-scale language models for diverse NLP tasks.
method Pre-trained Variational Autoencoder (VAE) on large text corpus, fine-tuned for various tasks.
result Optimus achieves state-of-the-art on VAE language modeling benchmarks.
We consider the extreme multi-label text classification (XMC) problem: given an input text, return the most relevant labels from a large label collection. For example, the input text could be a product description on Amazon.com and the labels could be product categories. XMC is an important yet challenging problem in t…
New measure corrects news bias in NLP stock return forecasting.
problem Improving stock return and volatility forecasting accuracy.
method Hype-Adjusted Probability Measure, sentiment score equation.
result Significantly improved forecast accuracy for U.S. semiconductor tickers.
This paper analyzes crypto white papers under MiCAR, highlighting NLP's role.
problem Regulatory changes in crypto white papers under MiCAR.
method Survey of existing NLP applications, analysis of MiCAR changes.
result NLP can assist in regulatory compliance and white paper analysis.
This paper improves NLP interpretability by using sentence segments instead of words.
problem Limitations of word-based sampling in explaining complex BERT models.
method Using sentence segments as elementary building blocks for NLP interpretability.
result Improved fidelity of the explainer on a benchmark classification task.
New algorithm learns low-rank matrices with linear number of samples.
problem Learning low-rank matrices efficiently in latent-variable applications.
method Proposed algorithm that uses linear number of samples in high dimension.
result Learning kimesk, rank-r, matrices requires $Ω( rac{kr}{ε^2})$ samples. Within the natural language processing (NLP) community, active learning has been widely investigated and applied in order to alleviate the annotation bottleneck faced by developers of new NLP systems and technologies. This paper presents the first theoretical analysis of stopping active learning based on stabilizing pr…
SusGen-GPT improves financial NLP and ESG report generation.
problem Lack of advanced NLP tools for finance and ESG domains.
method Developed SusGen-30K dataset and SusGen-GPT models.
result Achieved state-of-the-art performance in financial NLP tasks.
MDA optimizer performs similarly to SGD+M in CV and Adam in NLP.
problem Performance degradation due to choosing the wrong optimizer.
method Modernized Dual Averaging (MDA) optimizer, inspired by dual averaging.
result MDA performs as well as SGD+M in CV and as Adam in NLP.
We introduce HUBERT which combines the structured-representational power of Tensor-Product Representations (TPRs) and BERT, a pre-trained bidirectional Transformer language model. We show that there is shared structure between different NLP datasets that HUBERT, but not BERT, is able to learn and leverage. We validate …
Network representation learning (NRL) methods aim to map each vertex into a low dimensional space by preserving the local and global structure of a given network, and in recent years they have received a significant attention thanks to their success in several challenging problems. Although various approaches have been…
This paper explores NLP techniques for insurance, detailing methods and applications.
problem Extracting value from insurance reports using complex text data.
method Detailed explanation of NLP methods and their implementation in insurance.
result Enhanced risk monitoring and policyholder benefits through NLP.
Survey on using large models to train smaller datasets in NLP.
problem Lack of large datasets and computing resources for NLP tasks.
method Analysis of recent transfer learning approaches in NLP.
result Increased demand for transfer learning in NLP due to large models.
New pruning method retains model expressiveness for NLP tasks.
problem Pruning large pretrained transformer models for real-world deployment.
method Mixture Gaussian Prior Pruning (MGPP) algorithm.
result MGPP outperforms existing pruning methods in high sparsity settings.
BERT outperforms traditional machine learning in text classification tasks.
problem Comparing BERT to traditional machine learning methods for text classification.
method Empirical testing of BERT against TF-IDF-based machine learning models in various scenarios.
result BERT demonstrates superior performance and independence from text features.
Hungarian text processing improved with efficient, accurate NLP pipelines.
problem Improving text processing for Hungarian language.
method Implemented in spaCy framework, focusing on efficiency and accuracy.
result Near state-of-the-art performance in all text preprocessing steps.
HuSpaCy offers an industrial-grade Hungarian NLP toolkit.
problem Lack of suitable open-source Hungarian NLP pipelines.
method Built on spaCy, HuSpaCy includes lemmatization, morphosyntactic analysis, entity recognition, and word embeddings.
result HuSpaCy achieves high accuracy with resource-efficient prediction.
The Hype Index measures media attention to equities using NLP.
problem Quantifying media attention to equities for volatility analysis.
method Constructs News Count-Based and Capitalization Adjusted Hype Indices using NLP.
result The Hype Index family provides valuable tools for stock volatility analysis.
While state-of-the-art NLP explainability (XAI) methods focus on explaining per-sample decisions in supervised end or probing tasks, this is insufficient to explain and quantify model knowledge transfer during (un-)supervised training. Thus, for TX-Ray, we modify the established computer vision explainability principle…
The paper improves cryptocurrency price forecasting using deep learning and NLP on financial, blockchain, and social media data.
problem Improving cryptocurrency price forecasting accuracy and profitability.
method Integrates financial, blockchain, and social media data; applies BART MNLI model for sentiment analysis; uses deep learning NLP models; compares with traditional methods; uses local extrema as predictive targets.
result Significantly improves forecasting accuracy and profitability of cryptocurrency price predictions.
InvestLM is a financial domain LLM tuned on LLaMA-65B for investment advice.
problem Improving financial text understanding and advice generation for investment.
method Curated financial instruction dataset, LLaMA-65B, less-is-more-for-alignment approach.
result InvestLM provides comparable responses to state-of-the-art commercial models.
In this work we introduce malware detection from raw byte sequences as a fruitful research area to the larger machine learning community. Building a neural network for such a problem presents a number of interesting challenges that have not occurred in tasks such as image processing or NLP. In particular, we note that …
COCKATIEL explains neural net models on NLP tasks by identifying meaningful concepts.
problem Transformer models are complex and hard to interpret.
method COCKATIEL uses NMF and sensitivity analysis to identify and rank concepts used by the model.
result COCKATIEL provides accurate and meaningful explanations without affecting model performance.
This paper presents a deep-learning based traffic classification method for identifying multiple streaming video sources at the same time within an encrypted tunnel. The work defines a novel feature inspired by Natural Language Processing (NLP) that allows existing NLP techniques to help the traffic classification. The…
This paper explores how NLP enhances insurance data analysis.
problem Traditional insurance data limitations and need for alternative data.
method Application of NLP techniques to transform and analyze unstructured text data.
result NLP techniques improve insurance data analysis and risk assessment.
Study improves cryptocurrency price prediction using unlabeled text data.
problem Predicting cryptocurrency returns from unlabelled text data.
method Introduced weak learning approach to finetune BERT on unlabeled text data.
result Finetuning pretrained NLP models with weak labels enhances forecast accuracy.
NLP techniques improve drug discovery by analyzing chemical and protein text.
problem Improving drug discovery through better analysis of chemical and protein text.
method Natural language processing techniques applied to biochemical entities.
result Enhanced prediction of molecular properties and design of novel molecules.
Sumformer simplifies Transformers to handle long sequences efficiently.
problem Quadratic complexity of Transformers limits their use with long sequences.
method Introducing Sumformer, a simple architecture that universally approximates equivariant sequence-to-sequence functions.
result Sumformer achieves the first universal approximation results for Linformer and Performer.
Framework quantifies financial NLP robustness under regime shifts.
problem Semantic and causal drift in financial news narratives.
method Four metrics: FCAS, PCS, TSV, NLICS.
result Transformer models are more affected by semantic drift.
Detects systematic anomalies in consumer complaints using NLP.
problem Detecting small, frequent anomalies in consumer complaints.
method NLP conversion of narratives, followed by anomaly detection algorithm.
result Demonstrates effectiveness of NLP for detecting systematic anomalies.
BigBird improves transformer performance on NLP tasks with longer sequences.
problem Quadratic dependency on sequence length in transformer models.
method Sparse attention mechanism reducing quadratic dependency to linear.
result Significant improvement in performance on NLP tasks like QA and summarization.
BERTopic enhances stock market prediction by analyzing sentiment in topic models.
problem Improving stock price prediction accuracy using sentiment analysis.
method Employed BERTopic for sentiment analysis of stock market comments integrated with deep learning models.
result Enhanced model performance through topic sentiment integration.
This paper predicts legal proceedings status using NLP and machine learning.
problem Classify Brazilian legal proceedings into archived, active, and suspended categories.
method Combined NLP techniques with machine learning to classify legal proceedings sequences.
result Achieved maximum accuracy of 93% and top average F1 Scores of 89% (macro) and 93% (weighted).
Federated learning improves by training central model with client model outputs.
problem Direct averaging of client models is limited in FL.
method Ensemble distillation for model fusion.
result Central model trained faster with fewer communication rounds.
BERTino is a lightweight Italian DistilBERT model for NLP tasks.
problem High computational and memory demands of large language models.
method Developed a DistilBERT model tailored for Italian language.
result F1 scores comparable to BERTBASE with significant speed improvements.
Paper develops a method for valid inference using language model predictions from verbal autopsy narratives.
problem Valid inference from verbal autopsy narratives for public health decision-making.
method Develops multiPPI++ method for valid inference using NLP techniques for COD prediction.
result Demonstrates the effectiveness of multiPPI++ in handling transportability issues and recovering ground truth estimates.