Improved language model for French clinical reports achieves state-of-the-art performance in medical NLP tasks.
problem Lack of specialized language models for French clinical reports.
method Adapted a general pre-trained language model (CamemBERT) to French clinical reports using a corpus of 21M reports.
result Pretrained and fine-tuned models improved F1-score by 3 percentage points on APMed task.
Extracting information from electronic health records (EHR) is a challenging task since it requires prior knowledge of the reports and some natural language processing algorithm (NLP). With the growing number of EHR implementations, such knowledge is increasingly challenging to obtain in an efficient manner. We address…
Study re-evaluates MIMIC-III codes, finding many are under-coded.
problem Validity of MIMIC-III clinical codes is questionable.
method Open-source, reproducible methodology for assessing codes.
result Most frequently assigned codes are under-coded up to 35%
Objective: We investigate whether deep learning techniques for natural language processing (NLP) can be used efficiently for patient phenotyping. Patient phenotyping is a classification task for determining whether a patient has a medical condition, and is a crucial part of secondary analysis of healthcare data. We ass…
With widespread adoption of electronic health records, there is an increased emphasis for predictive models that can effectively deal with clinical time-series data. Powered by Recurrent Neural Network (RNN) architectures with Long Short-Term Memory (LSTM) units, deep neural networks have achieved state-of-the-art resu…
BERT-XML automates ICD coding from EHR notes using BERT pretraining.
problem Manual ICD coding is time-consuming and expensive in hospitals.
method Trained a BERT model on EHR notes, adapting it for multi-label attention.
result First large-scale ICD-10 classification model using millions of EHR notes.
Gaussian Processes outperform other models in estimating uncertainty for radiology report observation detection.
problem Uncertainty quantification in automatic data labelling for semi-supervised learning in clinical NLP.
method Investigation of uncertainty estimates from various predictive models using NLPP and MMPCL metrics.
result Gaussian Processes provide superior performance in quantifying uncertainty for radiology report observation detection.
Machine learning and C-NLP improve emergency department triage accuracy.
problem Improving accuracy in emergency department triage to manage increased volume.
method Historical EHR data, clinical natural language processing, and ML algorithms (KATE).
result KATE predicts ESI acuity assignments 75.9% of the time, significantly better than nurses.
QUACKIE creates a new benchmark for NLP interpretability.
problem Evaluating NLP interpretability methods is challenging due to biased ground truths.
method Formulated a custom classification task from question-answering datasets, generating unbiased ground truths.
result Demonstrated the effectiveness of current interpretability methods on the new benchmark.
This study uses NLP to detect financial risks from documents.
problem Detecting and predicting financial risks in documents.
method NLP model design, text preprocessing, feature extraction, machine learning.
result NLP model effectively identifies and predicts financial risks.
The digitalization of stored information in hospitals now allows for the exploitation of medical data in text format, as electronic health records (EHRs), initially gathered for other purposes than epidemiology. Manual search and analysis operations on such data become tedious. In recent years, the use of natural langu…
TX-Ray analyzes and quantifies model knowledge transfer in NLP.
problem Insufficient methods for explaining and quantifying model knowledge transfer in NLP.
method Modified computer vision explainability principle to NLP, visualizing feature preference distributions.
result TX-Ray reveals how self-supervised models learn linguistic abstractions and improves generalization.
New measure corrects news bias in NLP stock return forecasting.
problem Improving stock return and volatility forecasting accuracy.
method Hype-Adjusted Probability Measure, sentiment score equation.
result Significantly improved forecast accuracy for U.S. semiconductor tickers.
This paper analyzes crypto white papers under MiCAR, highlighting NLP's role.
problem Regulatory changes in crypto white papers under MiCAR.
method Survey of existing NLP applications, analysis of MiCAR changes.
result NLP can assist in regulatory compliance and white paper analysis.
This paper improves NLP interpretability by using sentence segments instead of words.
problem Limitations of word-based sampling in explaining complex BERT models.
method Using sentence segments as elementary building blocks for NLP interpretability.
result Improved fidelity of the explainer on a benchmark classification task.
Due to recent technical and scientific advances, we have a wealth of information hidden in unstructured text data such as offline/online narratives, research articles, and clinical reports. To mine these data properly, attributable to their innate ambiguity, a Word Sense Disambiguation (WSD) algorithm can avoid numbers…
SusGen-GPT improves financial NLP and ESG report generation.
problem Lack of advanced NLP tools for finance and ESG domains.
method Developed SusGen-30K dataset and SusGen-GPT models.
result Achieved state-of-the-art performance in financial NLP tasks.
MDA optimizer performs similarly to SGD+M in CV and Adam in NLP.
problem Performance degradation due to choosing the wrong optimizer.
method Modernized Dual Averaging (MDA) optimizer, inspired by dual averaging.
result MDA performs as well as SGD+M in CV and as Adam in NLP.
We introduce HUBERT which combines the structured-representational power of Tensor-Product Representations (TPRs) and BERT, a pre-trained bidirectional Transformer language model. We show that there is shared structure between different NLP datasets that HUBERT, but not BERT, is able to learn and leverage. We validate …
Survey on using large models to train smaller datasets in NLP.
problem Lack of large datasets and computing resources for NLP tasks.
method Analysis of recent transfer learning approaches in NLP.
result Increased demand for transfer learning in NLP due to large models.
New pruning method retains model expressiveness for NLP tasks.
problem Pruning large pretrained transformer models for real-world deployment.
method Mixture Gaussian Prior Pruning (MGPP) algorithm.
result MGPP outperforms existing pruning methods in high sparsity settings.
BERT outperforms traditional machine learning in text classification tasks.
problem Comparing BERT to traditional machine learning methods for text classification.
method Empirical testing of BERT against TF-IDF-based machine learning models in various scenarios.
result BERT demonstrates superior performance and independence from text features.
Hungarian text processing improved with efficient, accurate NLP pipelines.
problem Improving text processing for Hungarian language.
method Implemented in spaCy framework, focusing on efficiency and accuracy.
result Near state-of-the-art performance in all text preprocessing steps.
HuSpaCy offers an industrial-grade Hungarian NLP toolkit.
problem Lack of suitable open-source Hungarian NLP pipelines.
method Built on spaCy, HuSpaCy includes lemmatization, morphosyntactic analysis, entity recognition, and word embeddings.
result HuSpaCy achieves high accuracy with resource-efficient prediction.
The Hype Index measures media attention to equities using NLP.
problem Quantifying media attention to equities for volatility analysis.
method Constructs News Count-Based and Capitalization Adjusted Hype Indices using NLP.
result The Hype Index family provides valuable tools for stock volatility analysis.
The paper improves cryptocurrency price forecasting using deep learning and NLP on financial, blockchain, and social media data.
problem Improving cryptocurrency price forecasting accuracy and profitability.
method Integrates financial, blockchain, and social media data; applies BART MNLI model for sentiment analysis; uses deep learning NLP models; compares with traditional methods; uses local extrema as predictive targets.
result Significantly improves forecasting accuracy and profitability of cryptocurrency price predictions.
COCKATIEL explains neural net models on NLP tasks by identifying meaningful concepts.
problem Transformer models are complex and hard to interpret.
method COCKATIEL uses NMF and sensitivity analysis to identify and rank concepts used by the model.
result COCKATIEL provides accurate and meaningful explanations without affecting model performance.
Infrastructure monitors AI/ML radiology models across multiple sites.
problem Monitoring and improving AI/ML radiology models across multiple sites.
method Interactive radiology reporting, centralized cloud system, post-marketing surveillance.
result Efficient monitoring and iterative development of AI/ML models without radiologist burden.
This paper presents a deep-learning based traffic classification method for identifying multiple streaming video sources at the same time within an encrypted tunnel. The work defines a novel feature inspired by Natural Language Processing (NLP) that allows existing NLP techniques to help the traffic classification. The…
This paper explores how NLP enhances insurance data analysis.
problem Traditional insurance data limitations and need for alternative data.
method Application of NLP techniques to transform and analyze unstructured text data.
result NLP techniques improve insurance data analysis and risk assessment.
Study improves cryptocurrency price prediction using unlabeled text data.
problem Predicting cryptocurrency returns from unlabelled text data.
method Introduced weak learning approach to finetune BERT on unlabeled text data.
result Finetuning pretrained NLP models with weak labels enhances forecast accuracy.
NLP techniques improve drug discovery by analyzing chemical and protein text.
problem Improving drug discovery through better analysis of chemical and protein text.
method Natural language processing techniques applied to biochemical entities.
result Enhanced prediction of molecular properties and design of novel molecules.
Framework quantifies financial NLP robustness under regime shifts.
problem Semantic and causal drift in financial news narratives.
method Four metrics: FCAS, PCS, TSV, NLICS.
result Transformer models are more affected by semantic drift.
Detects systematic anomalies in consumer complaints using NLP.
problem Detecting small, frequent anomalies in consumer complaints.
method NLP conversion of narratives, followed by anomaly detection algorithm.
result Demonstrates effectiveness of NLP for detecting systematic anomalies.
BigBird improves transformer performance on NLP tasks with longer sequences.
problem Quadratic dependency on sequence length in transformer models.
method Sparse attention mechanism reducing quadratic dependency to linear.
result Significant improvement in performance on NLP tasks like QA and summarization.
BERTopic enhances stock market prediction by analyzing sentiment in topic models.
problem Improving stock price prediction accuracy using sentiment analysis.
method Employed BERTopic for sentiment analysis of stock market comments integrated with deep learning models.
result Enhanced model performance through topic sentiment integration.
This paper predicts legal proceedings status using NLP and machine learning.
problem Classify Brazilian legal proceedings into archived, active, and suspended categories.
method Combined NLP techniques with machine learning to classify legal proceedings sequences.
result Achieved maximum accuracy of 93% and top average F1 Scores of 89% (macro) and 93% (weighted).
BERTino is a lightweight Italian DistilBERT model for NLP tasks.
problem High computational and memory demands of large language models.
method Developed a DistilBERT model tailored for Italian language.
result F1 scores comparable to BERTBASE with significant speed improvements.
Paper develops a method for valid inference using language model predictions from verbal autopsy narratives.
problem Valid inference from verbal autopsy narratives for public health decision-making.
method Develops multiPPI++ method for valid inference using NLP techniques for COD prediction.
result Demonstrates the effectiveness of multiPPI++ in handling transportability issues and recovering ground truth estimates.
Researchers develop a method to interpret GNNs by identifying unnecessary edges in NLP models.
problem Understanding which parts of graphs contribute to NLP model predictions.
method A post-hoc method using differentiable edge masking to identify and drop unnecessary edges.
result Large proportions of edges can be dropped without affecting model performance, providing insights into model predictions.
Cryptonite tests NLP models with cryptic crossword clues.
problem Ambiguity in language poses a challenge for NLP models.
method Cryptic crossword dataset based on cryptic clues.
result Fine-tuning T5-Large on Cryptonite achieves only 7.6% accuracy.
Paper improves neural network models for MOOC student course prediction.
problem Improving neural network-based predictive methods for MOOC student course trajectory modeling.
method Enhanced LSTM networks and Transformer architecture applied to MOOC student data.
result Improved accuracy in predicting MOOC student course trajectories.
The lottery ticket hypothesis proposes that over-parameterization of deep neural networks (DNNs) aids training by increasing the probability of a "lucky" sub-network initialization being present rather than by helping the optimization process (Frankle & Carbin, 2019). Intriguingly, this phenomenon suggests that initial…
Study uses LLMs and NLP to analyze 14,342 algorithmic trading papers.
problem Analyzing the vast literature on algorithmic trading.
method Filtering and analyzing 136 million papers, comparing traditional and modern methods.
result Demonstrates the effectiveness of LLMs in refining and answering complex questions.
Optimal Word2Vec hyper-parameters improve NLP tasks.
problem Finding the best Word2Vec hyper-parameters for NLP tasks.
method Empirical evaluation of various hyper-parameter combinations on NLP tasks.
result The best hyper-parameters vary by task, and high analogy scores don't always correlate with performance.
SCROLLS benchmarks long text NLP tasks, improving existing models.
problem Short NLP benchmarks ignore long texts; SCROLLS addresses this.
method Handpicked long-text datasets for summarization, QA, and inference tasks.
result Improvement potential on SCROLLS tasks, as indicated by initial baselines.
Deep neural network models have recently achieved state-of-the-art performance gains in a variety of natural language processing (NLP) tasks (Young, Hazarika, Poria, & Cambria, 2017). However, these gains rely on the availability of large amounts of annotated examples, without which state-of-the-art performance is rare…
New algorithm avoids spurious sharpness minimization for NLP models.
problem SAM fails in NLP, leading to performance degradation.
method Developed Functional-SAM, which modifies logit statistics instead of function geometry.
result Functional-SAM and combined methods outperform AdamW and SAM in NLP tasks.