This paper presents several strategies that can improve neural network-based predictive methods for MOOC student course trajectory modeling, applying multiple ideas previously applied to tackle NLP (Natural Language Processing) tasks. In particular, this paper investigates LSTM networks enhanced with two forms of regul…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “NLP networks”
MDA optimizer performs similarly to SGD+M in CV and Adam in NLP.
problem Performance degradation due to choosing the wrong optimizer.
method Modernized Dual Averaging (MDA) optimizer, inspired by dual averaging.
result MDA performs as well as SGD+M in CV and as Adam in NLP.
Researchers develop a method to interpret GNNs by identifying unnecessary edges in NLP models.
problem Understanding which parts of graphs contribute to NLP model predictions.
method A post-hoc method using differentiable edge masking to identify and drop unnecessary edges.
result Large proportions of edges can be dropped without affecting model performance, providing insights into model predictions.
A benchmark for NLP models trained on text datasets.
problem Limited access to high-performance clusters for NAS experiments.
method Created a search space for recurrent neural networks on text datasets and trained 14k architectures.
result Demonstrated the potential of precomputed NAS results for NLP.
This paper presents a deep-learning based traffic classification method for identifying multiple streaming video sources at the same time within an encrypted tunnel. The work defines a novel feature inspired by Natural Language Processing (NLP) that allows existing NLP techniques to help the traffic classification. The…
The lottery ticket hypothesis proposes that over-parameterization of deep neural networks (DNNs) aids training by increasing the probability of a "lucky" sub-network initialization being present rather than by helping the optimization process (Frankle & Carbin, 2019). Intriguingly, this phenomenon suggests that initial…
Convolutional neural networks have been successfully applied to various NLP tasks. However, it is not obvious whether they model different linguistic patterns such as negation, intensification, and clause compositionality to help the decision-making process. In this paper, we apply visualization techniques to observe h…
Since deep learning became a key player in natural language processing (NLP), many deep learning models have been showing remarkable performances in a variety of NLP tasks, and in some cases, they are even outperforming humans. Such high performance can be explained by efficient knowledge representation of deep learnin…
BERTino is a lightweight Italian DistilBERT model for NLP tasks.
problem High computational and memory demands of large language models.
method Developed a DistilBERT model tailored for Italian language.
result F1 scores comparable to BERTBASE with significant speed improvements.
QUACKIE creates a new benchmark for NLP interpretability.
problem Evaluating NLP interpretability methods is challenging due to biased ground truths.
method Formulated a custom classification task from question-answering datasets, generating unbiased ground truths.
result Demonstrated the effectiveness of current interpretability methods on the new benchmark.
New method compresses LSTM networks using MPS tensor trains.
problem Challenges in maintaining performance of compressed RNNs.
method Use of MPS tensor trains for LSTM network compression.
result MPS tensor trains outperform MPOs in storage and inference time.
This study uses NLP to detect financial risks from documents.
problem Detecting and predicting financial risks in documents.
method NLP model design, text preprocessing, feature extraction, machine learning.
result NLP model effectively identifies and predicts financial risks.
We address the problem of graph classification based only on structural information. Inspired by natural language processing techniques (NLP), our model sequentially embeds information to estimate class membership probabilities. Besides, we experiment with NLP-like variational regularization techniques, making the mode…
Deep neural network models have recently achieved state-of-the-art performance gains in a variety of natural language processing (NLP) tasks (Young, Hazarika, Poria, & Cambria, 2017). However, these gains rely on the availability of large amounts of annotated examples, without which state-of-the-art performance is rare…
Optimal Word2Vec hyper-parameters improve NLP tasks.
problem Finding the best Word2Vec hyper-parameters for NLP tasks.
method Empirical evaluation of various hyper-parameter combinations on NLP tasks.
result The best hyper-parameters vary by task, and high analogy scores don't always correlate with performance.
New algorithm avoids spurious sharpness minimization for NLP models.
problem SAM fails in NLP, leading to performance degradation.
method Developed Functional-SAM, which modifies logit statistics instead of function geometry.
result Functional-SAM and combined methods outperform AdamW and SAM in NLP tasks.
ST-GAN predicts stock trends using financial news and data.
problem Predicting financial trends in stock markets.
method ST-GAN combines NLP and technical indicators using GAN technology.
result Significant improvement over existing models in stock price forecasting.
What Should I Learn First: Introducing LectureBank for NLP Education and Prerequisite Chain Learningcs.CY
Recent years have witnessed the rising popularity of Natural Language Processing (NLP) and related fields such as Artificial Intelligence (AI) and Machine Learning (ML). Many online courses and resources are available even for those without a strong background in the field. Often the student is curious about a specific…
New measure corrects news bias in NLP stock return forecasting.
problem Improving stock return and volatility forecasting accuracy.
method Hype-Adjusted Probability Measure, sentiment score equation.
result Significantly improved forecast accuracy for U.S. semiconductor tickers.
This paper analyzes crypto white papers under MiCAR, highlighting NLP's role.
problem Regulatory changes in crypto white papers under MiCAR.
method Survey of existing NLP applications, analysis of MiCAR changes.
result NLP can assist in regulatory compliance and white paper analysis.
This paper improves NLP interpretability by using sentence segments instead of words.
problem Limitations of word-based sampling in explaining complex BERT models.
method Using sentence segments as elementary building blocks for NLP interpretability.
result Improved fidelity of the explainer on a benchmark classification task.
Convolutional neural networks (CNNs) have recently emerged as a popular building block for natural language processing (NLP). Despite their success, most existing CNN models employed in NLP share the same learned (and static) set of filters for all input sentences. In this paper, we consider an approach of using a smal…
Model agnostic feature importance for DNNs in NLP.
problem Characterizing DNNs as black boxes and explaining their decision-making process.
method Phrase-wise feature importance calculation for model agnostic DNNs.
result Robust and generalizable approach to feature importance calculation.
This work explains why GANs are less used for NLP tasks.
problem Why adversarial approaches like GANs are not widely used for NLP tasks.
method Theoretical analysis and reductions showing that maximizing likelihood is equivalent to minimizing distinguishability for certain models.
result Maximizing likelihood is as effective as minimizing distinguishability for NLP tasks.
Improved NLP performance with fewer parameters and less data using conditional multi-task learning.
problem Challenges in transferring knowledge across different NLP tasks, including overfitting, forgetting, and negative transfer.
method Proposes a novel Transformer architecture with conditional attention and task-conditioned modules for efficient parameter sharing and mitigating forgetting.
result Achieves state-of-the-art performance on 26 NLP tasks with 66% less data and 50% fewer parameters compared to existing methods.
Improved language model for French clinical reports achieves state-of-the-art performance in medical NLP tasks.
problem Lack of specialized language models for French clinical reports.
method Adapted a general pre-trained language model (CamemBERT) to French clinical reports using a corpus of 21M reports.
result Pretrained and fine-tuned models improved F1-score by 3 percentage points on APMed task.
SusGen-GPT improves financial NLP and ESG report generation.
problem Lack of advanced NLP tools for finance and ESG domains.
method Developed SusGen-30K dataset and SusGen-GPT models.
result Achieved state-of-the-art performance in financial NLP tasks.
The paper quantifies the regularity of attention operations.
problem Quantifying the regularity of attention operations.
method Proposes a new mathematical framework using measure theory and integral operators.
result Proves attention operation is Lipschitz continuous on compact domains and provides an estimate of its Lipschitz constant.
Layer-wise relevance propagation (LRP) is a recently proposed technique for explaining predictions of complex non-linear classifiers in terms of input variables. In this paper, we apply LRP for the first time to natural language processing (NLP). More precisely, we use it to explain the predictions of a convolutional n…
We introduce HUBERT which combines the structured-representational power of Tensor-Product Representations (TPRs) and BERT, a pre-trained bidirectional Transformer language model. We show that there is shared structure between different NLP datasets that HUBERT, but not BERT, is able to learn and leverage. We validate …
When applying machine learning to problems in NLP, there are many choices to make about how to represent input texts. These choices can have a big effect on performance, but they are often uninteresting to researchers or practitioners who simply need a module that performs well. We propose an approach to optimizing ove…
Quantitative model predicts Sri Lankan stock market using NLP, clustering, and time-series forecasting.
problem Predicting economic regimes and market signals in Sri Lankan stock indices.
method Integrates NLP, clustering, and time-series forecasting; uses FinBERT for sentiment analysis, UMAP/HDBSCAN for clustering, and GRU/LSTM for forecasting.
result GRU model achieves 80.1% R-squared for daily closing price forecasts.
Survey on using large models to train smaller datasets in NLP.
problem Lack of large datasets and computing resources for NLP tasks.
method Analysis of recent transfer learning approaches in NLP.
result Increased demand for transfer learning in NLP due to large models.
New pruning method retains model expressiveness for NLP tasks.
problem Pruning large pretrained transformer models for real-world deployment.
method Mixture Gaussian Prior Pruning (MGPP) algorithm.
result MGPP outperforms existing pruning methods in high sparsity settings.
DCoM uses deep neural networks to detect semantic data types from raw column values.
problem Detecting semantic data types from dirty and unseen data.
method DCoM employs multi-input NLP-based deep neural networks trained on 686,765 data columns.
result DCoM outperforms existing methods significantly on 78 different semantic data types.
The paper analyzes how news sentiment of companies can affect market movements.
problem Understanding how news sentiment impacts market performance and volatility.
method Applied NLP techniques to analyze news sentiment of 87 companies over 7 years.
result Strong media sentiment towards one company can indicate significant changes in sentiment towards related companies.
BERT outperforms traditional machine learning in text classification tasks.
problem Comparing BERT to traditional machine learning methods for text classification.
method Empirical testing of BERT against TF-IDF-based machine learning models in various scenarios.
result BERT demonstrates superior performance and independence from text features.
Hungarian text processing improved with efficient, accurate NLP pipelines.
problem Improving text processing for Hungarian language.
method Implemented in spaCy framework, focusing on efficiency and accuracy.
result Near state-of-the-art performance in all text preprocessing steps.
HuSpaCy offers an industrial-grade Hungarian NLP toolkit.
problem Lack of suitable open-source Hungarian NLP pipelines.
method Built on spaCy, HuSpaCy includes lemmatization, morphosyntactic analysis, entity recognition, and word embeddings.
result HuSpaCy achieves high accuracy with resource-efficient prediction.
The Hype Index measures media attention to equities using NLP.
problem Quantifying media attention to equities for volatility analysis.
method Constructs News Count-Based and Capitalization Adjusted Hype Indices using NLP.
result The Hype Index family provides valuable tools for stock volatility analysis.
While state-of-the-art NLP explainability (XAI) methods focus on explaining per-sample decisions in supervised end or probing tasks, this is insufficient to explain and quantify model knowledge transfer during (un-)supervised training. Thus, for TX-Ray, we modify the established computer vision explainability principle…
The paper improves cryptocurrency price forecasting using deep learning and NLP on financial, blockchain, and social media data.
problem Improving cryptocurrency price forecasting accuracy and profitability.
method Integrates financial, blockchain, and social media data; applies BART MNLI model for sentiment analysis; uses deep learning NLP models; compares with traditional methods; uses local extrema as predictive targets.
result Significantly improves forecasting accuracy and profitability of cryptocurrency price predictions.
Paper proposes a hybrid model for financial time series prediction using sentiment analysis.
problem Challenges in forecasting in non-stationary, complex environments with heterogeneous data.
method Hybrid model combining GANs with NLP-based sentiment analysis.
result Hybrid model enhances robustness in non-stationary environments.
Solves the challenge of retrieving item-specific financial information from Form 10-Q filings.
problem Retrieving item-specific information from Form 10-Q filings with varying formats and machine-readable hierarchy.
method Complements a rule-based algorithm with a Convolutional Neural Network (CNN) image classifier to itemize 10-Q files.
result Demonstrates a generalized pipeline for rapid data retrieval from a large volume of textual data.
New findings show BERT subnetworks can train independently and transfer to various tasks.
problem Finding smaller subnetworks that can train independently and transfer to other tasks.
method Examined pre-trained BERT models for subnetworks that can train independently and transfer to various downstream tasks.
result Found subnetworks at 40% to 90% sparsity that can train independently and transfer to various tasks.
Fruit fly brain network learns word embeddings using sparse binary codes.
problem Learning semantic word representations from text.
method Inspired by mushroom body neural network, sparse binary hash codes.
result Fruit fly network achieves comparable NLP performance with reduced resources.
COCKATIEL explains neural net models on NLP tasks by identifying meaningful concepts.
problem Transformer models are complex and hard to interpret.
method COCKATIEL uses NMF and sensitivity analysis to identify and rank concepts used by the model.
result COCKATIEL provides accurate and meaningful explanations without affecting model performance.
A new method compresses NLP networks by using multiple subspaces instead of a single one.
problem Large errors in compressing NLP networks using a single subspace.
method Projective clustering to find multiple subspaces that minimize squared distances.
result Networks are more accurate and smaller compared to standard matrix factorization.