On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

count % of corpus

10 results for “DistilBERT”

BERTino: an Italian DistilBERT modelcs.CL

BERTino is a lightweight Italian DistilBERT model for NLP tasks.

problem High computational and memory demands of large language models.

method Developed a DistilBERT model tailored for Italian language.

result F1 scores comparable to BERTBASE with significant speed improvements.

BERTino DistilBERT Italian NLP Lightweight model F1 scores

2023-03-31abs ↗pdf ↗

Unveiling Hedge Funds: Topic Modeling and Sentiment Correlation with Fund Performanceq-fin.CP

Study uses topic modeling and sentiment analysis to uncover hedge fund performance insights.

problem Hedge fund opacity and limited disclosure make them hard to analyze.

method Applied topic modeling and sentiment analysis to hedge fund documents using DistilBERT and Top2Vec.

result Automated topic modeling and sentiment analysis can predict hedge fund performance.

hedge funds topic modeling sentiment analysis fund performance DistilBERT

2025-12-07abs ↗pdf ↗

StockEmotions: Discover Investor Emotions for Financial Sentiment Analysis and Multivariate Time Seriescs.CL

StockEmotions dataset for financial sentiment and emotion analysis.

problem Limited resources for financial sentiment analysis.

method Collects 10,000 English comments from StockTwits, categorizes emotions into 12 classes.

result DistilBERT outperforms other models in sentiment classification, and Temporal Attention LSTM model achieves best performance in multivariate time series forecasting.

financial sentiment emotion analysis StockEmotions DistilBERT Temporal Attention LSTM

2023-01-23abs ↗pdf ↗

Active Learning to Guide Labeling Efforts for Question Difficulty Estimationcs.LG

Active learning reduces labeling efforts for QDE models.

problem Efficiently labeling data for QDE models with minimal human effort.

method Active learning approach using PowerVariance acquisition function.

result Active learning achieves performance close to fully supervised models with only 10% labeled data.

active learning QDE Transformer DistilBERT Monte Carlo dropout

2024-09-14abs ↗pdf ↗

Comparing Natural Language Processing Techniques for Alzheimer's Dementia Prediction in Spontaneous Speechcs.LG

Models predict Alzheimer's Dementia from spontaneous speech with high accuracy.

problem Early diagnosis of Alzheimer's Dementia (AD) through spontaneous speech analysis.

method Compared natural language processing techniques including SVM, GBDT, CRFs, and Transformer-based models.

result Top models achieve 0.81-0.82 test set scores for AD vs controls and 4.58 RMSE for Mental Mini State Exam scores.

Alzheimer's Dementia Spontaneous Speech Natural Language Processing Support Vector Machines Gradient Boosting Decision Trees

2020-06-12abs ↗pdf ↗

EmTract: Extracting Emotions from Social Mediaq-fin.PR

EmTract extracts emotions from financial social media text.

problem Understanding investor emotions in financial markets.

method Annotated data, DistilBERT model, embedding space augmentation.

result EmTract outperforms existing emotion classifiers.

Emotion extraction Social media Financial markets Investor emotions Emojis

2021-12-07abs ↗pdf ↗

Classifying multilingual party manifestos: Domain transfer across country, time, and genrecs.CL

BERT models can classify multilingual party manifestos across different dimensions.

problem Costly manual annotation of large corpora in social science.

method Domain transfer across geographical locations, languages, time, and genre using fine-tuned transformer models.

result BERT models can be applied to future data with similar performance.

BERT domain transfer political manifestos multilingual social science

2023-07-31abs ↗pdf ↗

Cooking Is All About People: Comment Classification On Cookery Channels Using BERT and Classification Models (Malayalam-English Mix-Code)cs.CL

Comment classification on cookery channels using BERT and traditional models.

problem Volume of multilingual comments, variable lengths, slang, symbols, and abbreviations make comment classification challenging.

method Evaluated traditional machine learning models (Naive Bayes, KNN, SVM, Random Forest, Decision Trees) and BERT-based models (BERT, DISTILBERT, XLM) for multilingual comment classification.

result XLM was the top-performing BERT model with an accuracy of 67.31, while Random Forest with Term Frequency Vectorizer was the best traditional model with 63.59 accuracy.

comment classification multilingual cookery channels BERT traditional models

2020-06-15abs ↗pdf ↗

Compressed Deep Networks: Goodbye SVD, Hello Robust Low-Rank Approximationcs.LG

This paper improves neural network compression by using robust low-rank approximations.

problem Neural network compression sensitivity to outliers.

method Introduces robust low-rank approximations using

\ell_p

norms (for

p\in [1,2]

) and provides efficient algorithms.

result Achieves up to 28% compression with minimal accuracy loss compared to existing methods.

neural network compression robust low-rank approximation SVD Lasso Regression GLUE benchmark

2020-09-11abs ↗pdf ↗

Deep Learning Meets Projective Clusteringcs.LG

A new method compresses NLP networks by using multiple subspaces instead of a single one.

problem Large errors in compressing NLP networks using a single subspace.

method Projective clustering to find multiple subspaces that minimize squared distances.

result Networks are more accurate and smaller compared to standard matrix factorization.

deep learning projective clustering NLP networks compression subspaces

2020-10-08abs ↗pdf ↗