On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

count % of corpus

10 results for “pretrain-finetune”

Hierarchical Retrieval: The Geometry and a Pretrain-Finetune Recipecs.IR

Improves retrieval accuracy for hierarchical documents, especially for distant matches.

problem Limited expressive power of dual encoder models in hierarchical retrieval.

method Proves feasibility of DEs for HR, introduces pretrain-finetune recipe to improve long-distance retrieval.

result Pretrain-finetune boosts recall on long-distance pairs from 19% to 76%.

hierarchical retrieval pretrain-finetune long-distance retrieval Euclidean geometry wordnet hierarchy

2025-09-19abs ↗pdf ↗

The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shiftcs.LG

Study shows pretraining and finetuning can effectively tackle covariate shift in linear regression.

problem Linear regression under covariate shift where source and target distributions differ but conditional distribution remains similar.

method Pretraining on source data and finetuning on target data using online SGD.

result Transfer learning with

O(N^2)

source data is as effective as supervised learning with

N

target data.

covariate shift linear regression pretraining finetuning online SGD

2022-08-03abs ↗pdf ↗

Multi-Stage Influence Functioncs.LG

Paper introduces a multi-stage influence function to track model predictions.

problem Improving natural language processing and computer vision performance.

method Develops a multi-stage influence function to track predictions from finetuned models back to pretraining data.

result Identifies pretraining examples contributing most to finetuning task predictions.

influence function pretraining finetuning natural language processing computer vision

2020-07-17abs ↗pdf ↗

When Does Self-Supervision Help Graph Convolutional Networks?cs.LG

Self-supervision improves GCNs' generalizability and robustness.

problem Improving graph convolutional networks' performance.

method Three mechanisms of self-supervision, multi-task learning, and graph adversarial training.

result Self-supervision enhances GCNs' robustness and generalizability.

graph convolutional networks self-supervision multi-task learning graph adversarial training generalizability

2020-06-16abs ↗pdf ↗

TransFusion: Covariate-Shift Robust Transfer Learning for High-Dimensional Regressionstat.ML

Proposes a method to improve regression model performance with limited target data using fused-regularizer.

problem Model shifts and covariate shifts in high-dimensional regression.

method Two-step method with fused-regularizer to leverage source data for target task.

result Robust to covariate shifts, minimax-optimal under certain conditions, and validated by numerical tests.

transfer learning covariate shift high-dimensional regression fused-regularizer robustness

2024-04-01abs ↗pdf ↗

CERT: Contrastive Self-supervised Learning for Language Understandingcs.CL

CERT improves language understanding by contrastively learning sentence-level semantics.

problem Lack of sentence-level semantics in existing pretraining tasks.

method Contrastive self-supervised learning at the sentence level using back-translation augmentations.

result CERT outperforms BERT on 7 out of 11 GLUE benchmark tasks, achieving the same performance as BERT on 2 tasks.

BERT GPT contrastive learning language understanding pretraining

2020-05-16abs ↗pdf ↗

Physen-Noise2Noise: Physics-Guided Self-Supervised Defocus Deblurring with Bias Correction under Low-Light Conditionscs.CV

Physen-Noise2Noise tackles defocus deblurring in low-light conditions with physics-guided self-supervised learning.

problem Defocus deblurring in low-light conditions with complex biased noise.

method Physics-guided self-supervised deblurring framework that leverages noisy multi-frame observations and a learnable noise bias parameter.

result Physen-Noise2Noise consistently outperforms state-of-the-art methods in defocus deblurring with complex biased noise.

defocus deblurring self-supervised learning low-light conditions physics-guided complex biased noise

2026-05-23abs ↗pdf ↗

Enhancing Compositional Generalization via Compositional Feature Alignmentcs.CV

CFA improves model's ability to generalize across unseen domain-class combinations.

problem Challenges in real-world machine learning applications due to data distribution shifts and limited training data.

method Developed Compositional Feature Alignment (CFA) technique to improve CG ability of pretrained models.

result CFA outperforms common finetuning techniques in compositional generalization.

Compositional Generalization Feature Alignment Pretrained Models Data Distribution Shifts Multi-Class Multi-Domain

2024-02-05abs ↗pdf ↗

Hedging with Sparse Reward Reinforcement Learningq-fin.CP

This thesis proposes a derivatives hedging framework using deep learning and reinforcement learning.

problem Traditional hedging models fail in complex, uncertain markets due to assumptions like continuous trading and zero transaction costs.

method Integrates deep learning and reinforcement learning, using a spatiotemporal attention-based Transformer for probabilistic forecasting and hedging.

result The proposed method significantly outperforms traditional approaches in U.S. and Chinese financial markets.

derivatives hedging reinforcement learning Transformer sparse rewards

2025-03-06abs ↗pdf ↗

Attention-based CNN-LSTM and XGBoost hybrid model for stock predictionq-fin.ST

Hybrid model predicts stock prices with high accuracy.

problem Complex volatility of stock market makes traditional models unsatisfactory.

method Attention-based CNN-LSTM and XGBoost integrated model.

result Hybrid model improves prediction accuracy.

stock prediction hybrid model CNN-LSTM XGBoost attention mechanism

2022-04-06abs ↗pdf ↗