On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

count % of corpus

11 results for “prompt-tuning”

Fundamental Limits of Prompt Tuning Transformers: Universality, Capacity and Efficiencycs.LG

Single-head transformers with a single self-attention layer can approximate any sequence-to-sequence function and are efficient under certain conditions.

problem Statistical and computational limits of prompt tuning for transformer-based models.

method Investigation of single-head transformers with a single self-attention layer, proving universality and efficiency under SETH.

result Existence of almost-linear time prompt tuning inference algorithms under certain conditions.

transformers prompt tuning universality SETH approximation

2024-11-25abs ↗pdf ↗

InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understandingcs.CL

InfoPrompt improves soft prompt tuning by maximizing mutual information, leading to better performance.

problem High sensitivity of prompt tuning to initial conditions and insufficient task-relevant information.

method Develops an information-theoretic framework to maximize mutual information between prompts and model parameters, using novel loss functions.

result InfoPrompt accelerates convergence and outperforms traditional methods.

soft prompt tuning information-theoretic mutual information few-shot learning natural language understanding

2023-06-08abs ↗pdf ↗

On the Role of Attention in Prompt-tuningcs.LG

Study on how attention in prompt-tuning affects large language models.

problem Limited theoretical understanding of prompt-tuning and attention in LLMs.

method Exploration of prompt-tuning for one-layer attention architectures, contextual mixture-models, and self-contained prompt-attention model.

result Softmax-prompt-attention is more expressive than self-attention and linear-prompt-attention under contextual data model.

prompt-tuning attention mechanism large language models contextual mixture-models softmax-prompt-attention

2023-06-06abs ↗pdf ↗

ADAPT to Robustify Prompt Tuning Vision Transformerscs.LG

ADAPT improves robustness of Vision Transformers without full model fine-tuning.

problem Vulnerability of Vision Transformers to adversarial attacks.

method Parameter-efficient prompt tuning with ADAPT framework for adaptive adversarial training.

result ADAPT achieves robust accuracy of ~40% w.r.t. SOTA methods using only ~1% of the parameters.

Vision Transformers Adversarial Attacks Prompt Tuning Robustness Adaptive Training

2024-03-19abs ↗pdf ↗

Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuningcs.LG

Analysis of pretrained models' effectiveness in downstream tasks.

problem Understanding why pretrained models perform well in NLP tasks.

method Analyzed head and prompt tuning approaches using latent variable models.

result Prompt tuning provides stronger guarantees than head tuning.

pretrained models downstream tasks latent variable models HMM long-term memory

2021-06-17abs ↗pdf ↗

Prompt-tuning latent diffusion models for inverse problemscs.LG

New method improves image generation for inverse problems using text prompts.

problem Suboptimal performance of existing latent diffusion models for inverse problems.

method Prompt tuning and latent variable projection to optimize text embeddings.

result P2L method outperforms existing methods on various inverse problem tasks.

latent diffusion models inverse problems image generation text prompts super-resolution

2023-10-02abs ↗pdf ↗

Understanding Prompt Tuning and In-Context Learning via Meta-Learningcs.LG

This paper explains how to optimize prompts for model adaptation.

problem Understanding and optimizing prompt tuning for model adaptation.

method Bayesian view and meta-learning to explain prompt optimization.

result Optimal prompting can be studied formally as conditioning Bayesian predictors.

prompt tuning in-context learning meta-learning Bayesian predictors weight tuning

2025-05-22abs ↗pdf ↗

LLM Prompt Duel Optimizer: Efficient Label-Free Prompt Optimizationcs.CL

PDO optimizes LLM prompts without labels, improving performance.

problem Optimizing prompts for LLMs without access to labeled data.

method Pairwise preference feedback, dueling bandits, Thompson Sampling, mutation.

result PDO identifies stronger prompts than label-free methods.

LLM optimization prompt tuning label-free pairwise feedback bandit algorithms

2025-10-14abs ↗pdf ↗

Winter Soldier: Backdooring Language Models at Pre-Training with Indirect Data Poisoningcs.CR

Researchers show how to secretly train models with hidden data, detect usage with high confidence.

problem Protecting training data from traceability in large language models.

method Gradient-based optimization to learn secret sequences absent from training data.

result Secret sequences can be learned by models without performance degradation, detectable with high confidence.

language models indirect data poisoning membership inference hidden canaries pre-training

2025-06-17abs ↗pdf ↗

Generalization Analysis of Transformers in Distribution Regressionstat.ML

Transformers learn functionals from distributions without losing information.

problem Lack of rigorous mathematical theory supporting Transformer performance.

method Proposed a Transformer learning framework, attention operator, and distribution regression.

result Transformers can compress distributions into function representations without loss of information.

Transformers Distribution regression Attention operator Generalization bound Function learning

2026-06-28abs ↗pdf ↗

Market-Derived Financial Sentiment Analysis: Context-Aware Language Models for Crypto Forecastingcs.CE

Improved crypto market forecasting using historical price reactions to tweets.

problem Challenges in inferring market impact from human sentiment labels.

method Market-derived labeling approach to assign tweet sentiment labels based on historical price trends. Fine-tuned language model with context-aware prompt-tuning.

result 89.6% accuracy on Bitcoin news events, outperforming traditional fusion models.

Financial Sentiment Analysis Crypto Forecasting Language Models Market Dynamics Sentiment-Based Predictions

2025-02-17abs ↗pdf ↗