InfoPrompt improves soft prompt tuning by maximizing mutual information, leading to better performance.
problem High sensitivity of prompt tuning to initial conditions and insufficient task-relevant information.
method Develops an information-theoretic framework to maximize mutual information between prompts and model parameters, using novel loss functions.
result InfoPrompt accelerates convergence and outperforms traditional methods.
Single-head transformers with a single self-attention layer can approximate any sequence-to-sequence function and are efficient under certain conditions.
problem Statistical and computational limits of prompt tuning for transformer-based models.
method Investigation of single-head transformers with a single self-attention layer, proving universality and efficiency under SETH.
result Existence of almost-linear time prompt tuning inference algorithms under certain conditions.
Improved text generation with discourse-aware soft prompts.
problem Efficient fine-tuning methods don't generalize across all generation tasks.
method Hierarchical blocking and attention sparsity on prefix parameters.
result Structured design of prefix parameters yields more coherent generations.
Study on how attention in prompt-tuning affects large language models.
problem Limited theoretical understanding of prompt-tuning and attention in LLMs.
method Exploration of prompt-tuning for one-layer attention architectures, contextual mixture-models, and self-contained prompt-attention model.
result Softmax-prompt-attention is more expressive than self-attention and linear-prompt-attention under contextual data model.
This paper explains how to optimize prompts for model adaptation.
problem Understanding and optimizing prompt tuning for model adaptation.
method Bayesian view and meta-learning to explain prompt optimization.
result Optimal prompting can be studied formally as conditioning Bayesian predictors.
New framework improves interpretability of trainable prompts.
problem Improving task-specific LLM performance with soft prompts remains a black-box method.
method Developed a theoretical framework for evaluating interpretability of trainable prompts, inspired new objective functions.
result Found a fundamental trade-off between interpretability and task performance in trainable prompts.
ADAPT improves robustness of Vision Transformers without full model fine-tuning.
problem Vulnerability of Vision Transformers to adversarial attacks.
method Parameter-efficient prompt tuning with ADAPT framework for adaptive adversarial training.
result ADAPT achieves robust accuracy of ~40% w.r.t. SOTA methods using only ~1% of the parameters.
Emergent misalignment is influenced by training dynamics, model priors, and data.
problem Emergent misalignment in models
method Exploring training dynamics, model priors, and data
result Activation deltas before and after narrow fine-tuning correlate with their similarities when measured with the last prompt-token activations.
POUF fine-tunes large models without labeled data.
problem Lack of labeled data for fine-tuning large pre-trained models.
method Prompt-oriented unsupervised fine-tuning.
result Consistent improvements across various tasks.
The paper studies estimation rates for MoE models with a new prompt.
problem Estimating parameters in a softmax-contaminated MoE model.
method Analytic notion of distinguishability, minimax optimal estimation rates.
result Estimation rates are minimax optimal under distinguishability, but slower otherwise.
PDO optimizes LLM prompts without labels, improving performance.
problem Optimizing prompts for LLMs without access to labeled data.
method Pairwise preference feedback, dueling bandits, Thompson Sampling, mutation.
result PDO identifies stronger prompts than label-free methods.
New method improves image generation for inverse problems using text prompts.
problem Suboptimal performance of existing latent diffusion models for inverse problems.
method Prompt tuning and latent variable projection to optimize text embeddings.
result P2L method outperforms existing methods on various inverse problem tasks.
Soft Actor-Critic is a state-of-the-art reinforcement learning algorithm for continuous action settings that is not applicable to discrete action settings. Many important settings involve discrete actions, however, and so here we derive an alternative version of the Soft Actor-Critic algorithm that is applicable to dis…
Transformers can emulate various algorithms by prompting, proving universality.
problem How to emulate algorithms using fixed-weight Transformers.
method Two modes of in-context algorithm emulation: task-specific and prompt-programmable. Constructing prompts that encode algorithm parameters into token representations.
result Fixed-weight Transformers can emulate a broad class of algorithms via prompts.
FinRLlama wins FinRL Challenge 2024 by fine-tuning LLMs with market data.
problem Lack of contextual alignment for financial market applications in traditional LLMs.
method Fine-tuning LLaMA-3.2-3B-Instruct model with custom RLMF prompt design integrating historical data and reward feedback.
result RLMF-tuned FinRLlama framework outperforms baseline methods in signal consistency and trading outcomes.
Analysis of pretrained models' effectiveness in downstream tasks.
problem Understanding why pretrained models perform well in NLP tasks.
method Analyzed head and prompt tuning approaches using latent variable models.
result Prompt tuning provides stronger guarantees than head tuning.
ZegOT uses optimal transport to zero-shot segment images with text prompts.
problem Zero-shot semantic segmentation with limited image-text alignment knowledge.
method ZegOT uses optimal transport to match multiple text prompts with frozen image embeddings.
result ZegOT achieves state-of-the-art performance in zero-shot semantic segmentation.
Meta-SAC automatically tunes SAC's entropy temperature for better exploration.
problem Exploration-exploitation dilemma in reinforcement learning.
method Meta-SAC uses metagradient and a novel meta objective to automatically adjust SAC's entropy temperature.
result Meta-SAC outperforms SAC-v2 by 10% on the humanoid-v2 task.
New metric captures individual neuron tuning across neural networks.
problem Need a metric that respects individual neuron tuning across different neural networks.
method Derived a 'soft' permutation-based metric using optimal transport theory.
result Metric avoids counter-intuitive outcomes and captures geometric insights.
Generative Adapter adapts LMs with a single forward pass, reducing inference overhead.
problem Efficient adaptation of large language models for new contexts.
method Generative Adapter directly maps new contexts to low-rank LM adapters via self-supervised learning.
result Significant reduction in inference overhead with no need for fine-tuning.
Study analyzes convergence of parameter estimation in contaminated mixture of experts.
problem Challenges in learning from prompts in large-scale models.
method Convergence analysis, distinguishability condition, partial differential equations.
result Comprehensive convergence rates and minimax lower bounds for parameter estimation.
eva method improves RL models by adaptively creating prompts, boosting performance.
problem Fixed prompt distribution limits scalability of RL models post-training.
method Evolving Alignment via Asymmetric Self-Play (eva) approach.
result Significant performance boost on benchmarks, e.g. 51.6% to 60.1% win-rate.
New method optimizes diffusion models without fine-tuning, integrating soft value functions.
problem Optimizing natural design spaces of images, molecules, DNA, RNA, and protein sequences.
method Iterative sampling method integrating soft value functions into diffusion model inference.
result Directly utilizes non-differentiable features/reward feedback, applies to discrete diffusion models.
EQD model improves domain-specific QA by 0.6% to 10.5%.
problem Challenges in domain-specific quantitative reasoning for LLMs.
method Two-step fine-tuning framework guided by a reward function.
result EQD outperforms state-of-the-art models and prompting strategies.
We present Multitask Soft Option Learning(MSOL), a hierarchical multitask framework based on Planning as Inference. MSOL extends the concept of options, using separate variational posteriors for each task, regularized by a shared prior. This ''soft'' version of options avoids several instabilities during training in a …
We study the distribution of hard-, soft-, and adaptive soft-thresholding estimators within a linear regression model where the number of parameters k can depend on sample size n and may diverge with n. In addition to the case of known error-variance, we define and study versions of the estimators when the error-varian…
ICON learns differential equation operators from prompts, reducing retraining and improving few-shot learning.
problem Training neural networks to solve differential equations without retraining for new problems.
method In-Context Operator Networks (ICON) that learns operators from prompted data and applies them to new problems.
result ICON can generalize to new operators beyond the training distribution and requires only a few demos.
Curriculum learning improves model training efficiency and performance.
problem Expensive training of reasoning models using human or synthetic data.
method Autocurriculum, where models use their own performance to decide training problems.
result Autocurriculum reduces training costs and improves model performance.
Enhances LLMs for predicting stock movements by considering news dissemination and context.
problem Lack of consideration for news dissemination and insufficient contextual data in LLMs for stock price prediction.
method Clusters news for reach assessment, enriches prompts with specific data and instructions, fine-tunes an LLM using the dataset.
result Improves prediction accuracy by 8% compared to existing methods.
Fine-tunes LLMs to correct bias in predictions.
problem LLMs exhibit bias in predictions from data.
method Supervised fine-tuning with Low-Rank Adaptation (LoRA).
result Fine-tuning corrects bias in both controlled and real-world settings.
AutoML-GPT uses GPT to automate AI model training.
problem Manual model selection and tuning requires significant human effort.
method Develops task-oriented prompts and utilizes LLMs for automated training.
result Achieves remarkable results in various AI tasks.
Fine-tuning improves information conveyance in language models by reorganizing uncertainty into more informative sequences.
problem Uncertainty reduction in large language models through fine-tuning is not fully understood, especially regarding output length.
method Proposed Canopy Entropy (CE⋆) to measure uncertainty in both output length and sequence, capturing total Shannon entropy. result Fine-tuned models exhibit stronger positive correlation between entropy rate and semantic diversity, indicating more informative and semantically meaningful generations.
Bayesian approach quantifies uncertainty in LLM-based systems.
problem Uncertainty quantification in LLM-based systems, especially for high-stakes applications.
method Interpreting prompts as parameters in a Bayesian model, using MHLP for inference.
result Improvements in predictive accuracy and uncertainty quantification on various benchmarks.
This work shows fine-tuning improves LLM uncertainty estimates efficiently.
problem Ensuring high-stakes predictions from large language models.
method Fine-tuning on a small dataset of correct and incorrect answers.
result Fine-tuned LLMs produce good generalization and tractable uncertainty estimates.
This paper improves combine harvester performance using ANN-PSO hybrid model.
problem Improving performance of combine harvesters to minimize waste and reduce maintenance.
method Proposes a hybrid machine learning model combining artificial neural networks and particle swarm optimization.
result Demonstrates higher accuracy and stability in predicting optimal performance of combine harvesters.
We propose in this paper a differentiable learning loss between time series, building upon the celebrated dynamic time warping (DTW) discrepancy. Unlike the Euclidean distance, DTW can compare time series of variable size and is robust to shifts or dilatations across the time dimension. To compute DTW, one typically so…
Voice activity detection (VAD), which classifies frames as speech or non-speech, is an important module in many speech applications including speaker verification. In this paper, we propose a novel method, called self-adaptive soft VAD, to incorporate a deep neural network (DNN)-based VAD into a deep speaker embedding …
Data driven soft sensor design has recently gained immense popularity, due to advances in sensory devices, and a growing interest in data mining. While partial least squares (PLS) is traditionally used in the process literature for designing soft sensors, the statistical literature has focused on sparse learners, such …
Transfer learning improves dynamic model transfer for soft robotic hands.
problem Improving dynamic model transfer for underactuated soft robotic hands.
method Derive an upper bound on the Lyapunov exponent and use it to improve transfer.
result Significant improvement over traditional fine-tuning methods.
A new type of distributional regression tree uses soft split rules for better predictive performance.
problem Estimating complete conditional distributions in regression.
method Distributional adaptive soft regression trees using multivariate soft split rules.
result The method outperforms various benchmark methods, especially in complex non-linear interactions.
Fine-tuning harms in-context learning, but restricting updates to the value matrix improves zero-shot performance.
problem Fine-tuning harms in-context learning, reducing zero-shot performance on unseen tasks.
method Theoretical analysis of linear attention models, identifying conditions for degraded few-shot performance.
result Restricting updates to the value matrix improves zero-shot performance while preserving in-context learning.
SoftAD improves classification accuracy with less fine-tuning and fewer computational costs.
problem Improving classification accuracy with less fine-tuning and fewer computational costs.
method SoftAD is a softened, pointwise mechanism that downweights borderline points and limits the effects of outliers.
result SoftAD achieves classification accuracy competitive with flooding and SAM, with a smaller loss generalization gap and model norm.
This paper analyzes financial sentiment using LLMs and FinBERT, improving accuracy with few-shot examples.
problem Financial sentiment analysis for market evaluation.
method Application of large language models and FinBERT, with focus on prompt engineering and few-shot learning.
result GPT-4o achieves similar sentiment classification accuracy to FinBERT with fewer examples.
Self-play fine-tuning improves diffusion models for text-to-image generation.
problem Plateauing performance of diffusion models after data saturation.
method Self-play fine-tuning (SPIN-Diffusion) using competition among model versions.
result Significantly improved model performance and human preference alignment.
EASE optimizes exemplar selection for ICL in LLMs efficiently.
problem Efficiently selecting exemplars for in-context learning (ICL) in large language models (LLMs).
method EASE uses neural bandit algorithms to optimize exemplar sets, considering both exemplar ordering and instruction.
result EASE outperforms existing methods in finding optimal exemplar sets for all test queries.
Paper analyzes adaptive ISTA with MAD for LASSO problem.
problem Finding solutions to LASSO problems without tuning λ. method Adaptive ISTA with median absolute deviation (MAD) for estimating noise level.
result Local linear convergence and global convergence of the algorithm.
MallowsPO enhances LLM fine-tuning with a dispersion index of human preferences.
problem Lack of diversity in human preferences in DPO.
method Developed a dispersion index based on Mallows' theory to characterize preference diversity.
result Demonstrated improved performance in various tasks using the dispersion index.
LLM4Causal democratizes causal reasoning via fine-tuned LLMs.
problem Limited capability of LLMs in causal inference and interpretation.
method Fine-tuning an open-source LLM for causal tasks, proposing datasets for instruction tuning.
result LLM4Causal delivers end-to-end solutions for causal problems and interprets results easily.