New measure FTC quantifies how much a ReLU network can fine-tune.
problem Analyzing memorization capacity in fine-tuned neural networks.
method Defined Fine-Tuning Capacity (FTC) for additive fine-tuning of ReLU networks.
result Upper and lower bounds on FTC for 2 and 3-layer ReLU networks.
New method enhances model fine-tuning with minimal data.
problem Improving model performance on new tasks with limited data.
method Introducing α-LoRA, a reparameterization method for fine-tuning. result Enhanced generalization ability of fine-tuned models.
New method reduces fine-tuning cost for reused models.
problem Repeating fine-tuning costs with outdated foundation models.
method Portable Reward Tuning (PRT) trains a reward model to maximize the same loss function as fine-tuning.
result PRT achieves comparable accuracy to inference-time tuning with less inference cost.
UBM transfers bias mitigation from upstream to downstream tasks efficiently.
problem Bias in fine-tuned language models across various tasks.
method Apply bias mitigation to an upstream model, then fine-tune a downstream model on this mitigated model.
result UBM effects transfer to new downstream tasks, creating less biased models.
AFTER technique improves NLP models by preventing overfitting to task-specific domains.
problem Standard fine-tuning degrades pretraining domain representations.
method Complements task-specific loss with adversarial objective.
result AFTER leads to improved performance on various NLP tasks.
Proposes efficient calibration for indoor localization models.
problem Calibration data scarcity in wireless indoor localization.
method Uses synthetic labels and prediction sets to fine-tune a predictor and estimate bias.
result Yields rigorous coverage guarantees for prediction sets.
Improved RL training for DMs reduces mode collapse and preserves diversity.
problem Mode collapse and training instability in RL fine-tuned diffusion models.
method Dynamic hierarchical RL training with sliding-window parameter regularisation.
result Models trained with HRF achieve better preservation of diversity in downstream tasks.
In this work we focus on fine-tuning a pre-trained BERT model and applying it to patent classification. When applied to large datasets of over two millions patents, our approach outperforms the state of the art by an approach using CNN with word embeddings. In addition, we focus on patent claims without other parts in …
New findings on representation changes in transfer learning.
problem Understanding representation changes in transfer learning.
method Analysis of pre-trained and fine-tuned representations in transfer learning.
result Representation changes mainly occur in the head, but early layers are also important for cross-domain adaptation.
Zero-shot contrastive loss improves text-guided image style transfer without extra training.
problem Stochastic nature of diffusion models leads to trade-offs between style transformation and content preservation.
method Proposes a zero-shot contrastive loss for diffusion models that doesn't require additional fine-tuning or auxiliary networks.
result Method outperforms existing methods while preserving content and requiring no additional training.
Transformer models improve arithmetic accuracy with number decomposition.
problem Transformer models struggle with arithmetic operations without decomposition.
method Fine-tuning models with a pipeline that decomposes numbers into units, tens, etc.
result Accuracy increased by 63% in five-digit addition tasks.
BRAID fine-tunes diffusion models to optimize reward models in offline scenarios.
problem Combining generative modeling and model-based optimization in offline scenarios.
method Conservative fine-tuning of diffusion models using RL to optimize reward models.
result BRAID outperforms existing methods in offline data, avoiding invalid designs.
Private learning can perform well in high dimensions, contrary to known results.
problem When does differentially private learning not suffer in high dimensions?
method Introduced a condition called restricted Lipschitz continuity to derive improved bounds for excess empirical and population risks.
result Gradients in private fine-tuning of large models are mostly controlled by a few principal components, similar to conditions for convex settings.
Behavior Transfer improves reinforcement learning by leveraging pre-trained policies.
problem Efficient transfer of knowledge in reinforcement learning.
method Behavior Transfer (BT) that uses pre-trained policies for exploration.
result BT combined with pre-training leads to better solutions than without pre-training.
SPIN converts weak LLMs to strong ones using self-play.
problem Growing strong LLMs without human-annotated data.
method Self-play fine-tuning (SPIN) starting from a supervised fine-tuned model.
result SPIN significantly improves LLM performance across benchmarks.
New method prevents forgetting in LLMs by dynamically identifying task-specific subspaces.
problem Catastrophic forgetting in continual learning of LLMs.
method Adaptive Singular Value Decomposition (SVD) for constrained full fine-tuning.
result Achieves state-of-the-art results in continual learning benchmarks.
Curriculum learning improves model training efficiency and performance.
problem Expensive training of reasoning models using human or synthetic data.
method Autocurriculum, where models use their own performance to decide training problems.
result Autocurriculum reduces training costs and improves model performance.
Multi-task learning shares information between related tasks, sometimes reducing the number of parameters required. State-of-the-art results across multiple natural language understanding tasks in the GLUE benchmark have previously used transfer from a single large task: unsupervised pre-training with BERT, where a sep…
Improved financial VA intent classification accuracy.
problem Determining user intents for unseen open intents.
method Supervised pre-training of intent representations using prefix-tuning and fine-tuning.
result 1.63% - 2.07% higher accuracy on banking77 benchmark.
Learning robot tasks or controllers using deep reinforcement learning has been proven effective in simulations. Learning in simulation has several advantages. For example, one can fully control the simulated environment, including halting motions while performing computations. Another advantage when robots are involved…
Bayesian principles improve neural additive models for better feature selection and uncertainty.
problem Lack of calibrated uncertainties and feature selection in neural additive models.
method Augmenting NAMs with Bayesian principles to provide credible intervals, feature selection, and interaction ranking.
result Improved performance on tabular datasets and real-world medical tasks.
New method quantifies uncertainty in fine-tuned LLMs using LoRA ensembles.
problem Uncertainty in fine-tuned LLMs and how to trust their predictions.
method Posterior approximations using low-rank adaptation ensembles.
result Unexpected retention of acquired knowledge during fine-tuning in overfitting regime.
BERT fine-tuning is unstable due to optimization issues, not forgetting or dataset size.
problem Stability of fine-tuning BERT-based models across different random seeds.
method Analysis of BERT, RoBERTa, and ALBERT fine-tuned on GLUE datasets, identifying optimization difficulties as the cause of instability.
result Fine-tuning instability is due to optimization difficulties leading to vanishing gradients, not forgetting or dataset size.
Fine-tuning from pre-trained ImageNet models has become the de-facto standard for various computer vision tasks. Current practices for fine-tuning typically involve selecting an ad-hoc choice of hyperparameters and keeping them fixed to values normally used for training from scratch. This paper re-examines several comm…
Improved code translation by preserving structure with composed fine-tuning.
problem Improving code translation accuracy with unlabeled code outputs.
method Pre-trained denoiser to capture output structure, composed fine-tuning to fine-tune predictor.
result Composed fine-tuning significantly improves generalization over standard fine-tuning.
New mechanism protects neural network weights from privacy attacks during self-supervised learning.
problem Privacy risks during fine-tuning stage of self-supervised learning.
method Proposes a novel differential privacy mechanism using additive logistic noise.
result Reduces membership inference attack accuracy to 50% while maintaining below 5% performance loss.
The paper introduces a Hessian-based method to improve generalization in fine-tuned deep neural networks.
problem Improving generalization in fine-tuned deep neural networks, especially in noisy conditions.
method PAC-Bayesian analysis to identify a Hessian-based distance measure, proving generalization bounds, and developing an algorithm with a generalization error guarantee.
result Hessian-based distance measure correlates well with observed generalization gaps and can match the scale of these gaps in practice.
SoftAD improves classification accuracy with less fine-tuning and fewer computational costs.
problem Improving classification accuracy with less fine-tuning and fewer computational costs.
method SoftAD is a softened, pointwise mechanism that downweights borderline points and limits the effects of outliers.
result SoftAD achieves classification accuracy competitive with flooding and SAM, with a smaller loss generalization gap and model norm.
Fine-tuning LLMs improves capability but harms safety, study finds.
problem Balancing capability and safety in LLM fine-tuning.
method Theoretical framework and numerical experiments for two safety-aware fine-tuning strategies.
result Characterization of fundamental limits of safety-capability trade-off in LLM fine-tuning.
Optimizes sparse fine-tuning for privacy in neural networks.
problem Performance gap between DP-SGD and non-private fine-tuning.
method Optimization-based approach using private gradient information for selecting trainable weights.
result Our selection method leads to better prediction accuracy compared to existing approaches.
This paper examines data transfer methods to improve off-the-shelf Transformer models for retrosynthesis.
problem Improving prediction models for retrosynthesis using small datasets.
method Systematic examination of data transfer approaches on end-to-end generative models.
result Pre-training plus fine-tuning boosts the accuracy of a Transformer baseline, achieving state-of-the-art results.
The paper proposes DEA to make graph neural networks fairer in link prediction.
problem Graph neural networks can unfairly prioritize certain social groups in link prediction.
method Drop Edges and Adapt (DEA) fine-tuning strategy with covariance constraints.
result DEA improves fairness and accuracy in link prediction tasks.
Fine-tuning harms in-context learning, but restricting updates to the value matrix improves zero-shot performance.
problem Fine-tuning harms in-context learning, reducing zero-shot performance on unseen tasks.
method Theoretical analysis of linear attention models, identifying conditions for degraded few-shot performance.
result Restricting updates to the value matrix improves zero-shot performance while preserving in-context learning.
Measures consistency of tabular LLM predictions under fine-tuning multiplicity.
problem Conflicting predictions from fine-tuned tabular LLMs.
method Local stability measure in embedding space.
result Probabilistic guarantees on prediction consistency under multiplicity.
Self-play fine-tuning improves diffusion models for text-to-image generation.
problem Plateauing performance of diffusion models after data saturation.
method Self-play fine-tuning (SPIN-Diffusion) using competition among model versions.
result Significantly improved model performance and human preference alignment.
The paper develops a theory linking pretraining and fine-tuning in neural networks.
problem Understanding how initialization choices impact feature learning and generalization in neural networks.
method Analytical theory of diagonal linear networks, deriving generalization error as a function of initialization parameters and task statistics.
result Different initialization choices place networks into four fine-tuning regimes with varying abilities to support feature learning and generalization.
Compact models match or exceed GPT's performance in financial news sentiment analysis.
problem Improving financial sentiment analysis models without large computational costs.
method Fine-tuning non-generative, small-sized models (FinBERT, FinDRoBERTa) on a novel market score database.
result Fine-tuned models outperform GPT-3.5 and GPT-4 in zero-shot learning for financial news sentiment analysis.
LoRA and privacy: Random projections help but not always.
problem Ensuring differential privacy in LoRA fine-tuning.
method Wishart projection mechanism and noisy variants.
result LoRA is not inherently private, but low-rank fine-tuning can be more private.
Unsupervised pre-training improves model generalization, but lacks theoretical understanding.
problem Lack of theoretical understanding of unsupervised pre-training's impact on model generalization.
method Introduces a novel theoretical framework to analyze and enhance generalization.
result Enhances understanding of unsupervised pre-training and fine-tuning, proposing a new regularization method.
LP-FT improves personalized model training in FL by balancing generalization and personalization.
problem Federated Learning struggles with balancing global generalization and local personalization due to non-identical data distributions.
method Adapting Linear Probing followed by full Fine-Tuning (LP-FT) to the FL setting.
result LP-FT outperforms standard fine-tuning in balancing personalization and generalization across various datasets and PFT variants.
This paper describes FBK's submission to the end-to-end English-German speech translation task at IWSLT 2018. Our system relies on a state-of-the-art model based on LSTMs and CNNs, where the CNNs are used to reduce the temporal dimension of the audio input, which is in general much higher than machine translation input…
Improved fine-tuning with regularization and robustness for noisy labels.
problem Fine-tuning pre-trained models on small datasets can lead to overfitting and memorization.
method PAC-Bayes generalization bound analysis, layer-wise regularization, self-label-correction, label-reweighting.
result Improves performance by 1.76% on average for image classification tasks and 0.75% for few-shot classification.
Simplified trust region method reduces representation change during fine-tuning.
problem Stability and representational collapse in fine-tuning pre-trained models.
method Replaces adversarial objectives with parametric noise in trust region theory.
result Matches or exceeds previous trust region methods in performance and speed.
Adjoint Matching improves flow and diffusion models with reward fine-tuning.
problem Improving generative models with reward fine-tuning.
method Casting reward fine-tuning as stochastic optimal control (SOC) and enforcing a specific noise schedule.
result Adjoint Matching outperforms existing SOC algorithms.
Fine-tunes LLMs to correct bias in predictions.
problem LLMs exhibit bias in predictions from data.
method Supervised fine-tuning with Low-Rank Adaptation (LoRA).
result Fine-tuning corrects bias in both controlled and real-world settings.
Transfer learning, which allows a source task to affect the inductive bias of the target task, is widely used in computer vision. The typical way of conducting transfer learning with deep neural networks is to fine-tune a model pre-trained on the source task using data from the target task. In this paper, we propose an…
Improved sampling efficiency for molecular systems using path gradients after Flow Matching.
problem Improving sampling efficiency for complex molecular systems.
method Hybrid approach combining Flow Matching and path gradients.
result Up to a threefold increase in sampling efficiency for molecular systems.
Fine-tuning large language models requires minimal data, making them efficient.
problem Achieving state-of-the-art performance with large language models.
method Using BERT as an example, fine-tuning only the most critical layers of the pre-trained model.
result Fine-tuned models are close in parameter space to the pre-trained model, with many good solutions found in sparsified versions.