Two methods for model adaptation compared; fine-tuning outperforms Best-of-N in realizable settings.
problem Comparing methods for adapting large language models to new tasks.
method Supervised fine-tuning vs. Best-of-N approach.
result Supervised fine-tuning outperforms Best-of-N in realizable settings.
Theoretical framework explains why few epochs are enough for LLM fine-tuning.
problem Understanding why few epochs are sufficient for LLM fine-tuning.
method Combining early stopping theory with attention-based Neural Tangent Kernel (NTK) for LLMs.
result Formalizes convergence rate of attention-based fine-tuning with respect to sample size.
Study on semiconcavity of solutions to gradient obstacle problems on compact manifolds.
problem Gradient obstacle problems on compact Riemannian manifolds.
method Uniform semiconcavity estimates and fine convergence results for solutions and free boundaries.
result The elastic and λ-elastic sets of solutions converge to the cut locus and λ-cut locus of the manifold. EVA adapts LoRA for faster, more efficient fine-tuning.
problem Fast and efficient fine-tuning of large models for specific tasks.
method EVA uses directions capturing most activation variance for initialization, maximizing gradient signal and reducing parameters.
result EVA achieves faster convergence and higher average scores across tasks, reducing parameters.
Zeroth-order methods favor flat minima in machine learning.
problem Finding solutions with small Hessian trace in optimization.
method Zeroth-order optimization with two-point estimator.
result Zeroth-order optimization converges to flat minima.
LORENZA improves LLM fine-tuning efficiency and generalization.
problem Improving robustness and generalization of LLMs under hardware constraints.
method AdaZo-SAM and LORENZA, combining Adam and SAM with zeroth-order estimation and randomized SVD.
result LORENZA achieves better generalization and reduced memory consumption compared to existing methods.
BEMA reduces bias in EMA, leading to faster convergence and better performance.
problem Stochasticity in language model fine-tuning destabilizes training.
method Bias-Corrected Exponential Moving Average (BEMA) augmentation of EMA.
result BEMA leads to significantly improved convergence rates and final performance.
LoRA fine-tuning explained with gradient dynamics for low-rank perturbations.
problem Understanding why gradient descent converges to useful low-rank perturbations in LoRA fine-tuning.
method Generalized student-teacher setting with i.i.d. samples and online gradient descent.
result Gradient descent converges to the teacher model in dkO(1) iterations under certain conditions. Proposes a new RL method to fine-tune flow-based models with arbitrary rewards.
problem Challenges in fine-tuning continuous flow-based generative models with arbitrary reward functions.
method Online Reward-Weighted Conditional Flow Matching with Wasserstein-2 Regularization (ORW-CFM-W2)
result Achieves optimal policy convergence with controllable trade-offs between reward maximization and diversity preservation.
Multirate training speeds up neural network fine-tuning.
problem Efficiently fine-tuning deep neural networks.
method Partitioning neural network parameters into fast and slow parts, updating slowly over longer intervals.
result Significant computational speed-up for transfer learning tasks.
We analyse the fine convergence properties of one parameter families of hyperbolic metrics, on a fixed underlying surface, that move always in a horizontal direction, i.e. orthogonal to the action of diffeomorphisms.
Study on prescribing positive curvature with conical singularities on a sphere.
problem Prescribing positive curvature with conical singularities on a sphere.
method Fine analysis of bubble trees and an area identity in the convergence process.
result Criterion for nonexistence in an open region of the prescribing data.
FAST improves fast and stable task adaptation in DNNs.
problem Catastrophic forgetting in fine-tuned pretrained models.
method Introducing FAST, an easy-to-implement fine-tuning algorithm.
result FAST learns target tasks faster and retains source knowledge longer.
LEAK learns from mistakes to improve point cloud segmentation.
problem Improving point cloud semantic segmentation performance.
method Coarse-to-fine clustering, class-conditional prototypical feature alignment, fairness weighting.
result State-of-the-art performances on different architectures, datasets, and tasks.
Adaptive gradient methods are workhorses in deep learning. However, the convergence guarantees of adaptive gradient methods for nonconvex optimization have not been thoroughly studied. In this paper, we provide a fine-grained convergence analysis for a general class of adaptive gradient methods including AMSGrad, RMSPr…
Transformers learn unseen tasks via prompts without fine-tuning.
problem Understanding how transformers learn unseen tasks without additional fine-tuning.
method Structured data model, gradient descent, two-phase convergence analysis.
result Transformers can learn linear function classes via in-context learning.
Recent state-of-the-art language models utilize a two-phase training procedure comprised of (i) unsupervised pre-training on unlabeled text, and (ii) fine-tuning for a specific supervised task. More recently, many studies have been focused on trying to improve these models by enhancing the pre-training phase, either vi…
If f is a smooth function on a Hodge manifold, we construct a canonical sequence of real algebraic functions that converge to f in the smooth topology. The definition of of the approximants is inspired by Berezin-Toeplitz quantization. The proof follows quickly from known results of Fine, Liu and Ma.
LoRA-One uses one-step full gradient to align adapters for efficient large model fine-tuning.
problem Fine-tuning large language models efficiently and accurately.
method Properly initializing LoRA adapters using the one-step full gradient and incorporating preconditioners.
result LoRA-One achieves significant empirical improvements over existing methods.
Pruning + fine-tuning reduces model complexity and generalizes well for matrix sensing.
problem Reducing model complexity for matrix sensing problems.
method Group Lasso regularization and greedy pruning.
result Pruning results in a solution with minimum columns close to the ground truth.
Adaptive gradient methods like AdaGrad are widely used in optimizing neural networks. Yet, existing convergence guarantees for adaptive gradient methods require either convexity or smoothness, and, in the smooth setting, only guarantee convergence to a stationary point. We propose an adaptive gradient method and show t…
Research reveals deep networks often learn low-rank structures, leading to more efficient training and fine-tuning.
problem Efficient training and deployment of large-scale deep learning models.
method Complementary theoretical perspectives on low-rank structures during training and convergence, and practical applications of LoRA and masked training.
result Understanding and exploiting low-rank structures can improve efficiency and effectiveness of training and fine-tuning.
This paper establishes a theoretical foundation for super-models via domain adaptation.
problem Reducing computational and data costs in AI for small and medium-sized enterprises.
method Two-stage diffusion process modeling, including pre-training and fine-tuning stages, using the Uhlenbeck-Ornstein process.
result The generalization error of the fine-tuning stage is dominant in domain adaptation.
Drago optimizes DRO problems with faster convergence.
problem Distributionally robust optimization with closed, convex uncertainty sets.
method Primal-dual coupled variance reduction algorithm with cyclic and randomized updates.
result Achieves state-of-the-art linear convergence rate on strongly convex-strongly concave problems.
New framework improves LLM performance by avoiding forgetting during sequential training stages.
problem Forgetting during sequential training stages of LLMs.
method Proposes a joint post-training framework with theoretical convergence guarantees.
result Empirically outperforms sequential post-training framework by up to 23%.
Proposes a method to stabilize Black Box Variational Inference using the James-Stein estimator.
problem Stability issues and fine-tuning required in basic Black Box Variational Inference.
method Reframe stochastic gradient ascent as multivariate estimation problem using James-Stein estimator.
result Provides a simpler method with consistent performance in terms of model fit and convergence time.
Selecting the most appropriate data examples to present a deep neural network (DNN) at different stages of training is an unsolved challenge. Though practitioners typically ignore this problem, a non-trivial data scheduling method may result in a significant improvement in both convergence and generalization performanc…
Adaptive step-size method improves compressed SGD performance in machine learning.
problem Communication bottleneck in distributed and decentralized optimization.
method Developed an adaptive step-size method for compressed SGD.
result Order-optimal convergence rates for various objective functions.
Adaptive-SGD method optimizes machine learning training with dynamic batch and step sizes.
problem Optimizing machine learning training with adaptive batch and step sizes.
method Adaptive-SGD method that dynamically adjusts batch size and step size based on local curvature and probability of descent directions.
result Adaptive-SGD achieves global linear convergence on self-concordant functions and compares favorably to fine-tuned methods.
Contextual bandit learning is an increasingly popular approach to optimizing recommender systems via user feedback, but can be slow to converge in practice due to the need for exploring a large feature space. In this paper, we propose a coarse-to-fine hierarchical approach for encoding prior knowledge that drastically …
New SMC samplers improve stochastic optimisation efficiency.
problem Optimizing functions with intractable gradients in machine learning and statistics.
method Sequential Monte Carlo (SMC) samplers for stochastic optimisation.
result Significant computational gains achieved with SMC approximations.
New measure FTC quantifies how much a ReLU network can fine-tune.
problem Analyzing memorization capacity in fine-tuned neural networks.
method Defined Fine-Tuning Capacity (FTC) for additive fine-tuning of ReLU networks.
result Upper and lower bounds on FTC for 2 and 3-layer ReLU networks.
Alternative neural network training using monotone variational inequality.
problem Training neural networks efficiently and with guarantees.
method Using monotone variational inequality to solve non-convex problems efficiently.
result Our approach leads to fast convergence and competitive performance compared to traditional methods.
Paper resolves bias in ALFT training using generalized alignment games.
problem Systematic bias in estimating logarithmic rewards from small batches.
method Generalized Distributional Alignment Games, U-statistics, minimax polynomial estimators, Variance-Optimal Augmented Polynomial Optimization Program (AQP) Estimator.
result Proves optimal bias and accelerated convergence in ALFT training.
This paper re-evaluates hyperparameters for fine-tuning pre-trained models.
problem Current hyperparameter settings for fine-tuning are often ad-hoc and fixed.
method Empirical evaluation of learning rate, batch size, and momentum for fine-tuning.
result Optimal hyperparameters are not only dataset-dependent but also sensitive to domain similarity.
DEQs converge to optimal solutions with mild over-parameterization.
problem Training over-parameterized deep equilibrium models.
method Solves equilibrium point directly, uses gradient descent, and analyzes convergence via linear rate.
result Gradient descent converges to a globally optimal solution at a linear rate for quadratic loss.
A new method optimizes diffusion models with recursive likelihood ratios.
problem Efficiently aligning pre-trained diffusion models for specific applications.
method Recursive Likelihood Ratio (RLR) optimizer for Half-Order (HO) fine-tuning.
result The RLR method achieves unbiased and lower-variance gradients, improving model performance.
New method quantifies uncertainty in fine-tuned LLMs using LoRA ensembles.
problem Uncertainty in fine-tuned LLMs and how to trust their predictions.
method Posterior approximations using low-rank adaptation ensembles.
result Unexpected retention of acquired knowledge during fine-tuning in overfitting regime.
BERT fine-tuning is unstable due to optimization issues, not forgetting or dataset size.
problem Stability of fine-tuning BERT-based models across different random seeds.
method Analysis of BERT, RoBERTa, and ALBERT fine-tuned on GLUE datasets, identifying optimization difficulties as the cause of instability.
result Fine-tuning instability is due to optimization difficulties leading to vanishing gradients, not forgetting or dataset size.
Improved code translation by preserving structure with composed fine-tuning.
problem Improving code translation accuracy with unlabeled code outputs.
method Pre-trained denoiser to capture output structure, composed fine-tuning to fine-tune predictor.
result Composed fine-tuning significantly improves generalization over standard fine-tuning.
New findings on hyperbolicity of fine curve graphs and their subgraphs.
problem Investigating hyperbolicity of fine curve graphs and their subgraphs.
method Analyzing large subgraphs of fine curve graphs and computing distances in specific cases.
result Large subgraphs of fine curve graphs contain flats of every finite dimension, indicating they are not hyperbolic.
BERT, which stands for Bidirectional Encoder Representations from Transformers, is a recently introduced language representation model based upon the transfer learning paradigm. We extend its fine-tuning procedure to address one of its major limitations - applicability to inputs longer than a few hundred words, such as…
The paper introduces a Hessian-based method to improve generalization in fine-tuned deep neural networks.
problem Improving generalization in fine-tuned deep neural networks, especially in noisy conditions.
method PAC-Bayesian analysis to identify a Hessian-based distance measure, proving generalization bounds, and developing an algorithm with a generalization error guarantee.
result Hessian-based distance measure correlates well with observed generalization gaps and can match the scale of these gaps in practice.
Paper finds sparse representation of functions using inverse scale space flow.
problem Finding sparse representation of L2 functions. method Inverse scale space flow to minimize L2 loss. result Convergence to optimal solution in ideal and noisy cases.
Homotopy types of curve and arc complexes are studied.
problem Understanding the homotopy types of curve and arc complexes.
method Proving homotopy equivalence and contractibility of complexes.
result Fine curve complex is homotopy equivalent to curve complex, fine arc complex is contractible.
ESPO optimizes LLMs for complex tasks by balancing fine-grained updates and stability.
problem Gradient underutilization in sequence-level optimization.
method Entropy Grouping Importance Sampling and Entropy Adaptive Clipping.
result ESPO accelerates convergence and achieves state-of-the-art performance.
Fine-tuning LLMs improves capability but harms safety, study finds.
problem Balancing capability and safety in LLM fine-tuning.
method Theoretical framework and numerical experiments for two safety-aware fine-tuning strategies.
result Characterization of fundamental limits of safety-capability trade-off in LLM fine-tuning.
Optimizes sparse fine-tuning for privacy in neural networks.
problem Performance gap between DP-SGD and non-private fine-tuning.
method Optimization-based approach using private gradient information for selecting trainable weights.
result Our selection method leads to better prediction accuracy compared to existing approaches.