Parabolic mapping class acts on curve graphs of infinite type surfaces.
problem Understanding parabolic isometries on curve graphs of infinite type surfaces.
method Fine curve graph tools to prove existence of parabolic isometries.
result Existence of parabolic isometries on graphs of curves of infinite type surfaces.
The paper shows how to use fine shape to understand infinite-dimensional spaces.
problem Understanding infinite-dimensional metrizable spaces and their homology theories.
method Obtained results indicating fine shape is tractable and can be used for Polish spaces.
result Every Polish space is fine shape equivalent to the limit of an inverse sequence of simplicial maps.
Proves uniqueness of embedding complex manifold into infinite-dimensional space.
problem Balanced embedding of non-compact complex manifold into infinite-dimensional projective space.
method Fine estimates of asymptotics of a balanced embedding.
result Uniqueness of embedding proven.
New surfaces show horocyclic flow isn't always minimal.
problem Complex dynamics on infinite fineness surfaces.
method Construction of infinite hyperbolic surfaces.
result Horocyclic flow is not minimal on infinite fineness surfaces.
A new method optimizes diffusion models for fine-tuning tasks efficiently.
problem Optimizing diffusion models for downstream tasks using nested bilevel structures.
method Formalizes the challenge as a generative bilevel optimization problem and introduces a first-order bilevel framework.
result Our method outperforms existing fine-tuning and hyperparameter search baselines.
Efficient algorithms learn from coarse labels instead of fine grained ones.
problem Learning from coarse labels when fine labels are unavailable.
method Formalized coarse label settings, used a reduction for SQs, and provided efficient algorithms.
result Any problem learnable from fine labels can be learned efficiently from coarse labels.
Generates infinite-depth hierarchical clusters from few examples.
problem Inadequate finite-sample clustering methods for fine-scale hierarchical structures.
method Classification fields generated by a local refinement rule, approximated by predictors.
result Learned predictors can approximate infinite-depth hierarchical structures.
Stokes-Dirac structures are infinite-dimensional Dirac structures defined in terms of differential forms on a smooth manifold with boundary. These Dirac structures lay down a geometric framework for the formulation of Hamiltonian systems with a nonzero boundary energy flow. Simplicial triangulation of the underlaying m…
DEQs converge to optimal solutions with mild over-parameterization.
problem Training over-parameterized deep equilibrium models.
method Solves equilibrium point directly, uses gradient descent, and analyzes convergence via linear rate.
result Gradient descent converges to a globally optimal solution at a linear rate for quadratic loss.
The study analyzes transfer learning in infinite-width neural networks, improving generalization on target tasks.
problem Improving generalization in neural networks when using pretraining on a source task.
method Developed a theory under gradient flow for infinitely wide networks, analyzing fine-tuning and joint pretraining.
result Summary statistics of randomly initialized networks after pretraining are adaptive kernels that depend on both source and target data.
Let (X,ω) be a symplectic rational 4 manifold. We study the space of tamed almost complex structures Jω using a fine decomposition via smooth rational curves and a relative version of the infinite-dimensional Alexander duality. This decomposition provides new understandings of both the variation and stab…
We extend diffusion models to function spaces and introduce a new method for sampling from posterior distributions.
problem Sampling from posterior distributions in infinite-dimensional function spaces using diffusion models.
method Infinite-dimensional extension of Doob's h-transform, Supervised Guidance Training for efficient sampling. result We prove that diffusion models can be conditioned to sample from posterior distributions and introduce a simulation-free score matching objective.
Diffusion models generate private synthetic images with high quality.
problem Generating private synthetic images from sensitive datasets.
method Differentially private fine-tuning of pre-trained diffusion models.
result Improved synthetic data quality and accuracy compared to non-private models.
NN-GPR improves climate model predictions by preserving fine-scale spatial information.
problem Dilution of fine-scale spatial information and bias in model averaging.
method Gaussian process regression with an infinitely wide deep neural network.
result NN-GPR produces more accurate and detailed climate projections.
New principle reveals how neural networks learn complex interactions.
problem Understanding neural networks' success and complexity.
method Infinite-width networks, focusing on frequency and space.
result Fine-grained eigenstructure improves network learnability.
New measure FTC quantifies how much a ReLU network can fine-tune.
problem Analyzing memorization capacity in fine-tuned neural networks.
method Defined Fine-Tuning Capacity (FTC) for additive fine-tuning of ReLU networks.
result Upper and lower bounds on FTC for 2 and 3-layer ReLU networks.
Identifies most probable flows for Kunita SDEs in fluid dynamics.
problem Modeling stochastic processes with Eulerian noise and deterministic drifts.
method Equipping the domain with a Riemannian metric from the noise, solving the resulting PDEs.
result Most probable flows differ from deterministic flows, especially under noise.
This paper re-evaluates hyperparameters for fine-tuning pre-trained models.
problem Current hyperparameter settings for fine-tuning are often ad-hoc and fixed.
method Empirical evaluation of learning rate, batch size, and momentum for fine-tuning.
result Optimal hyperparameters are not only dataset-dependent but also sensitive to domain similarity.
New method quantifies uncertainty in fine-tuned LLMs using LoRA ensembles.
problem Uncertainty in fine-tuned LLMs and how to trust their predictions.
method Posterior approximations using low-rank adaptation ensembles.
result Unexpected retention of acquired knowledge during fine-tuning in overfitting regime.
BERT fine-tuning is unstable due to optimization issues, not forgetting or dataset size.
problem Stability of fine-tuning BERT-based models across different random seeds.
method Analysis of BERT, RoBERTa, and ALBERT fine-tuned on GLUE datasets, identifying optimization difficulties as the cause of instability.
result Fine-tuning instability is due to optimization difficulties leading to vanishing gradients, not forgetting or dataset size.
Improved code translation by preserving structure with composed fine-tuning.
problem Improving code translation accuracy with unlabeled code outputs.
method Pre-trained denoiser to capture output structure, composed fine-tuning to fine-tune predictor.
result Composed fine-tuning significantly improves generalization over standard fine-tuning.
New findings on hyperbolicity of fine curve graphs and their subgraphs.
problem Investigating hyperbolicity of fine curve graphs and their subgraphs.
method Analyzing large subgraphs of fine curve graphs and computing distances in specific cases.
result Large subgraphs of fine curve graphs contain flats of every finite dimension, indicating they are not hyperbolic.
The paper introduces a Hessian-based method to improve generalization in fine-tuned deep neural networks.
problem Improving generalization in fine-tuned deep neural networks, especially in noisy conditions.
method PAC-Bayesian analysis to identify a Hessian-based distance measure, proving generalization bounds, and developing an algorithm with a generalization error guarantee.
result Hessian-based distance measure correlates well with observed generalization gaps and can match the scale of these gaps in practice.
Homotopy types of curve and arc complexes are studied.
problem Understanding the homotopy types of curve and arc complexes.
method Proving homotopy equivalence and contractibility of complexes.
result Fine curve complex is homotopy equivalent to curve complex, fine arc complex is contractible.
Fine-tuning LLMs improves capability but harms safety, study finds.
problem Balancing capability and safety in LLM fine-tuning.
method Theoretical framework and numerical experiments for two safety-aware fine-tuning strategies.
result Characterization of fundamental limits of safety-capability trade-off in LLM fine-tuning.
Optimizes sparse fine-tuning for privacy in neural networks.
problem Performance gap between DP-SGD and non-private fine-tuning.
method Optimization-based approach using private gradient information for selecting trainable weights.
result Our selection method leads to better prediction accuracy compared to existing approaches.
Fine-tuning harms in-context learning, but restricting updates to the value matrix improves zero-shot performance.
problem Fine-tuning harms in-context learning, reducing zero-shot performance on unseen tasks.
method Theoretical analysis of linear attention models, identifying conditions for degraded few-shot performance.
result Restricting updates to the value matrix improves zero-shot performance while preserving in-context learning.
Measures consistency of tabular LLM predictions under fine-tuning multiplicity.
problem Conflicting predictions from fine-tuned tabular LLMs.
method Local stability measure in embedding space.
result Probabilistic guarantees on prediction consistency under multiplicity.
Self-play fine-tuning improves diffusion models for text-to-image generation.
problem Plateauing performance of diffusion models after data saturation.
method Self-play fine-tuning (SPIN-Diffusion) using competition among model versions.
result Significantly improved model performance and human preference alignment.
The paper develops a theory linking pretraining and fine-tuning in neural networks.
problem Understanding how initialization choices impact feature learning and generalization in neural networks.
method Analytical theory of diagonal linear networks, deriving generalization error as a function of initialization parameters and task statistics.
result Different initialization choices place networks into four fine-tuning regimes with varying abilities to support feature learning and generalization.
Automorphisms of fine 1-curve graph linked to surface homeomorphisms.
problem Understanding automorphisms of fine 1-curve graphs.
method Isomorphic mapping to surface homeomorphisms.
result Automorphism group is isomorphic to homeomorphism group of a surface.
New method reduces fine-tuning cost for reused models.
problem Repeating fine-tuning costs with outdated foundation models.
method Portable Reward Tuning (PRT) trains a reward model to maximize the same loss function as fine-tuning.
result PRT achieves comparable accuracy to inference-time tuning with less inference cost.
Compact models match or exceed GPT's performance in financial news sentiment analysis.
problem Improving financial sentiment analysis models without large computational costs.
method Fine-tuning non-generative, small-sized models (FinBERT, FinDRoBERTa) on a novel market score database.
result Fine-tuned models outperform GPT-3.5 and GPT-4 in zero-shot learning for financial news sentiment analysis.
Automorphisms of fine graphs for surfaces and tori are studied.
problem Understanding automorphisms of fine graphs for surfaces and tori.
method Extending previous results to tori and discussing smooth versions.
result Automorphism groups of fine graphs for surfaces and tori are naturally isomorphic to homeomorphism groups.
Fine-tuning large language models requires minimal data, making them efficient.
problem Achieving state-of-the-art performance with large language models.
method Using BERT as an example, fine-tuning only the most critical layers of the pre-trained model.
result Fine-tuned models are close in parameter space to the pre-trained model, with many good solutions found in sparsified versions.
Unsupervised pre-training improves model generalization, but lacks theoretical understanding.
problem Lack of theoretical understanding of unsupervised pre-training's impact on model generalization.
method Introduces a novel theoretical framework to analyze and enhance generalization.
result Enhances understanding of unsupervised pre-training and fine-tuning, proposing a new regularization method.
LP-FT improves personalized model training in FL by balancing generalization and personalization.
problem Federated Learning struggles with balancing global generalization and local personalization due to non-identical data distributions.
method Adapting Linear Probing followed by full Fine-Tuning (LP-FT) to the FL setting.
result LP-FT outperforms standard fine-tuning in balancing personalization and generalization across various datasets and PFT variants.
UBM transfers bias mitigation from upstream to downstream tasks efficiently.
problem Bias in fine-tuned language models across various tasks.
method Apply bias mitigation to an upstream model, then fine-tune a downstream model on this mitigated model.
result UBM effects transfer to new downstream tasks, creating less biased models.
Improved fine-tuning with regularization and robustness for noisy labels.
problem Fine-tuning pre-trained models on small datasets can lead to overfitting and memorization.
method PAC-Bayes generalization bound analysis, layer-wise regularization, self-label-correction, label-reweighting.
result Improves performance by 1.76% on average for image classification tasks and 0.75% for few-shot classification.
Fine shape of local compacta represented by ordinary maps.
problem Representing fine shape of local compacta.
method Constructing a space ∣X∣ for each local compactum X such that fine shape classes correspond to homotopy classes of maps to ∣X∣. result Fine shape classes from any locally compact metrizable space Y to X bijectively correspond to homotopy classes of maps from Y to ∣X∣. Simplified trust region method reduces representation change during fine-tuning.
problem Stability and representational collapse in fine-tuning pre-trained models.
method Replaces adversarial objectives with parametric noise in trust region theory.
result Matches or exceeds previous trust region methods in performance and speed.
Adjoint Matching improves flow and diffusion models with reward fine-tuning.
problem Improving generative models with reward fine-tuning.
method Casting reward fine-tuning as stochastic optimal control (SOC) and enforcing a specific noise schedule.
result Adjoint Matching outperforms existing SOC algorithms.
Fine-tunes LLMs to correct bias in predictions.
problem LLMs exhibit bias in predictions from data.
method Supervised fine-tuning with Low-Rank Adaptation (LoRA).
result Fine-tuning corrects bias in both controlled and real-world settings.
Transfer learning, which allows a source task to affect the inductive bias of the target task, is widely used in computer vision. The typical way of conducting transfer learning with deep neural networks is to fine-tune a model pre-trained on the source task using data from the target task. In this paper, we propose an…
Fine-tuning improves meta-learning by leveraging shared representations.
problem Meta-learning's challenge in rapidly learning new tasks.
method Theoretical framework and risk bounds on gradient descent fine-tuning.
result Fine-tuning-based methods can provably leverage shared structure.
New method enhances model fine-tuning with minimal data.
problem Improving model performance on new tasks with limited data.
method Introducing α-LoRA, a reparameterization method for fine-tuning. result Enhanced generalization ability of fine-tuned models.
Transformers fine-tuned on synthetic data boost tabular data classification performance.
problem Improving tabular data classification accuracy.
method Fine-tuning ICL-transformers on synthetic datasets with complex decision boundaries.
result Fine-tuned ICL-transformers outperform regular neural networks on real-world datasets.
A new method constrains deep networks during fine-tuning to improve generalization.
problem Improving generalization of fine-tuned deep networks.
method A neural network generalisation bound based on distance from initial weights constrains the hypothesis class to a small sphere.
result Empirical evaluation shows superior generalization performance compared to existing methods.