New method for tuning Graphical Lasso hyperparameters.
problem Tuning hyperparameters of Graphical Lasso.
method Bilevel optimization with first-order method.
result Derivation of Graphical Lasso Jacobian.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “tuning methods”
A new approach selects tuning parameters for embedding methods.
problem Difficulty in selecting tuning parameters for embedding methods.
method Minimize a stress notion to supervise tuning parameter selection.
result Uncover a new bias--variance tradeoff phenomenon.
Tuning-free OR-PCA improves scalability for large datasets.
problem Dataset sensitivity of OR-PCA tuning parameters.
method Implicit regularization through modified gradient descents.
result Comparable or better performance on simulated and real-world datasets.
ADAPT improves robustness of Vision Transformers without full model fine-tuning.
problem Vulnerability of Vision Transformers to adversarial attacks.
method Parameter-efficient prompt tuning with ADAPT framework for adaptive adversarial training.
result ADAPT achieves robust accuracy of ~40% w.r.t. SOTA methods using only ~1% of the parameters.
Simplified trust region method reduces representation change during fine-tuning.
problem Stability and representational collapse in fine-tuning pre-trained models.
method Replaces adversarial objectives with parametric noise in trust region theory.
result Matches or exceeds previous trust region methods in performance and speed.
Efficiently tunes hyperparameters with dynamic accuracy method.
problem Optimizing machine learning hyperparameters with inexact evaluations.
method Dynamic accuracy derivative-free optimization for hyperparameter tuning.
result Demonstrates robust and efficient hyperparameter tuning compared to fixed accuracy methods.
New method for hyperparameter tuning in sparse matrix factorization.
problem Hyperparameter tuning in sparse matrix factorization.
method Numerical method based on evaluating the zero point of normalization factor in sparse matrix prior.
result Our method outperforms existing algorithms in ground-truth sparse matrix reconstruction.
A cost-efficient method for hyperparameter tuning using multi-fidelity Bayesian optimization.
problem Expensive hyperparameter tuning with limited knowledge transfer methods.
method Amortized Auto-Tuning (AT2) framework for multi-task, multi-fidelity Bayesian optimization.
result AT2 leads to the best hyperparameter recommendation and is more cost-efficient.
New method fine-tunes discrete diffusion models for RLHF tasks.
problem Fine-tuning discrete diffusion models with policy gradient methods is challenging.
method Proposed SEPO algorithm for efficient fine-tuning over non-differentiable rewards.
result Numerical experiments show scalability and efficiency of SEPO.
InfoPrompt improves soft prompt tuning by maximizing mutual information, leading to better performance.
problem High sensitivity of prompt tuning to initial conditions and insufficient task-relevant information.
method Develops an information-theoretic framework to maximize mutual information between prompts and model parameters, using novel loss functions.
result InfoPrompt accelerates convergence and outperforms traditional methods.
Method constructs finance LLMs without instruction data using pretraining and model merging.
problem Developing domain-specific LLMs for finance is resource-intensive.
method Continual pretraining on financial data + model merging of instruction-tuned and domain-specific pretrained vectors.
result Successfully constructs instruction-tuned LLMs for finance without additional instruction data.
Study shows leaving some hyperparameters at default can outperform tuning them.
problem Importance of tuning hyperparameters in machine learning performance.
method Non-inferiority test and tuning risk methodology.
result Leaving some hyperparameters at default can outperform tuning them.
Improved financial VA intent classification accuracy.
problem Determining user intents for unseen open intents.
method Supervised pre-training of intent representations using prefix-tuning and fine-tuning.
result 1.63% - 2.07% higher accuracy on banking77 benchmark.
New method improves hyperparameter tuning efficiency across similar tasks.
problem Mismatch between evaluations in current and previous tasks.
method Nested drop-out and auto-relevance determination for learning basis functions of increasing complexity.
result Improves sample efficiency in hyperparameter tuning across different data regimes.
New method enhances model fine-tuning with minimal data.
problem Improving model performance on new tasks with limited data.
method Introducing -LoRA, a reparameterization method for fine-tuning.
result Enhanced generalization ability of fine-tuned models.
This paper introduces an efficient method for optimizing deep learning hyperparameters.
problem The high dependency of deep learning algorithms on hyper-parameters.
method Orthogonal Array Tuning Method (OATM) for deep learning hyper-parameter tuning.
result The proposed OATM method significantly saves tuning time compared to state-of-the-art methods.
Our method upweights easy samples to mitigate forgetting in fine-tuning.
problem Catastrophic forgetting in fine-tuning pre-trained models.
method Sample weighting based on pre-trained model's losses.
result Our method reduces forgetting by up to 0.8% on MetaMathQA while preserving more accuracy on pre-training datasets.
New method FedEx accelerates federated hyperparameter tuning.
problem Federated hyperparameter tuning challenges in distributed learning.
method FedEx method connecting to weight-sharing, adapted for federated optimization.
result FedEx outperforms natural baselines on various benchmarks.
Confidence bands for tuning curves improve hyperparameter comparison in NLP.
problem Ambiguity in comparing hyperparameter tuning methods.
method Constructs exact, simultaneous, and distribution-free confidence bands for tuning curves.
result Confidence bands provide a robust basis for comparing methods rigorously.
A new robust prefix-tuning framework improves model robustness against adversarial attacks.
problem Lack of robustness in prefix-tuning for adversarial attacks.
method Leveraging layerwise activations of pretrained models for additional prefix finetuning during the test phase.
result Framework substantially improves robustness over strong baselines while maintaining comparable accuracy on clean texts.
The paper introduces a Hessian-based method to improve generalization in fine-tuned deep neural networks.
problem Improving generalization in fine-tuned deep neural networks, especially in noisy conditions.
method PAC-Bayesian analysis to identify a Hessian-based distance measure, proving generalization bounds, and developing an algorithm with a generalization error guarantee.
result Hessian-based distance measure correlates well with observed generalization gaps and can match the scale of these gaps in practice.
New method quantifies uncertainty in fine-tuned LLMs using LoRA ensembles.
problem Uncertainty in fine-tuned LLMs and how to trust their predictions.
method Posterior approximations using low-rank adaptation ensembles.
result Unexpected retention of acquired knowledge during fine-tuning in overfitting regime.
This paper tackles hyperparameter tuning for large-scale kernel ridge regression.
problem Hyperparameter tuning is crucial but often left to users, hindering efficiency and usability.
method Proposes a complexity regularization criterion based on a data-dependent penalty for efficient optimization.
result Demonstrates the benefit of the proposed approach through extensive empirical evaluation.
Improved fine-tuning with regularization and robustness for noisy labels.
problem Fine-tuning pre-trained models on small datasets can lead to overfitting and memorization.
method PAC-Bayes generalization bound analysis, layer-wise regularization, self-label-correction, label-reweighting.
result Improves performance by 1.76% on average for image classification tasks and 0.75% for few-shot classification.
Self-play fine-tuning improves diffusion models for text-to-image generation.
problem Plateauing performance of diffusion models after data saturation.
method Self-play fine-tuning (SPIN-Diffusion) using competition among model versions.
result Significantly improved model performance and human preference alignment.
Tuning SVM and boosting models using optimization algorithms.
problem Tuning parameters for SVM and boosting models across various datasets.
method Used grid search to identify parameter ranges and optimization algorithms to select models.
result Optimization algorithms outperformed grid search in selecting well-performing models.
Cyclical learning rates improve DRL performance without manual tuning.
problem Manual hyperparameter tuning in DRL is time-consuming and error-prone.
method Proposes cyclical learning rates for DRL problems.
result Cyclical learning achieves similar or better results than fixed learning rates.
Efficiently tunes hyperparameters for knowledge graph embeddings.
problem Hyperparameter tuning is challenging and costly for knowledge graph embeddings.
method Augmented probabilistic framework with variational expectation-maximization.
result Achieved state-of-the-art performance in link prediction.
Three methods for tuning HMC diagonal scale matrices compared.
problem Improving Hamiltonian Monte Carlo efficiency with diagonal scale matrices.
method Three approaches: ISG, median crossing frequency, and estimated marginal standard deviations.
result ISG method leads to more efficient sampling in many cases.
Adjoint Matching improves flow and diffusion models with reward fine-tuning.
problem Improving generative models with reward fine-tuning.
method Casting reward fine-tuning as stochastic optimal control (SOC) and enforcing a specific noise schedule.
result Adjoint Matching outperforms existing SOC algorithms.
New algorithm tunes SGMCMC hyperparameters for scalable Bayesian inference.
problem Tuning hyperparameters for SGMCMC is challenging due to lack of principled methods.
method Proposes a bandit-based algorithm using Stein discrepancies to tune hyperparameters.
result The method effectively tunes SGMCMC hyperparameters for various applications.
New method reduces fine-tuning cost for reused models.
problem Repeating fine-tuning costs with outdated foundation models.
method Portable Reward Tuning (PRT) trains a reward model to maximize the same loss function as fine-tuning.
result PRT achieves comparable accuracy to inference-time tuning with less inference cost.
Optimizes sparse fine-tuning for privacy in neural networks.
problem Performance gap between DP-SGD and non-private fine-tuning.
method Optimization-based approach using private gradient information for selecting trainable weights.
result Our selection method leads to better prediction accuracy compared to existing approaches.
Study assesses hyperparameter tuning for causal inference with DML.
problem Optimizing hyperparameters for causal inference with DML.
method Empirical simulation study using DML approach.
result Hyperparameter tuning crucial for causal estimation with DML.
A new method optimizes diffusion models for fine-tuning tasks efficiently.
problem Optimizing diffusion models for downstream tasks using nested bilevel structures.
method Formalizes the challenge as a generative bilevel optimization problem and introduces a first-order bilevel framework.
result Our method outperforms existing fine-tuning and hyperparameter search baselines.
New algorithms eliminate stepsize tuning for bilevel optimization problems.
problem Bilevel optimization problems with unknown parameters and stepsizes.
method D-TFBO and S-TFBO algorithms with adaptive stepsizes.
result Achieve performance comparable to well-tuned approaches with theoretical guarantees.
Transfer learning, which allows a source task to affect the inductive bias of the target task, is widely used in computer vision. The typical way of conducting transfer learning with deep neural networks is to fine-tune a model pre-trained on the source task using data from the target task. In this paper, we propose an…
DPZero fine-tunes large models privately without backpropagation.
problem Memory and privacy challenges in fine-tuning large language models.
method DPZero uses zeroth-order methods for private fine-tuning, avoiding backpropagation.
result DPZero achieves private fine-tuning of RoBERTa and OPT on various tasks.
In this paper, we address the challenging problem of selecting tuning parameters for high-dimensional sparse regression. We propose a simple and computationally efficient method, called path thresholding (PaTh), that transforms any tuning parameter-dependent sparse regression algorithm into an asymptotically tuning-fre…
FGSM is more stable in adversarially robust transfer learning than PGD.
problem Computational efficiency in adversarially robust transfer learning.
method Revisited use of FGSM in adversarial fine-tuning.
result FGSM is more stable and efficient in adversarial fine-tuning.
A new method selects training samples for fine-tuning using validation set inference.
problem Selecting training examples for fine-tuning with limited target data.
method Invert train-validation roles; select samples affecting most predictions.
result Our method achieves lower test log-loss than state-of-the-art approaches.
PANDA improves linear discriminant analysis in high dimensions with minimal tuning.
problem Linear discriminant analysis in high-dimensional settings.
method PANDA: a tuning-insensitive method for linear discriminant analysis.
result PANDA achieves optimal convergence rates in estimation error and misclassification rate.
A common analytical problem in neuroscience is the interpretation of neural activity with respect to sensory input or behavioral output. This is typically achieved by regressing measured neural activity against known stimuli or behavioral variables to produce a "tuning function" for each neuron. Unfortunately, because …
Paper proposes a reinforcement learning framework for efficient hyper-parameter tuning of stochastic optimization algorithms.
problem Efficient tuning of hyper-parameters for stochastic optimization algorithms.
method Modeling hyper-parameter tuning as a Markov decision process and using policy gradient algorithms.
result The proposed framework significantly reduces the time required for hyper-parameter tuning compared to Bayesian optimization.
New method tunes SMC samplers efficiently without high costs.
problem Tuning SMC samplers with unadjusted kernels is challenging.
method Greedy Incremental Divergence Minimization (GIDM) for step size tuning.
result GIDM reduces KL divergence and tunes SMC samplers efficiently.
This paper optimizes binary linear classifiers by tuning their weight vectors.
problem Optimizing the weight vector of binary linear classifiers for better performance.
method Parameterization of the discriminant through a scalar to control trade-offs between informative and noisy terms.
result Weight vector tuning compensates for non-optimal native hyperparameters, improving classification performance.
This article reviews tuning parameter selection for high-dimensional regression.
problem Choosing the optimal tuning parameter for high-dimensional regression.
method The article discusses various strategies for selecting tuning parameters.
result The optimal tuning parameter depends on the design matrix and error distribution.
Fine-tuning improves meta-learning by leveraging shared representations.
problem Meta-learning's challenge in rapidly learning new tasks.
method Theoretical framework and risk bounds on gradient descent fine-tuning.
result Fine-tuning-based methods can provably leverage shared structure.