arXiv research

Improved neural population modeling using shared features and ensemble detection.

New model predicts neural network performance from early training epochs, incorporating architecture impact.

New metric captures individual neuron tuning across neural networks.

Confidence bands for tuning curves improve hyperparameter comparison in NLP.

Machine learning classifies surface wave dispersion curves from ambient noise.

New method adapts neural networks without losing prior knowledge.

HAMLET optimizes algorithm selection for machine learning tasks.

LoRA-Curve connects independent LoRA optima through continuous low-loss valleys, improving Bayesian model averaging.

Fine-tuning neural networks to guarantee performance on specific examples can also introduce incorrect inputs.

Optimizes sparse fine-tuning for privacy in neural networks.

SMART-FAN-Lasso fine-tunes neural networks for high-dimensional nonparametric regression.

Two retraining techniques outperform fine-tuning in neural network pruning.

Transfer learning boosts chemically accurate neural network potentials for organic molecules.

The paper introduces a Hessian-based method to improve generalization in fine-tuned deep neural networks.

Deep learning algorithms have achieved excellent performance lately in a wide range of fields (e.g., computer version). However, a severe challenge faced by deep learning is the high dependency on hyper-parameters. The algorithm results may fluctuate dramatically under the different configuration of hyper-parameters. A…

A common analytical problem in neuroscience is the interpretation of neural activity with respect to sensory input or behavioral output. This is typically achieved by regressing measured neural activity against known stimuli or behavioral variables to produce a "tuning function" for each neuron. Unfortunately, because …

Unified method visualizes curvature on curves and surfaces.

The paper develops a theory linking pretraining and fine-tuning in neural networks.

When applying Machine Learning techniques to problems, one must select model parameters to ensure that the system converges but also does not become stuck at the objective function's local minimum. Tuning these parameters becomes a non-trivial task for large models and it is not always apparent if the user has found th…

A new method constrains deep networks during fine-tuning to improve generalization.

New measure FTC quantifies how much a ReLU network can fine-tune.

Learning shrinks hard tail, improving inference performance.

In order to interact intelligently with objects in the world, animals must first transform neural population responses into estimates of the dynamic, unknown stimuli which caused them. The Bayesian solution to this problem is known as a Bayes filter, which applies Bayes' rule to combine population responses with the pr…

Echo State Networks (ESNs) are recurrent neural networks that only train their output layer, thereby precluding the need to backpropagate gradients through time, which leads to significant computational gains. Nevertheless, a common issue in ESNs is determining its hyperparameters, which are crucial in instantiating a …

ICE-Pruning accelerates deep neural network pruning by 9.61x.

Proposes a learning rate method for shallow nets based on gradient Lipschitz constant.

Self-supervised fine-tuning corrects SR CNNs for unseen models and artifacts.

Automates fitting semiconductor device models using approximate Bayesian computation.

Multirate training speeds up neural network fine-tuning.

New benchmark protocol evaluates neural network optimizers for efficiency and data shift sensitivity.

Study of closed real plane curves with hyperelliptic genus three solutions.

Paper proves multiplicative weight updates can train neural networks without learning rate tuning.

Cold posteriors improve Bayesian neural networks by reducing overestimation of aleatoric uncertainty.

Sherpa automates hyperparameter tuning for machine learning models.

Paper fine-tunes LLaMA-3-8B for financial NER using instruction and LoRA.

Theoretical framework explains why few epochs are enough for LLM fine-tuning.

Fine-tunes GNNs by preserving generative patterns to improve transferability.

Deep networks achieve linear separability through progressive folding of data in higher dimensions.

Fine-tuning normalization layers can reconstruct smaller networks.

In this paper, we consider the discrete deformation of the discrete space curves with constant torsion described by the discrete mKdV or the discrete sine-Gordon equations, and show that it is formulated as the torsion-preserving equidistant deformation on the osculating plane which satisfies the isoperimetric conditio…

Bayesian optimization sped up with importance sampling.

Alternative neural network training using monotone variational inequality.

The splendid success of convolutional neural networks (CNNs) in computer vision is largely attributable to the availability of massive annotated datasets, such as ImageNet and Places. However, in medical imaging, it is challenging to create such large annotated datasets, as annotating medical images is not only tedious…

PASHA optimizes model tuning for large datasets with limited resources.

Multi-label classification is an approach which allows a datapoint to be labelled with more than one class at the same time. A common but trivial approach is to train individual binary classifiers per label, but the performance can be improved by considering associations within the labels. Like with any machine learnin…

We formalize AURC and develop estimators for SC systems.

Classical Machine Learning (ML) pipelines often comprise of multiple ML models where models, within a pipeline, are trained in isolation. Conversely, when training neural network models, layers composing the neural models are simultaneously trained using backpropagation. We argue that the isolated training scheme of ML…

Improved hypernetwork for efficient neural network hyperparameter tuning.