arXiv research

Fine-tuning neural networks to guarantee performance on specific examples can also introduce incorrect inputs.

Two retraining techniques outperform fine-tuning in neural network pruning.

New measure FTC quantifies how much a ReLU network can fine-tune.

The paper develops a theory linking pretraining and fine-tuning in neural networks.

The paper introduces a Hessian-based method to improve generalization in fine-tuned deep neural networks.

Transfer learning boosts chemically accurate neural network potentials for organic molecules.

A new method constrains deep networks during fine-tuning to improve generalization.

Optimizes sparse fine-tuning for privacy in neural networks.

Self-supervised fine-tuning corrects SR CNNs for unseen models and artifacts.

Multirate training speeds up neural network fine-tuning.

ICE-Pruning accelerates deep neural network pruning by 9.61x.

Alternative neural network training using monotone variational inequality.

SMART-FAN-Lasso fine-tunes neural networks for high-dimensional nonparametric regression.

Fine-tuning normalization layers can reconstruct smaller networks.

The splendid success of convolutional neural networks (CNNs) in computer vision is largely attributable to the availability of massive annotated datasets, such as ImageNet and Places. However, in medical imaging, it is challenging to create such large annotated datasets, as annotating medical images is not only tedious…

Feature representations from pre-trained deep neural networks have been known to exhibit excellent generalization and utility across a variety of related tasks. Fine-tuning is by far the simplest and most widely used approach that seeks to exploit and adapt these feature representations to novel tasks with limited data…

Classical Machine Learning (ML) pipelines often comprise of multiple ML models where models, within a pipeline, are trained in isolation. Conversely, when training neural network models, layers composing the neural models are simultaneously trained using backpropagation. We argue that the isolated training scheme of ML…

Transfer learning, which allows a source task to affect the inductive bias of the target task, is widely used in computer vision. The typical way of conducting transfer learning with deep neural networks is to fine-tune a model pre-trained on the source task using data from the target task. In this paper, we propose an…

Behavior Transfer improves reinforcement learning by leveraging pre-trained policies.

Neural network quantization procedure is the necessary step for porting of neural networks to mobile devices. Quantization allows accelerating the inference, reducing memory consumption and model size. It can be performed without fine-tuning using calibration procedure (calculation of parameters necessary for quantizat…

The paper proposes DEA to make graph neural networks fairer in link prediction.

POET enables large neural network training on tiny devices with reduced energy.

New debiasing methods for fine-tuned neural networks.

Improved drug-protein interaction prediction using FTL method.

SimCLR pre-training improves CNN performance with fewer labels.

Linearized neural networks provide a fast and interpretable way to adapt models to new settings.

New method adapts neural networks without losing prior knowledge.

New theory predicts deep neural networks can operate in an extended critical regime without fine-tuning.

The condition number predicts efficient information encoding in neural units, aiding model fine-tuning.

Optimizes pruning masks for neural networks using probabilistic fine-tuning and PAC-Bayes bounds.

In this paper, we present a novel approach for fine-tuning a decoder-side neural network in the context of image compression, such that the weight-updates are better compressible. At encoder side, we fine-tune a pre-trained artifact removal network on target data by using a compression objective applied on the weight-u…

FAST improves fast and stable task adaptation in DNNs.

Method makes non-interpretable models more intervenable.

Paper proposes neural networks for automatically naming assembly functions.

BayesAdapter turns pre-trained NNs into reliable BNNs with minimal overhead.

Fine-tunes GNNs by preserving generative patterns to improve transferability.

Deep neural networks have achieved impressive performance in many applications but their large number of parameters lead to significant computational and storage overheads. Several recent works attempt to mitigate these overheads by designing compact networks using pruning of connections. However, we observe that most …

In this study, we propose the integration of competitive learning into convolutional neural networks (CNNs) to improve the representation learning and efficiency of fine-tuning. Conventional CNNs use back propagation learning, and it enables powerful representation learning by a discrimination task. However, it require…

We ask whether the neural network interpretation methods can be fooled via adversarial model manipulation, which is defined as a model fine-tuning step that aims to radically alter the explanations without hurting the accuracy of the original models, e.g., VGG19, ResNet50, and DenseNet121. By incorporating the interpre…

Transformers fine-tuned on synthetic data boost tabular data classification performance.

New method removes specific training data influence from neural networks.

Unsupervised pre-training improves model generalization, but lacks theoretical understanding.

Deep-Lock secures DNN models with secret keys.

Fine-tuning improves meta-learning by leveraging shared representations.

AdaRound improves post-training quantization of neural networks.

The low-rank tensor approximation is very promising for the compression of deep neural networks. We propose a new simple and efficient iterative approach, which alternates low-rank factorization with a smart rank selection and fine-tuning. We demonstrate the efficiency of our method comparing to non-iterative ones. Our…

A new method for efficient neural network fine-tuning using queryable low-rank update atoms.

New method improves reliability of depth estimation models.