arXiv research

Data augmentation methods improve worst-case model performance.

Last layer retraining improves robustness to spurious correlations without high computational costs.

This research shows loss weighting remains effective in last layer retraining despite model overparameterization.

RAD improves robustness to domain annotation noise without explicit domain annotations.

New method assesses individual training points' privacy risk without retraining.

Residual Networks with convolutional layers are widely used in the field of machine learning. Since they effectively extract features from input data by stacking multiple layers, they can achieve high accuracy in many applications. However, the stacking of many layers raises their computation costs. To address this pro…

CLAPS improves conformal regression by adaptively scaling interval widths based on last-layer Laplace uncertainty.

In this paper, we present experimental results obtained from retraining the last layer of the Inception v3 model in classifying images of human faces into one of five basic face shapes. The accuracy of the retrained Inception v3 model was compared with that of the following classification methods that uses facial landm…

New method improves BLL models for complex datasets.

Optimizes neural networks' last layer with closed-form solutions.

Improved uncertainty estimation in neural networks with VBLL.

Layer fusion reduces deep neural network layers with minimal loss in accuracy.

Optimizes weights for better model performance in shifting data.

Study on self-consuming generative models with diverse human curation, focusing on convergence and stability.

Last-layer approximation improves UQ performance without sacrificing computational efficiency.

Recent DNN pruning algorithms have succeeded in reducing the number of parameters in fully connected layers, often with little or no drop in classification accuracy. However, most of the existing pruning schemes either have to be applied during training or require a costly retraining procedure after pruning to regain c…

. In this paper, an effective computer-aided diagnosis (CAD) system is presented to detect MI signals using the convolution neural network (CNN) for urban healthcare in smart cities. Two types of transfer learning techniques are employed to retrain the pre-trained VGG-Net (Fine-tuning and VGG-Net as fixed feature extra…

The most common method for DNN pruning is hard thresholding of network weights, followed by retraining to recover any lost accuracy. Recently developed smart pruning algorithms use the DNN response over the training set for a variety of cost functions to determine redundant network weights, leading to less accuracy deg…

Quantized deep neural networks (QDNNs) are attractive due to their much lower memory storage and faster inference speed than their regular full precision counterparts. To maintain the same performance level especially at low bit-widths, QDNNs must be retrained. Their training involves piecewise constant activation func…

Bayesian approach learns invariances from data alone, but last layer approximation is not always sufficient.

Gradient ascent method successfully removes specific data points from neural networks without retraining.

Dynamic pruning during training reduces deep network complexity without significant accuracy loss.

Proposes PSCs for UQ in deep nets without retraining.

Proposes DAK model for improved GP computations.

Our paper explains deep neural collapse in multiple layers.

Wide neural networks' last hidden layers split into groups of redundant neurons.

Deep neural networks converge to Gaussian mixtures as layer width increases.

We analyze neural collapse in neural networks, showing that features collapse to vertices of a Simplex ETF.

Bayesian neural network models improve uncertainty quantification in multivariate regression.

One of the main challenges of deep learning methods is the choice of an appropriate training strategy. In particular, additional steps, such as unsupervised pre-training, have been shown to greatly improve the performances of deep structures. In this article, we propose an extra training step, called post-training, whi…

Bayesian optimization improved with variational last layer training.

A new algorithm learns from raw feature vectors using deep neural networks and UCB for exploration.

AFR simplifies reducing reliance on spurious features, improving model performance.

End-to-end automatic speech recognition (ASR) models are increasingly large and complex to achieve the best possible accuracy. In this paper, we build an AutoML system that uses reinforcement learning (RL) to optimize the per-layer compression ratios when applied to a state-of-the-art attention based end-to-end ASR mod…

Proposes QEP to mitigate quantization error propagation in layer-wise post-training quantization.

Normalization methods play an important role in enhancing the performance of deep learning while their theoretical understandings have been limited. To theoretically elucidate the effectiveness of normalization, we quantify the geometry of the parameter space determined by the Fisher information matrix (FIM), which als…

Recent machine learning methods use increasingly large deep neural networks to achieve state of the art results in various tasks. The gains in performance come at the cost of a substantial increase in computation and storage requirements. This makes real-time implementations on limited resources hardware a challenging …

Study clarifies Bayesian generalization error in CBM for 3-layered linear neural networks.

Proposes a method to retrain neural networks incrementally for continuous data flow.

LLEB uses a learned prior to quantify neural network uncertainty.

The monotonic linear interpolation in deep networks often leads to plateaus, revealing biases in optimization.

Deep linear networks exhibit collapsing features and classifiers across datasets.

Retraining with predicted labels improves model accuracy in noisy settings.

In this paper we propose a Bayesian method for estimating architectural parameters of neural networks, namely layer size and network depth. We do this by learning concrete distributions over these parameters. Our results show that regular networks with a learnt structure can generalise better on small datasets, while f…

This paper extends neural collapse to regression problems, revealing key features and structures.

We improve neural network explainability by bypassing batch normalization.

DeltaGrad rapidly retrain models with minimal data changes.

This work justifies neural collapse under MSE loss and analyzes the optimization landscape.