arXiv research

CLAPS improves conformal regression by adaptively scaling interval widths based on last-layer Laplace uncertainty.

Data augmentation methods improve worst-case model performance.

New method improves BLL models for complex datasets.

Last layer retraining improves robustness to spurious correlations without high computational costs.

Optimizes neural networks' last layer with closed-form solutions.

Improved uncertainty estimation in neural networks with VBLL.

Last-layer approximation improves UQ performance without sacrificing computational efficiency.

This research shows loss weighting remains effective in last layer retraining despite model overparameterization.

Bayesian approach learns invariances from data alone, but last layer approximation is not always sufficient.

Proposes DAK model for improved GP computations.

Our paper explains deep neural collapse in multiple layers.

Wide neural networks' last hidden layers split into groups of redundant neurons.

Deep neural networks converge to Gaussian mixtures as layer width increases.

We analyze neural collapse in neural networks, showing that features collapse to vertices of a Simplex ETF.

Bayesian neural network models improve uncertainty quantification in multivariate regression.

One of the main challenges of deep learning methods is the choice of an appropriate training strategy. In particular, additional steps, such as unsupervised pre-training, have been shown to greatly improve the performances of deep structures. In this article, we propose an extra training step, called post-training, whi…

Bayesian optimization improved with variational last layer training.

A new algorithm learns from raw feature vectors using deep neural networks and UCB for exploration.

RAD improves robustness to domain annotation noise without explicit domain annotations.

Normalization methods play an important role in enhancing the performance of deep learning while their theoretical understandings have been limited. To theoretically elucidate the effectiveness of normalization, we quantify the geometry of the parameter space determined by the Fisher information matrix (FIM), which als…

Study clarifies Bayesian generalization error in CBM for 3-layered linear neural networks.

LLEB uses a learned prior to quantify neural network uncertainty.

The monotonic linear interpolation in deep networks often leads to plateaus, revealing biases in optimization.

The lack of transparency of neural networks stays a major break for their use. The Layerwise Relevance Propagation technique builds heat-maps representing the relevance of each input in the model s decision. The relevance spreads backward from the last to the first layer of the Deep Neural Network. Layer-wise Relevance…

Deep linear networks exhibit collapsing features and classifiers across datasets.

This paper extends neural collapse to regression problems, revealing key features and structures.

This work justifies neural collapse under MSE loss and analyzes the optimization landscape.

In this paper, a multi-layer architecture (in a hierarchical fashion) by stacking various Kernel Ridge Regression (KRR) based Auto-Encoder for one-class classification is proposed and is referred as MKOC. MKOC has many layers of Auto-Encoders to project the input features into new feature space and the last layer was r…

Develops multi-modal neural network models for improved prediction and uncertainty quantification.

New method assesses individual training points' privacy risk without retraining.

For any positive integer $k$, there exist neural networks with $Θ(k^3)$ layers, $Θ(1)$ nodes per layer, and $Θ(1)$ distinct parameters which can not be approximated by networks with $\mathcal{O}(k)$ layers unless they are exponentially large --- they must possess $Ω(2^k)$ nodes. This result is proved here for a class o…

New conditions ensure deep neural networks can approximate any function on non-Euclidean spaces.

Deep neural networks progressively transform their inputs across multiple processing layers. What are the geometrical properties of the representations learned by these networks? Here we study the intrinsic dimensionality (ID) of data-representations, i.e. the minimal number of parameters needed to describe a represent…

New method calibrates LLMs for safety-critical tasks with scalable Bayesian inference.

Neural collapse occurs in normalized features over a Riemannian manifold.

Improved DNN calibration without sacrificing accuracy.

We introduce a temperature into the exponential function and replace the softmax output layer of neural nets by a high temperature generalization. Similarly, the logarithm in the log loss we use for training is replaced by a low temperature logarithm. By tuning the two temperatures we create loss functions that are non…

ReLU networks trained with MILPs match deep learning accuracy.

The study estimates the expressiveness of GCNs with bounds on the number of linear regions.

A new method for estimating uncertainty in deep neural networks.

Classical person re-identification approaches assume that a person of interest has appeared across different cameras and can be queried by one of the existing images. However, in real-world surveillance scenarios, frequently no visual information will be available about the queried person. In such scenarios, a natural …

Over the last few years, Deep Neural Networks (DNNs) have become ubiquitous owing to their high accuracy on real-world tasks. However, this increase in accuracy comes at the cost of computationally expensive models leading to higher prediction latencies. Prior efforts to reduce this latency such as quantization, model …

SGD's training dynamics align with Hessian and gradient spectra in high-dimensional classification tasks.

Study explains how noisyGD with DP improves feature learning despite high dimensionality.

Study on Neural Collapse limits in deep learning.

This paper presents a novel technique based on gradient boosting to train the final layers of a neural network (NN). Gradient boosting is an additive expansion algorithm in which a series of models are trained sequentially to approximate a given function. A neural network can also be seen as an additive expansion where…

Deep random feature models are analyzed for their performance with exact asymptotic expressions.

In general, a self-attention mechanism has been applied for speaker embedding encoding. Previous studies focused on training the self-attention in a high-level layer, such as the last pooling layer. However, the effect of low-level features was reduced in the speaker embedding encoding. Therefore, we propose masked cro…