arXiv research

The IM effect helps detect anomalies by memorizing inliers early.

A novel framework IMBoost improves outlier detection by leveraging the inlier memorization effect.

ODIM detects outliers by under-fitting generative models, outperforming other methods.

ALTBI enhances outlier detection by maximizing the inlier-memorization effect.

Theoretical guarantees for STE, a robust subspace recovery method.

We propose a neural network for unsupervised anomaly detection with a novel robust subspace recovery layer (RSR layer). This layer seeks to extract the underlying subspace from a latent representation of the given data and removes outliers that lie away from this subspace. It is used within an autoencoder. The encoder …

We propose an inlier-based outlier detection method capable of both identifying the outliers and explaining why they are outliers, by identifying the outlier-specific features. Specifically, we employ an inlier-based outlier detection criterion, which uses the ratio of inlier and test probability densities as a measure…

This paper presents a remarkably simple, yet powerful, algorithm termed Coherence Pursuit (CoP) to robust Principal Component Analysis (PCA). As inliers lie in a low dimensional subspace and are mostly correlated, an inlier is likely to have strong mutual coherence with a large number of data points. By contrast, outli…

Neural networks memorize exceptions, leading to poor generalization.

Local data coverage governs memorization in diffusion models.

Geometric framework explains memorization in generative models.

PRAE identifies outliers and reconstructs inliers in autoencoders.

The paper identifies patterns in language model weights used for memorizing paragraphs.

Consistency distillation reduces memorization in diffusion models without harming sample quality.

Study shows how deep generative models can memorize data.

Linear regression models contaminated by Gaussian noise (inlier) and possibly unbounded sparse outliers are common in many signal processing applications. Sparse recovery inspired robust regression (SRIRR) techniques are shown to deliver high quality estimation performance in such regression models. Unfortunately, most…

We study finite sample expressivity, i.e., memorization power of ReLU networks. Recent results require $N$ hidden nodes to memorize/interpolate arbitrary $N$ data points. In contrast, by exploiting depth, we show that 3-layer ReLU networks with $Ω(\sqrt{N})$ hidden nodes can perfectly memorize most datasets with $N$ po…

Memorizing rare examples helps neural networks generalize better.

Optimal ReLU networks can memorize any separable set of points with a small number of parameters.

LLMs can memorize economic data and recall exact values before their training cutoff.

New method reduces memorization in diffusion models without sacrificing image quality.

Logarithmic network width suffices for robust memorization.

New approach shows data memorization trade-offs in large models.

Study compares memorization of SimCLR to supervised and random labels training.

Generative diffusion models gradually memorize training data, losing independent dimensions.

There is an increasingly apparent need for validating the classifications made by deep learning systems in safety-critical applications like autonomous vehicle systems. A number of recent papers have proposed methods for detecting anomalous image data that appear different from known inlier data samples, including reco…

Deep networks preferentially learn shared features, avoiding memorization in early layers.

Deep networks can memorize random labels; symmetric loss improves this.

This paper explores memorization in adversarial training and proposes a mitigation algorithm.

Study shows FL reduces unintended memorization by clustering data and using strong user-level privacy.

ResMem improves model generalization by explicitly memorizing residuals.

Paper introduces RAS for robust MTL with contamination.

Diffusion models can memorize training data, limiting their creativity and privacy.

RAF model explains neural networks' dual rule learning and fact memorization.

Learning to solve sequential tasks with recurrent models requires the ability to memorize long sequences and to extract task-relevant features from them. In this paper, we study the memorization subtask from the point of view of the design and training of recurrent neural networks. We propose a new model, the Linear Me…

Gradient descent memorizes many Gaussians efficiently.

Introduces Relational Privacy (RP) to control relation memorization in question answering models.

The bias potential model explains how generative models can generalize or memorize samples.

The study uncovers the conditions under which diffusion models memorize or generalize.

State-of-the-art results on image recognition tasks are achieved using over-parameterized learning algorithms that (nearly) perfectly fit the training set and are known to fit well even random labels. This tendency to memorize the labels of the training data is not explained by existing theoretical analyses. Memorizati…

Deep models memorize training data in geophysical inversion, leading to biased posterior distributions.

New training method for ReLU networks achieves optimal weight size for memorization.

Diffusion models generalize well until a threshold is reached, preventing memorization.

Large learning rates prevent memorization in denoising score matching.

Transformers with CoT don't enhance reasoning power across all tasks.

Consider a dataset of vector-valued observations that consists of noisy inliers, which are explained well by a low-dimensional subspace, along with some number of outliers. This work describes a convex optimization problem, called REAPER, that can reliably fit a low-dimensional model to this type of data. This approach…

Study interprets neural network generalization and memorization on corrupted data.

Early stopping improves generalization in overparameterized diffusion models.