arXiv research

The IM effect helps detect anomalies by memorizing inliers early.

Introduces Relational Privacy (RP) to control relation memorization in question answering models.

This paper explores memorization in adversarial training and proposes a mitigation algorithm.

INN method refines clean labeled data from noisy labels.

Study shows FL reduces unintended memorization by clustering data and using strong user-level privacy.

A novel framework IMBoost improves outlier detection by leveraging the inlier memorization effect.

Deep networks learn clean structure before memorizing corrupted labels, leaving a spectral signature in gradient centered scatter.

ALTBI enhances outlier detection by maximizing the inlier-memorization effect.

Deep models memorize training data in geophysical inversion, leading to biased posterior distributions.

State-of-the-art results on image recognition tasks are achieved using over-parameterized learning algorithms that (nearly) perfectly fit the training set and are known to fit well even random labels. This tendency to memorize the labels of the training data is not explained by existing theoretical analyses. Memorizati…

Sample selection approaches are popular in robust learning from noisy labels. However, how to properly control the selection process so that deep networks can benefit from the memorization effect is a hard problem. In this paper, motivated by the success of automated machine learning (AutoML), we model this issue as a …

Diffusion models generalize well until a threshold is reached, preventing memorization.

Researchers validate ML scenario generators by checking dependencies and detecting memorization effects.

We examine the role of memorization in deep learning, drawing connections to capacity, generalization, and adversarial robustness. While deep networks are capable of memorizing noise data, our results suggest that they tend to prioritize learning simple patterns first. In our experiments, we expose qualitative differen…

Early stopping improves generalization in overparameterized diffusion models.

Mathematical analysis improves SGMs, resolving memorization issues.

Data selection boosts fact memorization in language models.

Study interprets neural network generalization and memorization on corrupted data.

This paper examines how Higher-Order Langevin Dynamics reduces memorization in diffusion models.

Neural networks memorize exceptions, leading to poor generalization.

Local data coverage governs memorization in diffusion models.

Geometric framework explains memorization in generative models.

The paper identifies patterns in language model weights used for memorizing paragraphs.

Consistency distillation reduces memorization in diffusion models without harming sample quality.

New algorithm interpolates data with neural nets, independent of sample size.

Study shows how deep generative models can memorize data.

New findings show tool-augmented models can recall unlimited facts, outperforming purely memorized models.

We study finite sample expressivity, i.e., memorization power of ReLU networks. Recent results require $N$ hidden nodes to memorize/interpolate arbitrary $N$ data points. In contrast, by exploiting depth, we show that 3-layer ReLU networks with $Ω(\sqrt{N})$ hidden nodes can perfectly memorize most datasets with $N$ po…

Memorizing rare examples helps neural networks generalize better.

Solves memorization in diffusion models for manifold data.

Optimal ReLU networks can memorize any separable set of points with a small number of parameters.

LLMs can memorize economic data and recall exact values before their training cutoff.

New method reduces memorization in diffusion models without sacrificing image quality.

We present a novel approach to leverage large unlabeled datasets by pre-training state-of-the-art deep neural networks on randomly-labeled datasets. Specifically, we train the neural networks to memorize arbitrary labels for all the samples in a dataset and use these pre-trained networks as a starting point for regular…

Spaced repetition is a technique for efficient memorization which uses repeated, spaced review of content to improve long-term retention. Can we find the optimal reviewing schedule to maximize the benefits of spaced repetition? In this paper, we introduce a novel, flexible representation of spaced repetition using the …

Logarithmic network width suffices for robust memorization.

New approach shows data memorization trade-offs in large models.

Study compares memorization of SimCLR to supervised and random labels training.

Generative diffusion models gradually memorize training data, losing independent dimensions.

State of the art deep generative networks are capable of producing images with such incredible realism that they can be suspected of memorizing training images. It is why it is not uncommon to include visualizations of training set nearest neighbors, to suggest generated images are not simply memorized. We demonstrate …

Deep networks preferentially learn shared features, avoiding memorization in early layers.

Deep networks can memorize random labels; symmetric loss improves this.

ResMem improves model generalization by explicitly memorizing residuals.

Diffusion models can memorize training data, limiting their creativity and privacy.

RAF model explains neural networks' dual rule learning and fact memorization.

Learning to solve sequential tasks with recurrent models requires the ability to memorize long sequences and to extract task-relevant features from them. In this paper, we study the memorization subtask from the point of view of the design and training of recurrent neural networks. We propose a new model, the Linear Me…

Gradient descent memorizes many Gaussians efficiently.

The bias potential model explains how generative models can generalize or memorize samples.