arXiv research

The study examines when MAML's objective has a benign landscape.

Paper analyzes Transformer learning dynamics, proving benign landscape for in-context learning.

Wide deep neural networks are easy to optimize without constraints.

The paper analyzes phase retrieval under limited samples, ensuring a benign local landscape for convergence.

New findings on optimization landscape of Toeplitz covariance estimation.

We study nonconvex optimization landscapes for learning overcomplete representations, including learning (i) sparsely used overcomplete dictionaries and (ii) convolutional dictionaries, where these unsupervised learning problems find many applications in high-dimensional data analysis. Despite the empirical success of …

This paper introduces SRPR for robust phase retrieval with smoothed loss functions.

Deeper models have a more favorable optimization landscape, making them more robust to noise.

Optimizes MMD learning for generative models with theoretical guarantees.

This work analyzes a two-stage algorithm for single index models, showing precise asymptotics of gradient descent.

New theory shows predictive coding makes learning landscape easier to navigate.

WSFN overcomes saddle points for non-convex functionals in Wasserstein space.

Improved optimization guarantees for deep learning models with Nesterov acceleration.

We consider Fair Principal Component Analysis (FPCA) and search for a low dimensional subspace that spans multiple target vectors in a fair manner. FPCA is defined as a non-concave maximization of the worst projected target norm within a given set. The problem arises in filter design in signal processing, and when inco…

We examine the squared error loss landscape of shallow linear neural networks. We show---with significantly milder assumptions than previous works---that the corresponding optimization problems have benign geometric properties: there are no spurious local minima and the Hessian at every saddle point has at least one ne…

Neural collapse occurs in normalized features over a Riemannian manifold.

Algorithm aligns 3D density maps using Wasserstein distance.

Sharp condition found for Burer-Monteiro method to work for MaxCut-type SDPs.

Study on benign overfitting in leaky ReLUs with moderate input dimensions.

BOinG optimizes HPO problems by focusing on promising local regions.

We provide new theoretical insights on why over-parametrization is effective in learning neural networks. For a $k$ hidden node shallow network with quadratic activation and $n$ training data points, we show as long as $k \ge \sqrt{2n}$, over-parametrization enables local search algorithms to find a \emph{globally} op…

Recent results in the literature indicate that a residual network (ResNet) composed of a single residual block outperforms linear predictors, in the sense that all local minima in its optimization landscape are at least as good as the best linear predictor. However, these results are limited to a single residual block …

New framework analyzes effectiveness of neural network-based combinatorial problem solvers.

Attention models can overfit without harming test performance.

ResNet models overfit benignly on Cifar10 but not on ImageNet due to label noise.

Paper finds conditions for benign overfitting in neural networks.

New insights into when benign overfitting occurs in linear and classification tasks.

New insights into why neural networks can overfit without interpolating data.

Study on size and depth of neural networks for approximating benign functions, showing barriers and explicit results.

Two-layer CNNs can overfit without harm under certain conditions.

We analyze neural collapse in neural networks, showing that features collapse to vertices of a Simplex ETF.

Two-layer ReLU networks can overfit without harm, study finds.

New framework solves dynamic bilevel optimization problems in reinforcement learning.

Meta learning works well with overparameterized models, a phenomenon called 'benign overfitting'.

Deep neural networks can generalize well even with perfect fits to noisy data.

The paper explores how benign overfitting occurs in heavy-tailed input distributions.

New analysis shows bias term affects conditions for benign overfitting in linear classifiers.

The paper explores how noise in features can lead to benign overfitting in machine learning models.

ELS framework improves safety alignment by dynamically steering LLMs towards helpful responses.

Adversarial training can lead to overfitting without compromising robustness.

Neural networks can interpolate noisy data and still generalize well.

New neural networks learn single-index models efficiently.

Unified framework explains why overfitting is benign in interpolating learning.

Nonnegative low-rank matrix recovery can have spurious local minima.

Study reveals learning curves and benign overfitting in spectral algorithms for large dimensions.

Linear models can overfit without harming OOD generalization under certain conditions.

New study finds many neural networks are not benignly overfitting.

ABO extends RLS for online learning in non-stationary time-series, improving accuracy and speed.