arXiv research

The paper explores the information-theoretic nature of excess risk in machine learning.

Paper improves risk bounds for nonconvex-strongly-concave minimax problems.

Paper proposes ZO-SMD for MERO, achieving optimal convergence rates.

Deep neural network with l_1-regularization achieves nearly optimal risk bounds.

Optimizes differentially private kernel learning with random projection.

The paper provides theoretical guarantees for neural network-based anomaly detection.

Paper provides optimal statistical guarantees for adversarial robustness in Gaussian classification.

Last SGD iterate bounds for overparameterized linear regression.

Paper tackles linear models with missing values, achieving minimax optimal results.

Study on kernel regression risk in high dimensions using Pinsker bound.

The paper analyzes how optimization algorithms affect the generalization of minimax models.

Paper introduces robust kernel ridge regression using Cauchy loss for handling various noise types.

New research shows that binary classification can be done with noisy data, but only if there are clean samples available.

Study on optimal rate of kernel regression for large-dimensional data.

Develops a robust learning method for unknown context distributions.

This paper analyzes neural network classifiers' performance in binary classification.

We consider a standard binary classification problem. The performance of any binary classifier based on the training data is characterized by the excess risk. We study Bahadur's type exponential bounds on the minimax accuracy confidence function based on the excess risk. We study how this quantity depends on the comple…

Unified framework for fair classification with group-blindness/awareness guarantees.

Two-layer ReLU networks outperform kernel methods in teacher-student settings.

SATL adapts to varying smoothness in hypothesis transfer learning.

We consider random-design linear prediction and related questions on the lower tail of random matrices. It is known that, under boundedness constraints, the minimax risk is of order $d/n$ in dimension $d$ with $n$ samples. Here, we study the minimax expected excess risk over the full linear class, depending on the dist…

Paper derives convergence rates for NPMLE in Hellinger distance using deep neural networks.

This paper investigates WDRO for nonparametric regression, achieving robustness against distributional uncertainty.

A central result in statistical theory is Pinsker's theorem, which characterizes the minimax rate in the normal means model of nonparametric estimation. In this paper, we present an extension to Pinsker's theorem where estimation is carried out under storage or communication constraints. In particular, we place limits …

The paper analyzes kernel classifiers' performance in Sobolev spaces and proves their optimality.

Gradient descent methods for deep ReLU networks achieve optimal generalization rates.

Paper analyzes sparse aggregation in GLMs with Kullback-Leibler risk bounds.

Upper bounds and lower bounds show ERM outperforms DG methods in various settings.

We formulate the notion of minimax estimation under storage or communication constraints, and prove an extension to Pinsker's theorem for nonparametric estimation over Sobolev ellipsoids. Placing limits on the number of bits used to encode any estimator, we give tight lower and upper bounds on the excess risk due to qu…

Paper analyzes minimax risks of personalized federated learning algorithms.

The paper analyzes risk bounds and Rademacher complexity in batch RL.

Improved DP SO with large Lipschitz parameters, handling outliers and heavy-tailed data.

We consider the binary classification problem in a setup that preserves the privacy of the original sample. We provide a privacy mechanism that is locally differentially private and then construct a classifier based on the private sample that is universally consistent in Euclidean spaces. Under stronger assumptions, we…

Introduces relative information gain for improving Gaussian process regression rates.

Paper analyzes risk bounds for in-context learning in multiclass classification.

Deep learning with noisy gradient descent outperforms linear estimators in high dimensions.

New framework analyzes deep learning optimization with finite width networks, revealing generalization gaps and excess risks.

Optimizes privacy-preserving optimization for heavy-tailed data.

Proposes RaT to mitigate bias in student-teacher estimation.

New method TLC improves transductive learning bounds.

We consider the problem of binary classification with abstention in the relatively less studied \emph{bounded-rate} setting. We begin by obtaining a characterization of the Bayes optimal classifier for an arbitrary input-label distribution $P_{XY}$. Our result generalizes and provides an alternative proof for the resul…

We present a novel notion of complexity that interpolates between and generalizes some classic existing complexity notions in learning theory: for estimators like empirical risk minimization (ERM) with arbitrary bounded losses, it is upper bounded in terms of data-independent Rademacher complexity; for generalized Baye…

GD outperforms ridge regression and SGD in linear regression problems.

Develops a deep learning framework for various data types.

The paper analyzes prediction error in nonstationary settings using weighted risk minimization.

This work characterizes the benefits of averaging schemes widely used in conjunction with stochastic gradient descent (SGD). In particular, this work provides a sharp analysis of: (1) mini-batching, a method of averaging many samples of a stochastic gradient to both reduce the variance of the stochastic gradient estima…

The study provides theoretical guarantees for the statistical performance of optimal decision trees.

Two-layer neural networks can overfit without increasing risk when data is noisy.