arXiv research

The paper generalizes offset Rademacher complexities to convex and non-convex problems.

New bounds for non-convex estimators without Bernstein condition.

We consider regression with square loss and general classes of functions without the boundedness assumption. We introduce a notion of offset Rademacher complexity that provides a transparent way to study localization both in expectation and in high probability. For any (possibly non-convex) class, the excess loss of a …

The paper introduces gapped scale-sensitive dimensions to improve learning rate bounds.

Study mirrors descent's early stopping for linear and kernel models, improving risk guarantees.

Extends inequality for Rademacher complexities using $p$-stable variables.

The paper bounds the complexity of GCNs using Rademacher complexity.

Improved generalization bounds for CNNs using Rademacher complexity.

We show that the Rademacher complexity of any $\mathbb{R}^{K}$-valued function class composed with an $\ell_{\infty}$-Lipschitz function is bounded by the maximum Rademacher complexity of the restriction of the function class along each coordinate, times a factor of $\tilde{O}(\sqrt{K})$.

Analyzes the complexity of linear hypothesis sets using Rademacher complexity.

We develop a technique for deriving data-dependent error bounds for transductive learning algorithms based on transductive Rademacher complexity. Our technique is based on a novel general error bound for transduction in terms of transductive Rademacher complexity, together with a novel bounding technique for Rademacher…

The paper analyzes risk bounds and Rademacher complexity in batch RL.

New findings show Rademacher complexities are not crucial for learning complexities.

The paper analyzes adversarial robustness for linear models and neural networks using Rademacher complexity.

Great successes of deep neural networks have been witnessed in various real applications. Many algorithmic and implementation techniques have been developed, however, theoretical understanding of many aspects of deep neural networks is far from clear. A particular interesting issue is the usefulness of dropout, which w…

We show how to control the generalization error of time series models wherein past values of the outcome are used to predict future values. The results are based on a generalization of standard i.i.d. concentration inequalities to dependent data without the mixing assumptions common in the time series setting. Our proo…

This paper provides a general result on controlling local Rademacher complexities, which captures in an elegant form to relate the complexities with constraint on the expected norm to the corresponding ones with constraint on the empirical norm. This result is convenient to apply in real applications and could yield re…

We study an equivalence of (i) deterministic pathwise statements appearing in the online learning literature (termed \emph{regret bounds}), (ii) high-probability tail bounds for the supremum of a collection of martingales (of a specific form arising from uniform laws of large numbers for martingales), and (iii) in-expe…

For a finite function class we describe the large sample limit of the sequential Rademacher complexity in terms of the viscosity solution of a $G$-heat equation. In the language of Peng's sublinear expectation theory, the same quantity equals to the expected value of the largest order statistics of a multidimensional $…

The developments of Rademacher complexity and PAC-Bayesian theory have been largely independent. One exception is the PAC-Bayes theorem of Kakade, Sridharan, and Tewari (2008), which is established via Rademacher complexity theory by viewing Gibbs classifiers as linear operators. The goal of this paper is to extend thi…

Transductive learning considers situations when a learner observes $m$ labelled training points and $u$ unlabelled test points with the final goal of giving correct answers for the test points. This paper introduces a new complexity measure for transductive learning called Permutational Rademacher Complexity (PRC) and …

Discussion of ``2004 IMS Medallion Lecture: Local Rademacher complexities and oracle inequalities in risk minimization'' by V. Koltchinskii [arXiv:0708.0083]

Discussion of ``2004 IMS Medallion Lecture: Local Rademacher complexities and oracle inequalities in risk minimization'' by V. Koltchinskii [arXiv:0708.0083]

Discussion of ``2004 IMS Medallion Lecture: Local Rademacher complexities and oracle inequalities in risk minimization'' by V. Koltchinskii [arXiv:0708.0083]

Discussion of ``2004 IMS Medallion Lecture: Local Rademacher complexities and oracle inequalities in risk minimization'' by V. Koltchinskii [arXiv:0708.0083]

Discussion of "2004 IMS Medallion Lecture: Local Rademacher complexities and oracle inequalities in risk minimization" by V. Koltchinskii [arXiv:0708.0083]

Statistical learning theory provides bounds of the generalization gap, using in particular the Vapnik-Chervonenkis dimension and the Rademacher complexity. An alternative approach, mainly studied in the statistical physics literature, is the study of generalization in simple synthetic-data models. Here we discuss the c…

Logistic regression gets a new, simpler uniform bound.

The contraction inequality for Rademacher averages is extended to Lipschitz functions with vector-valued domains, and it is also shown that in the bounding expression the Rademacher variables can be replaced by arbitrary iid symmetric and sub-gaussian variables. Example applications are given for multi-category learnin…

We present a novel notion of complexity that interpolates between and generalizes some classic existing complexity notions in learning theory: for estimators like empirical risk minimization (ERM) with arbitrary bounded losses, it is upper bounded in terms of data-independent Rademacher complexity; for generalized Baye…

We present a study of generalization for data-dependent hypothesis sets. We give a general learning guarantee for data-dependent hypothesis sets based on a notion of transductive Rademacher complexity. Our main result is a generalization bound for data-dependent hypothesis sets expressed in terms of a notion of hypothe…

Many machine learning models are vulnerable to adversarial attacks; for example, adding adversarial perturbations that are imperceptible to humans can often make machine learning models produce wrong predictions with high confidence. Moreover, although we may obtain robust models on the training dataset via adversarial…

Quantum reservoirs risk bounds are analyzed using Rademacher complexity.

Paper introduces a new method for classifying interval-valued time series.

Majorizing measures control sequential complexities for online learning.

Regularization of Deep Neural Networks (DNNs) for the sake of improving their generalization capability is important and challenging. The development in this line benefits theoretical foundation of DNNs and promotes their usability in different areas of artificial intelligence. In this paper, we investigate the role of…

LocalDrop uses local Rademacher complexity for neural network regularization.

Estimates neural network errors for classification problems.

In this paper, we define dual geodesic trihedron(dual Darboux frame) of a spacelike ruled surface. Then, we study Mannheim offsets of spacelike ruled surfaces in dual Lorentzian space by considering the E. Study Mapping. We represent spacelike ruled surfaces by dual Lorentzian unit spherical curves and define Mannheim …

New bounds explain modern machine learning algorithms' generalization.

Neural ODEs simplified using Chen-Fliess series for Rademacher complexity analysis.

In this paper, we study Mannheim surface offsets in dual space. By the aid of the E. Study Mapping, we consider ruled surfaces as dual unit spherical curves and define the Mannheim offsets of the ruled surfaces by means of dual geodesic trihedron (dual Darboux frame). We obtain the relationships between the invariants …

In this paper, using the classifications of timelike and spacelike ruled surfaces, we study the Mannheim offsets of timelike ruled surfaces in Minkowski 3-space. Firstly, we define the Mannheim offsets of a timelike ruled surface by considering the Lorentzian casual character of the offset surface. We obtain that the M…

New bound explains why high-rank neural nets generalize well.

Proves new concentration inequalities for sub-gaussian and sub-exponential variables.

Study excess capacity in neural networks using Rademacher complexity.

In this study, we give the dual characterizations of Mannheim offsets of the ruled surface in terms of their integral invariants and the new characterization of the Mannheim offsets of developable surface. Furthermore, we obtain the relationships between the area of projections of spherical images for Mannheim offsets …

Improved sample complexity for ReLU networks with norm constraints.