arXiv research

Study on size and depth of neural networks for approximating benign functions, showing barriers and explicit results.

New bounds for multi-task learning with varying task sizes.

A key problem in research on adversarial examples is that vulnerability to adversarial examples is usually measured by running attack algorithms. Because the attack algorithms are not optimal, the attack algorithms are prone to overestimating the size of perturbation needed to fool the target model. In other words, the…

Proves bounds on spanning two-forests and random cut sizes.

New bounds found for nodal sets on special manifolds.

Paper introduces new bounds linking data compressibility to generalization error.

We give a covering number bound for deep learning networks that is independent of the size of the network. The key for the simple analysis is that for linear classifiers, rotating the data doesn't affect the covering number. Thus, we can ignore the rotation part of each layer's linear transformation, and get the coveri…

Recently it has been shown that the step sizes of a family of variance reduced gradient methods called the JacSketch methods depend on the expected smoothness constant. In particular, if this expected smoothness constant could be calculated a priori, then one could safely set much larger step sizes which would result i…

This paper compares Transformers and RNNs in various tasks, showing size differences.

In Statistical Learning, the Vapnik-Chervonenkis (VC) dimension is an important combinatorial property of classifiers. To our knowledge, no theoretical results yet exist for the VC dimension of edited nearest-neighbour (1NN) classifiers with reference set of fixed size. Related theoretical results are scattered in the …

Novel bounds for SGLD show generalization error decreases with more samples.

The condensed nearest neighbor (CNN) algorithm is a heuristic for reducing the number of prototypical points stored by a nearest neighbor classifier, while keeping the classification rule given by the reduced prototypical set consistent with the full set. I present an upper bound on the number of prototypical points ac…

New theorem bounds group quotient size to subgroups index.

We construct near-optimal coresets for kernel density estimates for points in $\mathbb{R}^d$ when the kernel is positive definite. Specifically we show a polynomial time construction for a coreset of size $O(\sqrt{d}/\varepsilon\cdot \sqrt{\log 1/\varepsilon} )$, and we show a near-matching lower bound of size $Ω(\min\…

We propose a new stochastic first-order algorithmic framework to solve stochastic composite nonconvex optimization problems that covers both finite-sum and expectation settings. Our algorithms rely on the SARAH estimator introduced in (Nguyen et al, 2017) and consist of two steps: a proximal gradient and an averaging s…

We bound the size of $d$-dimensional cubulations of finitely presented groups. We apply this bound to obtain acylindrical accessibility for actions on CAT(0) cube complexes and bounds on curves on surfaces.

Paper proposes a cost-sensitive conformal training method with provably controllable learning bounds.

Reduces policy space complexity for reinforcement learning.

We consider the dynamics of a linear stochastic approximation algorithm driven by Markovian noise, and derive finite-time bounds on the moments of the error, i.e., deviation of the output of the algorithm from the equilibrium point of an associated ordinary differential equation (ODE). We obtain finite-time bounds on t…

This paper explores adversarial training limits and improves model robustness against norm-bounded perturbations.

This article studies the achievable guarantees on the error rates of certain learning algorithms, with particular focus on refining logarithmic factors. Many of the results are based on a general technique for obtaining bounds on the error rates of sample-consistent classifiers with monotonic error regions, in the real…

The paper improves methods for estimating set size using samples.

Paper improves CMAB regret bounds by reducing batch-size dependency.

Paper classifies totally symmetric sets in groups and bounds their sizes.

The paper improves regret bounds for admission control in queueing systems.

We study the sample complexity of learning neural networks, by providing new bounds on their Rademacher complexity assuming norm constraints on the parameter matrix of each layer. Compared to previous work, these complexity bounds have improved dependence on the network depth, and under some additional assumptions, are…

New Sauer inequality improves multiclass hypothesis class bounds.

We obtain the first positive results for bounded sample compression in the agnostic regression setting with the $\ell_p$ loss, where $p\in [1,\infty]$. We construct a generic approximate sample compression scheme for real-valued function classes exhibiting exponential size in the fat-shattering dimension but independen…

Vector-valued learning, where the output space admits a vector-valued structure, is an important problem that covers a broad family of important domains, e.g. multi-task learning and transfer learning. Using local Rademacher complexity and unlabeled data, we derive novel semi-supervised excess risk bounds for general v…

New algorithms verify and search causal graphs with minimal interventions.

We prove a natural inequality which implies the known lower bounds for the $(n-1)$-dimensional Hausdorff measure of nodal sets for smooth compact manifolds.

ProxSPS improves on SPS for regularization tasks, offering better stability and performance.

There has been renewed recent interest in developing effective lower bounds for Dynamic Time Warping (DTW) distance between time series. These have many applications in time series indexing, clustering, forecasting, regression and classification. One of the key time series classification algorithms, the nearest neighbo…

We provide non-asymptotic bounds for the well-known temporal difference learning algorithm TD(0) with linear function approximators. These include high-probability bounds as well as bounds in expectation. Our analysis suggests that a step-size inversely proportional to the number of iterations cannot guarantee optimal …

The paper shows that relaxing assumptions about causal graphs can lead to exponentially large equivalence classes.

We consider the problem of providing nonparametric confidence guarantees for undirected graphs under weak assumptions. In particular, we do not assume sparsity, incoherence or Normality. We allow the dimension $D$ to increase with the sample size $n$. First, we prove lower bounds that show that if we want accurate infe…

Study on kernel regression risk in high dimensions using Pinsker bound.

New analysis for black-box learning without gradients, improving generalization bounds.

New bounds on efficiency for conformalized regression methods.

The classical $k$-means algorithm for partitioning $n$ points in $\mathbb{R}^d$ into $k$ clusters is one of the most popular and widely spread clustering methods. The need to respect prescribed lower bounds on the cluster sizes has been observed in many scientific and business applications. In this paper, we present an…

This study tightens bounds on how GD and SGD generalize in smooth convex optimization problems.

The study examines how equivariance in networks affects generalization error using PAC-Bayesian bounds.

New study shows ERMs can fail in convex optimization with high dimensionality.

The paper analyzes the expected size of conformal prediction sets.

The paper studies batch decompositions of random datasets with probabilistic similarity constraints.

Bounds neural network output distribution to Gaussian for random initialization.

Study on U-statistics with heavy-tailed samples, providing tail bounds and LDP.

Monotone neural networks can approximate and interpolate functions efficiently.