arXiv research

Bounds on VC dimension for 1NN classifiers with fixed prototype sets.

Vapnik-Chervonenkis (VC) dimension is a fundamental measure of the generalization capacity of learning algorithms. However, apart from a few special cases, it is hard or impossible to calculate analytically. Vapnik et al. [10] proposed a technique for estimating the VC dimension empirically. While their approach behave…

Study on VC dimension of GCNNs with input resolution effects.

This paper explains double descent using VC theory.

Unified derivation of PAC-Bayes and MI bounds for general VC classes with fast rates.

The paper connects GNNs to VC dimension theory to study their generalization performance.

New neural network class reduces VC dimension, leading to better generalization.

New bounds on learning from multiple distributions for VC classes.

One of the earliest conjectures in computational learning theory-the Sample Compression conjecture-asserts that concept classes (equivalently set systems) admit compression schemes of size linear in their VC dimension. To-date this statement is known to be true for maximum classes---those that possess maximum cardinali…

New robust estimators achieve subgaussian bounds using VC-dimension.

Lower bounds set for infinite-precision transformers.

Adversarial robust learning improved for transductive setting.

The VC-dimension of a set system is a way to capture its complexity and has been a key parameter studied extensively in machine learning and geometry communities. In this paper, we resolve two longstanding open problems on bounding the VC-dimension of two fundamental set systems: $k$-fold unions/intersections of half-s…

Derives VC bounds for deep neural networks using radius-margin approach.

Study extends GNN VC dimension bounds to Pfaffian activation functions.

Paper generalizes strategic classification framework and introduces SVC for PAC-learning.

Algorithm learns from both labeled and arbitrary test examples, giving guarantees for bounded VC dimension classes.

Boosting combines weak hypotheses to create accurate predictions under bounded VC dimension.

New complexity measure ADL connects to classical complexity measures.

The study analyzes decision trees on real and categorical features, deriving bounds on their VC dimension and proposing improved pruning algorithms.

Study private query release with public data, reducing sample sizes.

Lower bounds on Bayes risk for realizable models derived using information theory.

We explore in some detail the notion of algorithmic stability as a viable framework for analyzing the generalization error of learning algorithms. We introduce the new notion of training stability of a learning algorithm and show that, in a general setting, it is sufficient for good bounds on generalization error. In t…

New algorithms achieve better regret bounds for online classification with relaxed benchmarks.

The recently proposed Minimal Complexity Machine (MCM) finds a hyperplane classifier by minimizing an exact bound on the Vapnik-Chervonenkis (VC) dimension. The VC dimension measures the capacity of a learning machine, and a smaller VC dimension leads to improved generalization. On many benchmark datasets, the MCM gene…

The existence of evasion attacks during the test phase of machine learning algorithms represents a significant challenge to both their deployment and understanding. These attacks can be carried out by adding imperceptible perturbations to inputs to generate adversarial examples and finding effective defenses and detect…

New method reduces sample complexity for robust learning.

Simulators enable learning with generalization guarantees in computationally bounded worlds.

New framework ties learning algorithms to data recognition using CMI.

Learnable multiclass hypothesis classes don't always have a sample compression scheme of fixed size.

VC dimensions of group CNNs are infinite for certain kernels and groups.

Paper extends learning theory to dependent data with uniform risk bounds.

FastVoiceGrad speeds up VC to one step, matching or surpassing quality.

Study robust learning without knowing perturbation sets, using interactions with attackers.

Deep Heaviside networks are limited but can be improved with connections or linear neurons.

We will establish that the VC dimension of the class of d-dimensional ellipsoids is (d^2+3d)/2, and that maximum likelihood estimate with N-component d-dimensional Gaussian mixture models induces a geometric class having VC dimension at least N(d^2+3d)/2. Keywords: VC dimension; finite dimensional ellipsoid; Gaussian m…

New research limits what GNNs can compute and generalizes their performance.

Reduces multiclass and regression compression schemes to binary ones.

PAC-Bayes framework fails on simple 1D linear classification task.

VC classes can be robustly learned but only by misusing the learning method.

The paper develops bounds for predictive values in binary classification.

New Sauer inequality improves multiclass hypothesis class bounds.

We show that the sets in a family with finite VC dimension can be uniformly approximated within a given error by a finite partition. Immediate corollaries include the fact that VC classes have finite bracketing numbers, satisfy uniform laws of averages under strong dependence, and exhibit uniform mixing. Our results ar…

Although voice conversion (VC) algorithms have achieved remarkable success along with the development of machine learning, superior performance is still difficult to achieve when using nonparallel data. In this paper, we propose using a cycle-consistent adversarial network (CycleGAN) for nonparallel data-based VC train…

New bounds explain modern machine learning algorithms' generalization.

Optimal ReLU networks can memorize any separable set of points with a small number of parameters.

Contradiction graphs reveal VC dimension threshold.

Unified framework for proving generalization bounds in machine learning.