One of the earliest conjectures in computational learning theory-the Sample Compression conjecture-asserts that concept classes (equivalently set systems) admit compression schemes of size linear in their VC dimension. To-date this statement is known to be true for maximum classes---those that possess maximum cardinali…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “General VC Classes”
New neural network class reduces VC dimension, leading to better generalization.
problem VC theory struggles with explaining small generalization errors in overparametrized neural networks.
method Developed hyperplane arrangement neural networks (HANNs) and used sample compression analysis.
result HANNs can have significantly smaller VC dimension than the number of weights, yet remain highly expressive.
VC dimension of ellipsoidsmath.CO
We will establish that the VC dimension of the class of d-dimensional ellipsoids is (d^2+3d)/2, and that maximum likelihood estimate with N-component d-dimensional Gaussian mixture models induces a geometric class having VC dimension at least N(d^2+3d)/2. Keywords: VC dimension; finite dimensional ellipsoid; Gaussian m…
Unified derivation of PAC-Bayes and MI bounds for general VC classes with fast rates.
problem Generalization bounds for machine learning models with VC classes.
method Unified derivation of conditional PAC-Bayesian and mutual information bounds, including MAC-Bayesian bounds.
result Nontrivial bounds for general VC classes and faster rates for specific conditions.
We show that the sets in a family with finite VC dimension can be uniformly approximated within a given error by a finite partition. Immediate corollaries include the fact that VC classes have finite bracketing numbers, satisfy uniform laws of averages under strong dependence, and exhibit uniform mixing. Our results ar…
New bounds on learning from multiple distributions for VC classes.
problem Understanding the sample complexity of learning from multiple data distributions.
method Analyzing the gap between known upper and lower bounds for PAC-learnable classes.
result Recent progress on sample complexity for VC dimension d classes on k distributions.
Contradiction graphs reveal VC dimension threshold.
problem Determining VC dimension of concept classes.
method Study contradiction graphs of binary concept classes.
result Single contradiction graph determines VC dimension.
We study the question of learning an adversarially robust predictor. We show that any hypothesis class with finite VC dimension is robustly PAC learnable with an improper learning rule. The requirement of being improper is necessary as we exhibit examples of hypothesis classes with finite VC…
We consider solutions to the complex Trkalian equation,~$ \vec{\nabla} \times \vc = \vc ,$ where~$\vc$ is a 3 component vector function with each component in the complex field, and may be expressed in the form~$ \vc = e^{ig} \vec{\nabla} F, $ with~ real and~ complex. We find, there are precisely two classes of s…
Study on proper learning under relaxed worst-case robust loss for VC classes.
problem Proper adversarially robust PAC learning under relaxed worst-case robust loss.
method Introduced a family of robust loss relaxations and showed their effectiveness for proper learnability.
result VC classes are properly PAC learnable with sample complexity close to standard PAC learning setup.
The existence of evasion attacks during the test phase of machine learning algorithms represents a significant challenge to both their deployment and understanding. These attacks can be carried out by adding imperceptible perturbations to inputs to generate adversarial examples and finding effective defenses and detect…
Learnable multiclass hypothesis classes don't always have a sample compression scheme of fixed size.
problem The limitation of sample compression schemes for multiclass hypothesis classes.
method Analysis of DS dimension and sample compression schemes.
result Learnable multiclass hypothesis classes do not always have a sample compression scheme of fixed size.
Algorithm learns from both labeled and arbitrary test examples, giving guarantees for bounded VC dimension classes.
problem Learning from arbitrary test examples, not just perturbations.
method Selective transductive learning algorithm that outputs abstaining predictions.
result Nontrivial guarantees for bounded VC dimension classes with arbitrary train and test distributions.
Reduces multiclass and regression compression schemes to binary ones.
problem Developing efficient learning algorithms for multiclass and regression problems.
method Reduces sample compression schemes for binary classes to multiclass and regression settings.
result Establishes new compression schemes for multiclass and regression problems.
For any family of measurable sets in a probability space, we show that either (i) the family has infinite Vapnik-Chervonenkis (VC) dimension or (ii) for every epsilon > 0 there is a finite partition pi such the pi-boundary of each set has measure at most epsilon. Immediate corollaries include the fact that a family wit…
The study analyzes decision trees on real and categorical features, deriving bounds on their VC dimension and proposing improved pruning algorithms.
problem Understanding the generalization properties of decision trees on different types of features.
method Introducing partitioning functions, relating them to growth functions and VC dimension, and deriving bounds for decision stumps and trees of various structures.
result Exact VC dimension of decision stumps and improved pruning algorithms for binary trees.
Improved multi-group learning with group-realizable concepts.
problem Enhancing multi-group learning efficiency.
method Empirical risk minimization over group-realizable concepts.
result Improved sample complexity in group-realizable settings.
Characterizes distribution-free rates in unbalanced classification problems.
problem Minimizing error under two different distributions in unbalanced settings.
method Characterizes minimax rates over all pairs of distributions using a geometric condition.
result Identifies a dichotomy between hard and easy classes based on a three-points-separation condition.
We explore in some detail the notion of algorithmic stability as a viable framework for analyzing the generalization error of learning algorithms. We introduce the new notion of training stability of a learning algorithm and show that, in a general setting, it is sufficient for good bounds on generalization error. In t…
This paper explains double descent using VC theory.
problem Understanding the generalization of overfitting neural networks.
method VC-theoretical analysis of double descent.
result Double descent can be explained by classical VC-generalization bounds.
In response to a 1997 problem of M. Vidyasagar, we state a criterion for PAC learnability of a concept class under the family of all non-atomic (diffuse) measures on the domain . The uniform Glivenko--Cantelli property with respect to non-atomic measures is no longer a necessary condition, and consisten…
VC dimensions of group CNNs are infinite for certain kernels and groups.
problem Estimating the generalization capacity of group convolutional neural networks.
method Identifying precise VC dimension estimates for simple sets of group CNNs.
result Two-parameter families of convolutional neural networks have an infinite VC dimension for infinite groups and certain kernels.
Study on VC dimension of GCNNs with input resolution effects.
problem Understanding the generalization capabilities of GCNNs.
method Derived upper and lower bounds for VC dimension, analyzed factors affecting it.
result Extended previous results on VC dimension of GCNNs, providing insights into input resolution dependence.
Comparative learning combines realizable and agnostic settings for two hypothesis classes, reducing sample complexity.
problem Learning with two hypothesis classes in a more general setting than single hypothesis classes.
method Introduces comparative learning, defines mutual VC dimension and Littlestone dimension, and applies insights to multiaccuracy and multicalibration.
result Sample complexity of comparative learning is characterized by mutual VC dimension and Littlestone dimension.
Adversarial robust learning improved for transductive setting.
problem Adversarial robust learning in transductive setting.
method Simple transductive learner for bounded VC dimension classes.
result Robust error rate linear in VC dimension, adaptive to perturbation complexity.
New findings on null measurability in symmetrization interface of VC learning.
problem Null measurability issues in symmetrization interface of VC learning.
method Formalized in Lean 4, using Choquet capacitability and patching properties.
result Null-measurable bad event not Borel measurable, separating regularity levels.
Boosting combines weak hypotheses to create accurate predictions under bounded VC dimension.
problem How to combine weak hypotheses to achieve accurate predictions efficiently.
method Designing a novel boosting algorithm with complex aggregation rules for bounded VC dimension classes.
result The new boosting algorithm requires fewer weak hypotheses than classical lower bounds, provided they belong to a bounded VC class.
Study private query release with public data, reducing sample sizes.
problem Answering a wide range of statistical queries while maintaining privacy.
method Combines public and private samples to answer queries with differential privacy.
result Private and public sample complexities for different query classes.
Algorithm learns without knowing distribution, reducing error.
problem Sequential prediction with adversarial injections and abstentions.
method Boosting procedure of weak learners for general VC classes.
result Sublinear error guarantees for general VC classes.
New method reduces sample complexity for robust learning.
problem Developing simple, sample-efficient learning algorithms for robust classification.
method Tolerant empirical risk minimization (RERM) for robust learning.
result Tolerant RERM requires only O(VC(H)dlog(D/γδ)/ε^2) samples for robustness regions of diameter D.
FastVoiceGrad speeds up VC to one step, matching or surpassing quality.
problem Slow inference in multi-step diffusion-based VC.
method Adversarial Conditional Diffusion Distillation (ACDD) for one-step diffusion.
result One-shot VC with superior or comparable performance to multi-step methods.
Paper combines RL with policy regularization for inventory policies.
problem Optimizing inventory policies using RL and dynamic programming.
method Hybrid approach combining RL with policy regularization.
result Generalization guarantees for inventory policies using VC theory.
Vapnik-Chervonenkis (VC) dimension is a fundamental measure of the generalization capacity of learning algorithms. However, apart from a few special cases, it is hard or impossible to calculate analytically. Vapnik et al. [10] proposed a technique for estimating the VC dimension empirically. While their approach behave…
New bounds explain modern machine learning algorithms' generalization.
problem Explaining generalization behavior of modern machine learning algorithms.
method Proposes a new complexity measure based on empirical Rademacher complexity of an algorithm- and data-dependent hypothesis class.
result Obtains novel bounds with finite fractal dimension, simplifies proofs, and recovers known results.
Although voice conversion (VC) algorithms have achieved remarkable success along with the development of machine learning, superior performance is still difficult to achieve when using nonparallel data. In this paper, we propose using a cycle-consistent adversarial network (CycleGAN) for nonparallel data-based VC train…
WL meet VCcs.LG
The paper connects GNNs to VC dimension theory to study their generalization performance.
problem Understanding GNNs' ability to make meaningful predictions beyond the training set.
method Using Vapnik-Chervonenkis (VC) dimension theory in two settings: no upper bound on graph order and known upper bound.
result Tight connections between GNNs' bitlength, number of colors, and VC dimension in different settings.
A new model for sequential prediction handles adversarial examples by allowing abstention.
problem Sequential prediction algorithms fail with adversarial examples, leading to incorrect predictions.
method Proposes a new model that allows abstention from predictions on adversarial examples, scaling error with VC dimension.
result A learner's error scales with the VC dimension of the hypothesis class, matching the stochastic setting.
Develops higher arity VC theory and characterizes PAC learning in product spaces.
problem Characterizing PAC learning in multi-dimensional product spaces.
method Introduces higher arity VC dimension, generalizes Haussler packing lemma, and develops hypergraph regularity lemma.
result Characterizes higher arity PAC learning in n-fold product spaces.
Improved private agnostic learning with near-optimal sample complexity.
problem Private agnostic learning with arbitrary privacy parameters.
method Near-optimal sample complexity construction.
result Near-optimal extra sample complexity of \(\widetilde{O}(\mathrm{VC}(\mathcal{C})/α^2)\) for any \(\varepsilon \leq 1\).
A theory for approximating complex concepts with simple decision trees.
problem Approximating complex concepts with simple decision trees.
method Introducing interpretable approximations, studying binary concept approximation by decision trees.
result A trichotomy of cases for approximating a binary concept by decision trees based on a simple class.
Every classical knot is band-pass equivalent to the unknot or the trefoil. The band-pass class of a knot is a concordance invariant. Every ribbon knot, for example, is band-pass equivalent to the unknot. Here we introduce the long virtual knot concordance group . It is shown that for every concordance cla…
Simulators enable learning with generalization guarantees in computationally bounded worlds.
problem Generalization in learning theory
method Simulatable processes
result Recovery of PAC learning guarantees with VC dimension
The paper explores learning from label proportions, showing differences in efficiency between LLP and PAC learning.
problem Learning from label proportions (LLP) in unlabeled data with given label proportions.
method Formal definition and computational complexity analysis of LLP learning.
result LLP learning is more restrictive than PAC learning for finite VC classes, and some classes are uncharacterizable.
Paper generalizes strategic classification framework and introduces SVC for PAC-learning.
problem Strategic manipulation of testing data to fool classifiers.
method Unified framework for strategic classification, strategic VC-dimension (SVC).
result Characterizes the learnability and computational tractability of linear classifiers.
Deep Heaviside networks are limited but can be improved with connections or linear neurons.
problem Limited expressivity of deep Heaviside networks.
method Including skip connections or linear activation neurons improves expressivity.
result Lower and upper bounds for VC dimensions and approximation rates are derived.
New insights into learning from distributional adversaries and private data.
problem Understanding minimal assumptions for learning and generalization under distributional constraints.
method Generalized smoothness as a characterization of learnability and privacy under distributional adversaries.
result Near complete characterization of families that admit learnability and privacy under distributional adversaries.
AttS2S-VC: Sequence-to-Sequence Voice Conversion with Attention and Context Preservation Mechanismseess.AS
This paper describes a method based on a sequence-to-sequence learning (Seq2Seq) with attention and context preservation mechanism for voice conversion (VC) tasks. Seq2Seq has been outstanding at numerous tasks involving sequence modeling such as speech synthesis and recognition, machine translation, and image captioni…
The paper explores how to reduce classification tasks to optimization problems in Euclidean space.
problem Understanding the minimum dimension needed for reducing classification tasks to optimization problems.
method Developed a generalization of the Borsuk-Ulam Theorem to analyze the expressivity of reductions.
result The minimum Euclidean dimension required can be exponentially larger than the VC dimension, even for slightly non-trivial reductions.