arXiv research

Paper generalizes strategic classification framework and introduces SVC for PAC-learning.

VC dimensions of group CNNs are infinite for certain kernels and groups.

We will establish that the VC dimension of the class of d-dimensional ellipsoids is (d^2+3d)/2, and that maximum likelihood estimate with N-component d-dimensional Gaussian mixture models induces a geometric class having VC dimension at least N(d^2+3d)/2. Keywords: VC dimension; finite dimensional ellipsoid; Gaussian m…

Study on VC dimension of GCNNs with input resolution effects.

Contradiction graphs reveal VC dimension threshold.

Vapnik-Chervonenkis (VC) dimension is a fundamental measure of the generalization capacity of learning algorithms. However, apart from a few special cases, it is hard or impossible to calculate analytically. Vapnik et al. [10] proposed a technique for estimating the VC dimension empirically. While their approach behave…

Improved multi-group learning with group-realizable concepts.

Investigates the impact of finite VC dimension on neural network approximation and learning.

The existence of evasion attacks during the test phase of machine learning algorithms represents a significant challenge to both their deployment and understanding. These attacks can be carried out by adding imperceptible perturbations to inputs to generate adversarial examples and finding effective defenses and detect…

Adversarial robust learning improved for transductive setting.

In Statistical Learning, the Vapnik-Chervonenkis (VC) dimension is an important combinatorial property of classifiers. To our knowledge, no theoretical results yet exist for the VC dimension of edited nearest-neighbour (1NN) classifiers with reference set of fixed size. Related theoretical results are scattered in the …

The paper connects GNNs to VC dimension theory to study their generalization performance.

New neural network class reduces VC dimension, leading to better generalization.

The VC-dimension of a set system is a way to capture its complexity and has been a key parameter studied extensively in machine learning and geometry communities. In this paper, we resolve two longstanding open problems on bounding the VC-dimension of two fundamental set systems: $k$-fold unions/intersections of half-s…

The recently proposed Minimal Complexity Machine (MCM) finds a hyperplane classifier by minimizing an exact bound on the Vapnik-Chervonenkis (VC) dimension. The VC dimension measures the capacity of a learning machine, and a smaller VC dimension leads to improved generalization. On many benchmark datasets, the MCM gene…

Study extends GNN VC dimension bounds to Pfaffian activation functions.

New robust estimators achieve subgaussian bounds using VC-dimension.

Learnable multiclass hypothesis classes don't always have a sample compression scheme of fixed size.

The Vapnik-Chervonenkis (VC) dimension of a collection of subsets of a set is an important combinatorial concept in settings such as discrete geometry and machine learning. In this paper we prove that the VC dimension of the family of $d$-dimensional cubes in $\mathbb R^d$ is $\lfloor(3d+1)/2\rfloor$.

The study analyzes decision trees on real and categorical features, deriving bounds on their VC dimension and proposing improved pruning algorithms.

New complexity measure ADL connects to classical complexity measures.

We study the question of learning an adversarially robust predictor. We show that any hypothesis class $\mathcal{H}$ with finite VC dimension is robustly PAC learnable with an improper learning rule. The requirement of being improper is necessary as we exhibit examples of hypothesis classes $\mathcal{H}$ with finite VC…

Lower bounds set for infinite-precision transformers.

Algorithm learns from both labeled and arbitrary test examples, giving guarantees for bounded VC dimension classes.

Adversarial attacks during the testing phase of neural networks pose a challenge for the deployment of neural networks in security critical settings. These attacks can be performed by adding noise that is imperceptible to humans on top of the original data. By doing so, an attacker can create an adversarial sample, whi…

Study private query release with public data, reducing sample sizes.

New framework for robust uncertainty quantification in strategic settings.

Novel framework for teaching complexity in machine teaching models.

In response to a 1997 problem of M. Vidyasagar, we state a criterion for PAC learnability of a concept class $\mathscr C$ under the family of all non-atomic (diffuse) measures on the domain $Ω$. The uniform Glivenko--Cantelli property with respect to non-atomic measures is no longer a necessary condition, and consisten…

Deep Heaviside networks are limited but can be improved with connections or linear neurons.

Study strategic dynamic pricing for buyers with unknown manipulation costs.

Study on proper learning under relaxed worst-case robust loss for VC classes.

For any family of measurable sets in a probability space, we show that either (i) the family has infinite Vapnik-Chervonenkis (VC) dimension or (ii) for every epsilon > 0 there is a finite partition pi such the pi-boundary of each set has measure at most epsilon. Immediate corollaries include the fact that a family wit…

New framework reduces strategic manipulation cost for minority groups in fair classification.

Study compares employers with and without anticipating strategic labor force responses.

One of the earliest conjectures in computational learning theory-the Sample Compression conjecture-asserts that concept classes (equivalently set systems) admit compression schemes of size linear in their VC dimension. To-date this statement is known to be true for maximum classes---those that possess maximum cardinali…

New algorithm learns optimal policies in strategic MDPs with private types.

Deep neural network (DNN) has demonstrated its success in multiple domains. However, DNN models are inherently vulnerable to adversarial examples, which are generated by adding adversarial perturbations to benign inputs to fool the DNN model to misclassify. In this paper, we present a cross-layer strategic ensemble fra…

New findings show strategic interactions can undermine model expressiveness in machine learning.

Consequential decision-making typically incentivizes individuals to behave strategically, tailoring their behavior to the specifics of the decision rule. A long line of work has therefore sought to counteract strategic behavior by designing more conservative decision boundaries in an effort to increase robustness to th…

The paper explores how to reduce classification tasks to optimization problems in Euclidean space.

New method learns robustly with less data, bridging theory and practice.

Strategic feature selection in high-stakes domains like healthcare.

Study allocates resources to strategic agents while balancing cost and incentives.

The paper analyzes trading strategies in a competitive market with incomplete information.

Algorithm learns optimal coordination for strategic agents in uncertain settings.

Randomised classifiers outperform deterministic ones in strategic classification.

Study one-shot strategic classification under unknown costs, improving worst-case accuracy.