The developments of Rademacher complexity and PAC-Bayesian theory have been largely independent. One exception is the PAC-Bayes theorem of Kakade, Sridharan, and Tewari (2008), which is established via Rademacher complexity theory by viewing Gibbs classifiers as linear operators. The goal of this paper is to extend thi…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “PAC theory”
Paper analyzes error exponent in agnostic PAC learning.
problem Analyzing performance of agnostic PAC learning.
method Using error exponent from Information Theory to analyze PAC learning.
result Improved distribution-dependent error exponent for agnostic learning.
This work provides statistical guarantees for VAEs using PAC-Bayesian theory.
problem Theoretical properties of VAEs remain open questions.
method PAC-Bayesian theory to derive statistical guarantees.
result Upper bounds on Wasserstein distance between input and generative model.
Meta-learning bounds derived using PAC-Bayes theory for improved generalization.
problem Uncertainty in generalization performance for meta-learning with new tasks.
method PAC-Bayes relative entropy bounds and empirical risk minimization (ERM) method.
result Competitive generalization performance and rapid convergence with data-dependent prior.
Introduces PAC-Bayes bounds for understanding learning procedures.
problem Understanding the generalization ability of learning procedures.
method PAC-Bayesian bounds and their applications to neural networks.
result Simplified version of localization technique described.
New PAC-Bayesian bounds explain few-shot learning performance gaps.
problem Gap between PAC-Bayesian theory and practice in few-shot learning.
method Developed new PAC-Bayesian bounds for few-shot learning, derived MAML and Reptile from these bounds, and introduced a new PACMAML algorithm.
result PAC-Bayesian bounds explain performance of MAML and Reptile, and outperform existing algorithms.
PAC-Bayesian theory applied to learning optimization algorithms with generalization guarantees.
problem Learning optimization algorithms with provable generalization guarantees and explicit trade-offs.
method PAC-Bayes theory applied to learning-to-optimize, reformulating the learning procedure into a one-dimensional minimization problem.
result Learned optimization algorithms outperform deterministic worst-case analysis algorithms, even in the limit case of guaranteed convergence.
Survey on learning Boolean functions in computational theory.
problem Learning Boolean function classes in computational theory.
method Overview of known results in PAC and related models.
result Discussion of various learning results for Boolean functions.
New PAC-Bayes bounds derived using Legendre transform and f-divergences.
problem Deriving PAC-Bayes bounds under various assumptions.
method Combining Legendre transform and Fenchel--Young inequality to derive change-of-measure inequalities.
result Extended PAC-Bayesian guarantees under tailored assumptions.
New PAC-Bayesian bounds for multi-view learning using Rényi divergence.
problem Applying PAC-Bayesian theory to multi-view learning.
method Introducing novel PAC-Bayesian bounds based on Rényi divergence for multi-view learning.
result Efficient optimization algorithms that align with theoretical bounds.
PAC-Bayesian theory applied to data-dependent hypothesis sets yields uniform generalization bounds.
problem Proving uniform generalization bounds for data-dependent hypothesis sets.
method Applying PAC-Bayesian framework on 'random sets' and considering data-dependent hypothesis sets.
result Data-dependent uniform generalization bounds are proven, providing tighter and unified results.
Extends PAC learning theory to handle partial concepts with special properties.
problem Traditional PAC learning theory cannot handle tasks with special data properties.
method Introduces partial concepts and new PAC learning framework.
result Partial concept classes cannot be captured by traditional PAC theory.
Develops higher arity VC theory and characterizes PAC learning in product spaces.
problem Characterizing PAC learning in multi-dimensional product spaces.
method Introduces higher arity VC dimension, generalizes Haussler packing lemma, and develops hypergraph regularity lemma.
result Characterizes higher arity PAC learning in n-fold product spaces.
The VAE's reconstruction ability is studied using PAC-Bayes theory.
problem Understanding the performance of VAEs for unseen data.
method PAC-Bayes theory is applied to analyze VAE's reconstruction error.
result Generalization bounds on VAE's reconstruction error are provided.
Paper develops a new generalization bound using PAC-Bayes theory and Gibbs distributions.
problem Limits of traditional generalization bounds due to complexity measures.
method Leverages PAC-Bayes bounds with Gibbs distributions to derive a flexible generalization bound.
result Derives a generalization bound that can adapt to both hypothesis class and task complexity.
New method reduces computational cost for estimating PAC-Bayes bounds.
problem High computational cost in estimating PAC-Bayes bounds.
method General alternative method that makes computational savings.
result Reduces computational cost on the order of the dataset size.
PAC-Bayes bounds for Gibbs posteriors derived via singular learning theory.
problem Generalization bounds for overparameterized models with data-dependent priors.
method Explicit non-asymptotic PAC-Bayes bounds using singular learning theory.
result Explicit posterior-averaged risk bounds for overparameterized models.
PAC-Bayesian bounds for stochastic LTI systems derived.
problem Error bounds for stochastic LTI systems.
method PAC-Bayesian theory applied to autonomous stochastic LTI models.
result Error bounds for stochastic LTI systems derived.
New PAC-Bayes bounds use Wasserstein distances to improve generalization.
problem Lack of geometric properties in existing PAC-Bayes bounds.
method Developed new PAC-Bayes bounds with Wasserstein distances.
result Optimization guarantees translate to good generalization abilities.
Survey and compare PAC-Bayes bounds for bandit problems.
problem Designing and evaluating bandit algorithms with strong performance guarantees.
method PAC-Bayes bounds applied to bandit problems.
result PAC-Bayes bounds useful for offline bandit algorithms, but loose for online algorithms.
Two different views on machine learning problem: Applied learning (machine learning with business applications) and Agnostic PAC learning are formalized and compared here. I show that, under some conditions, the theory of PAC Learnable provides a way to solve the Applied learning problem. However, the theory requires t…
PAC learning simplified as bipartite matching.
problem Efficiently solving PAC learning problems.
method Transductive learning and one-inclusion graphs.
result PAC learning can be reduced to bipartite matching.
New empirical PAC-Bayes bound for Markov chains with finite state space.
problem Lack of empirical bounds for Markov chains with temporal dependence.
method Proved a new PAC-Bayes bound for Markov chains, providing an empirical pseudo-spectral gap.
result First fully empirical PAC-Bayes bound for Markov chains with finite state space.
New PAC-Bayes bounds for unbounded loss functions.
problem Generalization bounds for learning problems with unbounded loss functions.
method Introducing HYPE, a new notion for loss range, and deriving a novel PAC-Bayesian generalization bound.
result PAC-Bayes framework extended to unbounded loss functions.
We propose an algorithm combining calibrated prediction and generalization bounds from learning theory to construct confidence sets for deep neural networks with PAC guarantees---i.e., the confidence set for a given input contains the true label with high probability. We demonstrate how our approach can be used to cons…
We exhibit a strong link between frequentist PAC-Bayesian risk bounds and the Bayesian marginal likelihood. That is, for the negative log-likelihood loss function, we show that the minimization of PAC-Bayesian generalization risk bounds maximizes the Bayesian marginal likelihood. This provides an alternative explanatio…
New PAC-Bayes training method improves model generalization for unbounded loss.
problem Improving generalization of complex models under unbounded loss.
method Established new PAC-Bayes bound for unbounded loss, jointly training prior and posterior.
result Outperforms existing PAC-Bayes training algorithms and matches ERM accuracy.
Paper confirms winning tickets have sharp minima, useful for generalization.
problem Explaining why over-parameterized models generalize well despite having sharp minima.
method PAC-Bayesian theory applied to analyze winning tickets and their generalization behavior.
result PAC-Bayesian theory confirms winning tickets have relatively sharp minima, a disadvantage for generalization.
The study improves PAC-Bayesian bounds for adversarial generative models.
problem Improving generalization bounds for adversarial generative models.
method Extending PAC-Bayesian theory to generative models, developing bounds for Wasserstein and total variation distances.
result New training objectives for Wasserstein and Energy-Based GANs.
The paper explores learning from label proportions, showing differences in efficiency between LLP and PAC learning.
problem Learning from label proportions (LLP) in unlabeled data with given label proportions.
method Formal definition and computational complexity analysis of LLP learning.
result LLP learning is more restrictive than PAC learning for finite VC classes, and some classes are uncharacterizable.
Study shows transductive learning is equivalent to PAC learning for most natural loss functions.
problem Understanding the relationship between transductive and PAC learning models.
method Extending existing results and developing new techniques to analyze the equivalence of the two models.
result Transductive learning is essentially equivalent to PAC learning for realizable learning with most natural loss functions.
New PAC-Bayesian bounds improve Sliced-Wasserstein distances.
problem Improving statistical properties of Sliced-Wasserstein distances.
method Leveraging PAC-Bayesian theory to provide bounds and learning procedures.
result PAC-Bayesian generalization bounds for adaptive SW distances.
Unified treatment of PAC-Bayesian and information-theoretic generalization bounds.
problem Generalization capabilities of machine learning algorithms.
method PAC-Bayesian and information-theoretic perspectives.
result Unified treatment and modular structure of proofs.
New risk measures control subgroup imbalances, improving PAC-Bayesian bounds.
problem Insufficient risk bounds for subgroup imbalances in data.
method Introduce constrained f-entropic risk measures and derive PAC-Bayesian bounds.
result First disintegrated PAC-Bayesian guarantees beyond standard risks.
Develops a theory to make learning solutions fair and safe.
problem Ensuring learning solutions are unbiased and safe in critical applications.
method Generates a generalization theory based on PAC learning framework, introduces constrained learning algorithm.
result Proves that constrained learning is as learnable as unconstrained learning, provides practical algorithm.
Paper introduces new PAC-Bayesian bounds for multi-view domain adaptation.
problem Lack of attention to multi-view learning in domain adaptation.
method Adapted distance measure for multi-view domain adaptation using Pac-Bayesian theory.
result Introduced novel Pac-Bayesian bounds for multi-view domain adaptation.
Enhances predictive models against misspecification and outliers.
problem Suboptimal generalization under misspecification and outliers.
method Combines PAC ensemble bounds with a generalized logarithm score function.
result Produces predictive distributions resistant to both misspecification and outliers.
MMD-Balls as Credal Sets: A PAC-Bayesian Framework for Epistemic Uncertainty in Test-Time Adaptationcs.LG
New framework improves model reliability under distribution shifts.
problem Lack of formal guarantees connecting shift magnitude to prediction reliability in TTA methods.
method Develops a PAC-Bayesian framework interpreting MMD-balls as credal sets.
result Establishes generalization bounds and provides epistemic uncertainty quantification.
This work extends PAC-Bayesian learning guarantees to non-compact symmetries and non-invariant data.
problem Lack of theoretical guarantees explaining the benefits of symmetries in machine learning models.
method Adapting and tightening PAC-Bayes bounds for non-compact symmetries and non-invariant data distributions.
result Theoretical evidence that symmetric models are preferable for symmetric data, beyond compact groups and invariant distributions.
We present a comprehensive study of multilayer neural networks with binary activation, relying on the PAC-Bayesian theory. Our contributions are twofold: (i) we develop an end-to-end framework to train a binary activated deep neural network, (ii) we provide nonvacuous PAC-Bayesian generalization bounds for binary activ…
The paper establishes bounds for transductive learning using information theory.
problem Transductive learning generalization gap control.
method Information theory, PAC-Bayes, mutual information, conditional mutual information, different information measures.
result Established transductive information-theoretic and PAC-Bayesian bounds.
New method learns robustly with less data, bridging theory and practice.
problem Adversarial robust learning with metric perturbation.
method Tolerant adversarial PAC-learning with perturb-and-smooth approach and compression-based algorithm.
result First PAC-type guarantees for popular adversarial learning techniques.
New PAC-Bayesian framework for flexible meta-learning.
problem Limitations in transferring knowledge between tasks in meta-learning.
method PAC-Bayesian theory applied to learning the learning algorithm.
result Flexibility in meta-learning mechanisms and improved prediction quality.
IIC provides a PAC-Bayes bound for interpolating models, revealing factors affecting generalization.
problem Theoretical challenges in understanding overparameterized models and their performance.
method PAC-Bayesian perspective applied to the Interpolating Information Criterion (IIC).
result Test error for overparameterized models achieving zero training error depends on various factors.
We propose an extensive analysis of the behavior of majority votes in binary classification. In particular, we introduce a risk bound for majority votes, called the C-bound, that takes into account the average quality of the voters and their average disagreement. We also propose an extensive PAC-Bayesian analysis that …
In this paper, we provide two main contributions in PAC-Bayesian theory for domain adaptation where the objective is to learn, from a source distribution, a well-performing majority vote on a different target distribution. On the one hand, we propose an improvement of the previous approach proposed by Germain et al. (2…
Survey of reinforcement learning guarantees with data constraints.
problem Guaranteeing near-optimal policies with limited data in reinforcement learning.
method Coverage-Structure-Objective (CSO) framework to decompose sample complexity results.
result Progress on PAC guarantees for reinforcement learning, covering various models and settings.
This paper provides a theoretical analysis of domain adaptation based on the PAC-Bayesian theory. We propose an improvement of the previous domain adaptation bound obtained by Germain et al. in two ways. We first give another generalization bound tighter and easier to interpret. Moreover, we provide a new analysis of t…