Paper analyzes error exponent in agnostic PAC learning.
problem Analyzing performance of agnostic PAC learning.
method Using error exponent from Information Theory to analyze PAC learning.
result Improved distribution-dependent error exponent for agnostic learning.
Meta-learning bounds derived using PAC-Bayes theory for improved generalization.
problem Uncertainty in generalization performance for meta-learning with new tasks.
method PAC-Bayes relative entropy bounds and empirical risk minimization (ERM) method.
result Competitive generalization performance and rapid convergence with data-dependent prior.
New PAC-Bayesian bounds explain few-shot learning performance gaps.
problem Gap between PAC-Bayesian theory and practice in few-shot learning.
method Developed new PAC-Bayesian bounds for few-shot learning, derived MAML and Reptile from these bounds, and introduced a new PACMAML algorithm.
result PAC-Bayesian bounds explain performance of MAML and Reptile, and outperform existing algorithms.
Survey on learning Boolean functions in computational theory.
problem Learning Boolean function classes in computational theory.
method Overview of known results in PAC and related models.
result Discussion of various learning results for Boolean functions.
PAC-Bayesian theory applied to learning optimization algorithms with generalization guarantees.
problem Learning optimization algorithms with provable generalization guarantees and explicit trade-offs.
method PAC-Bayes theory applied to learning-to-optimize, reformulating the learning procedure into a one-dimensional minimization problem.
result Learned optimization algorithms outperform deterministic worst-case analysis algorithms, even in the limit case of guaranteed convergence.
Algorithm constructs confidence sets for deep neural networks with PAC guarantees.
problem Ensuring reliable predictions for deep neural networks with high confidence.
method Combines calibrated prediction and learning theory bounds.
result Constructs PAC confidence sets for various deep models.
New PAC-Bayesian bounds for multi-view learning using Rényi divergence.
problem Applying PAC-Bayesian theory to multi-view learning.
method Introducing novel PAC-Bayesian bounds based on Rényi divergence for multi-view learning.
result Efficient optimization algorithms that align with theoretical bounds.
Introduces PAC-Bayes bounds for understanding learning procedures.
problem Understanding the generalization ability of learning procedures.
method PAC-Bayesian bounds and their applications to neural networks.
result Simplified version of localization technique described.
Extends PAC learning theory to handle partial concepts with special properties.
problem Traditional PAC learning theory cannot handle tasks with special data properties.
method Introduces partial concepts and new PAC learning framework.
result Partial concept classes cannot be captured by traditional PAC theory.
Develops higher arity VC theory and characterizes PAC learning in product spaces.
problem Characterizing PAC learning in multi-dimensional product spaces.
method Introduces higher arity VC dimension, generalizes Haussler packing lemma, and develops hypergraph regularity lemma.
result Characterizes higher arity PAC learning in n-fold product spaces.
This work provides statistical guarantees for VAEs using PAC-Bayesian theory.
problem Theoretical properties of VAEs remain open questions.
method PAC-Bayesian theory to derive statistical guarantees.
result Upper bounds on Wasserstein distance between input and generative model.
New PAC-Bayes bounds derived using Legendre transform and f-divergences.
problem Deriving PAC-Bayes bounds under various assumptions.
method Combining Legendre transform and Fenchel--Young inequality to derive change-of-measure inequalities.
result Extended PAC-Bayesian guarantees under tailored assumptions.
Paper extends PAC-Bayesian theory using shifted Rademacher processes.
problem Improving PAC-Bayesian bounds for fast rates.
method Using shifted Rademacher processes to match Catoni's bounds and derive new fast-rate bounds.
result New fast-rate PAC-Bayes bounds derived in terms of empirical risk surface flatness.
PAC-Bayes bounds for Gibbs posteriors derived via singular learning theory.
problem Generalization bounds for overparameterized models with data-dependent priors.
method Explicit non-asymptotic PAC-Bayes bounds using singular learning theory.
result Explicit posterior-averaged risk bounds for overparameterized models.
Two different views on machine learning problem: Applied learning (machine learning with business applications) and Agnostic PAC learning are formalized and compared here. I show that, under some conditions, the theory of PAC Learnable provides a way to solve the Applied learning problem. However, the theory requires t…
Paper develops a new generalization bound using PAC-Bayes theory and Gibbs distributions.
problem Limits of traditional generalization bounds due to complexity measures.
method Leverages PAC-Bayes bounds with Gibbs distributions to derive a flexible generalization bound.
result Derives a generalization bound that can adapt to both hypothesis class and task complexity.
PAC learning simplified as bipartite matching.
problem Efficiently solving PAC learning problems.
method Transductive learning and one-inclusion graphs.
result PAC learning can be reduced to bipartite matching.
New PAC-Bayes bounds use Wasserstein distances to improve generalization.
problem Lack of geometric properties in existing PAC-Bayes bounds.
method Developed new PAC-Bayes bounds with Wasserstein distances.
result Optimization guarantees translate to good generalization abilities.
The paper explores learning from label proportions, showing differences in efficiency between LLP and PAC learning.
problem Learning from label proportions (LLP) in unlabeled data with given label proportions.
method Formal definition and computational complexity analysis of LLP learning.
result LLP learning is more restrictive than PAC learning for finite VC classes, and some classes are uncharacterizable.
PAC-Bayesian bounds for stochastic LTI systems derived.
problem Error bounds for stochastic LTI systems.
method PAC-Bayesian theory applied to autonomous stochastic LTI models.
result Error bounds for stochastic LTI systems derived.
Study shows transductive learning is equivalent to PAC learning for most natural loss functions.
problem Understanding the relationship between transductive and PAC learning models.
method Extending existing results and developing new techniques to analyze the equivalence of the two models.
result Transductive learning is essentially equivalent to PAC learning for realizable learning with most natural loss functions.
New PAC-Bayes bounds for unbounded loss functions.
problem Generalization bounds for learning problems with unbounded loss functions.
method Introducing HYPE, a new notion for loss range, and deriving a novel PAC-Bayesian generalization bound.
result PAC-Bayes framework extended to unbounded loss functions.
Develops a theory to make learning solutions fair and safe.
problem Ensuring learning solutions are unbiased and safe in critical applications.
method Generates a generalization theory based on PAC learning framework, introduces constrained learning algorithm.
result Proves that constrained learning is as learnable as unconstrained learning, provides practical algorithm.
Survey and compare PAC-Bayes bounds for bandit problems.
problem Designing and evaluating bandit algorithms with strong performance guarantees.
method PAC-Bayes bounds applied to bandit problems.
result PAC-Bayes bounds useful for offline bandit algorithms, but loose for online algorithms.
Unified treatment of PAC-Bayesian and information-theoretic generalization bounds.
problem Generalization capabilities of machine learning algorithms.
method PAC-Bayesian and information-theoretic perspectives.
result Unified treatment and modular structure of proofs.
Paper introduces new PAC-Bayesian bounds for multi-view domain adaptation.
problem Lack of attention to multi-view learning in domain adaptation.
method Adapted distance measure for multi-view domain adaptation using Pac-Bayesian theory.
result Introduced novel Pac-Bayesian bounds for multi-view domain adaptation.
New PAC-Bayesian framework for flexible meta-learning.
problem Limitations in transferring knowledge between tasks in meta-learning.
method PAC-Bayesian theory applied to learning the learning algorithm.
result Flexibility in meta-learning mechanisms and improved prediction quality.
Formally proves machine learning for simple classifiers.
problem Proving PAC learnability for decision stumps.
method Formal proof in Lean, separating deterministic and probabilistic proofs.
result Formal proof of PAC learnability for decision stumps.
The paper establishes bounds for transductive learning using information theory.
problem Transductive learning generalization gap control.
method Information theory, PAC-Bayes, mutual information, conditional mutual information, different information measures.
result Established transductive information-theoretic and PAC-Bayesian bounds.
This work extends PAC-Bayesian learning guarantees to non-compact symmetries and non-invariant data.
problem Lack of theoretical guarantees explaining the benefits of symmetries in machine learning models.
method Adapting and tightening PAC-Bayes bounds for non-compact symmetries and non-invariant data distributions.
result Theoretical evidence that symmetric models are preferable for symmetric data, beyond compact groups and invariant distributions.
Enhances predictive models against misspecification and outliers.
problem Suboptimal generalization under misspecification and outliers.
method Combines PACm ensemble bounds with a generalized logarithm score function. result Produces predictive distributions resistant to both misspecification and outliers.
Paper confirms winning tickets have sharp minima, useful for generalization.
problem Explaining why over-parameterized models generalize well despite having sharp minima.
method PAC-Bayesian theory applied to analyze winning tickets and their generalization behavior.
result PAC-Bayesian theory confirms winning tickets have relatively sharp minima, a disadvantage for generalization.
New method learns robustly with less data, bridging theory and practice.
problem Adversarial robust learning with metric perturbation.
method Tolerant adversarial PAC-learning with perturb-and-smooth approach and compression-based algorithm.
result First PAC-type guarantees for popular adversarial learning techniques.
New method improves meta-learning scalability and generalization.
problem Overfitting in meta-learning with limited related tasks.
method PAC-Bayesian theory and optimal hyper-posterior (PACOH).
result PACOH provides better performance guarantees and scalability.
Study binary activated deep neural networks using PAC-Bayesian theory.
problem Generalization bounds for binary activated deep neural networks.
method Developed an end-to-end framework and provided PAC-Bayesian generalization bounds.
result Nonvacuous PAC-Bayesian generalization bounds for binary activated deep neural networks.
PAC-Bayesian theory applied to data-dependent hypothesis sets yields uniform generalization bounds.
problem Proving uniform generalization bounds for data-dependent hypothesis sets.
method Applying PAC-Bayesian framework on 'random sets' and considering data-dependent hypothesis sets.
result Data-dependent uniform generalization bounds are proven, providing tighter and unified results.
New PAC-Bayes training method improves model generalization for unbounded loss.
problem Improving generalization of complex models under unbounded loss.
method Established new PAC-Bayes bound for unbounded loss, jointly training prior and posterior.
result Outperforms existing PAC-Bayes training algorithms and matches ERM accuracy.
The VAE's reconstruction ability is studied using PAC-Bayes theory.
problem Understanding the performance of VAEs for unseen data.
method PAC-Bayes theory is applied to analyze VAE's reconstruction error.
result Generalization bounds on VAE's reconstruction error are provided.
Unified approach to neural network learning with PAC-Bayes bounds.
problem Overfitting and extrapolation issues in neural networks.
method Combines variational inference and PAC-Bayes for scalable learning.
result Validates theory and shows better generalization on high-dimensional tasks.
New method reduces computational cost for estimating PAC-Bayes bounds.
problem High computational cost in estimating PAC-Bayes bounds.
method General alternative method that makes computational savings.
result Reduces computational cost on the order of the dataset size.
New PAC-Bayesian bounds improve Sliced-Wasserstein distances.
problem Improving statistical properties of Sliced-Wasserstein distances.
method Leveraging PAC-Bayesian theory to provide bounds and learning procedures.
result PAC-Bayesian generalization bounds for adaptive SW distances.
Framework for private, noise-tolerant, and efficient learning algorithms.
problem Private and efficient learning of large-margin halfspaces in noisy environments.
method Simple framework using differential privacy and noise tolerance conditions.
result Noise-tolerant and private PAC learners for large-margin halfspaces with sample complexity independent of dimension.
The paper bridges theory and practice in query-driven selectivity learning.
problem Insufficient theoretical understanding of query-driven selectivity learning.
method Demonstrates learnability of selectivity predictors and establishes favorable OOD generalization error bounds.
result Theoretical advances improve OOD generalization of query-driven selectivity models.
Survey of reinforcement learning guarantees with data constraints.
problem Guaranteeing near-optimal policies with limited data in reinforcement learning.
method Coverage-Structure-Objective (CSO) framework to decompose sample complexity results.
result Progress on PAC guarantees for reinforcement learning, covering various models and settings.
We propose an extensive analysis of the behavior of majority votes in binary classification. In particular, we introduce a risk bound for majority votes, called the C-bound, that takes into account the average quality of the voters and their average disagreement. We also propose an extensive PAC-Bayesian analysis that …
PAC-Bayes theory improves ICP efficiency and coverage.
problem Inefficient and unreliable uncertainty estimates in deep learning models.
method PAC-Bayes theory to optimize model and score function parameters for efficient and reliable prediction sets.
result Generalization bounds on coverage and efficiency of optimized prediction sets.
Study analyzes EU in variational inference for Bayesian deep learning.
problem Analyzing epistemic uncertainty in variational inference for Bayesian deep learning.
method Theoretical analysis and derivation of relations between generalization error and EU measurements. Proposing a new objective function for VI.
result Proposed new objective function significantly improves EU evaluation over existing VI methods.
Unified PAC-Bayesian framework for deep learning generalization.
problem Limitations of existing PAC-Bayesian norm-based bounds for deep neural networks.
method Unified framework using anisotropic Gaussian posteriors and sensitivity matrix.
result Comparable or tighter generalization bounds compared to state-of-the-art approaches.