PAC-Bayes framework fails on simple 1D linear classification task.
problem Proving the learnability of simple 1D linear classification tasks using PAC-Bayes bounds.
method Demonstrated a specific 1D linear classification task that PAC-Bayes cannot analyze.
result PAC-Bayes framework cannot prove learnability of simple 1D linear classification tasks.
Meta-learning bounds derived using PAC-Bayes theory for improved generalization.
problem Uncertainty in generalization performance for meta-learning with new tasks.
method PAC-Bayes relative entropy bounds and empirical risk minimization (ERM) method.
result Competitive generalization performance and rapid convergence with data-dependent prior.
New PAC-Bayes bounds for heavy-tailed losses using supermartingales.
problem Extending PAC-Bayes bounds to heavy-tailed losses.
method Using supermartingales and bounded variance assumption.
result PAC-Bayes generalization bounds for heavy-tailed losses.
New PAC-Bayes bounds use Wasserstein distances to improve generalization.
problem Lack of geometric properties in existing PAC-Bayes bounds.
method Developed new PAC-Bayes bounds with Wasserstein distances.
result Optimization guarantees translate to good generalization abilities.
This paper improves meta-learning by developing new PAC-Bayes bounds.
problem Meta-learning generalization gap across multiple tasks.
method Upper bounding convex functions linking environment and task-level losses.
result New PAC-Bayes bounds for meta-learning with improved algorithms.
This work uses PAC-Bayes for structured prediction with ILE, yielding insights and algorithms.
problem Structured prediction with interdependent outputs and implicit loss embeddings.
method PAC-Bayes perspective applied to ILE framework, deriving generalization bounds and learning algorithms.
result Two learning algorithms derived from PAC-Bayes bounds, analyzed and implemented.
Unified framework for anytime-valid PAC-Bayes bounds.
problem Deriving time-uniform PAC-Bayes bounds for stochastic processes.
method Combines four tools: nonnegative supermartingales, method of mixtures, Donsker-Varadhan formula, and Ville's inequality.
result Unified PAC-Bayes theorem for a wide class of discrete stochastic processes.
Paper develops a new generalization bound using PAC-Bayes theory and Gibbs distributions.
problem Limits of traditional generalization bounds due to complexity measures.
method Leverages PAC-Bayes bounds with Gibbs distributions to derive a flexible generalization bound.
result Derives a generalization bound that can adapt to both hypothesis class and task complexity.
Data-dependent PAC-Bayes priors via differential privacy improve generalization bounds.
problem Creating valid generalization bounds for unknown data distributions.
method Using ε-differential privacy to construct data-dependent priors, leading to valid PAC-Bayes bounds.
result Data-dependent priors via differential privacy yield nonvacuous generalization bounds.
PAC-Bayesian framework for fairness in stochastic and deterministic classifiers.
problem Theoretical guarantees on fairness for balancing predictive risk and fairness constraints.
method PAC-Bayesian framework for both stochastic and deterministic classifiers, covering a broad class of fairness measures.
result Derives generalization bounds for fairness, demonstrating tightness with empirical evaluation.
Paper develops PAC-Bayes bounds for unknown linear systems.
problem Learning controllers for unknown stochastic linear discrete-time systems.
method PAC-Bayes framework for data-dependent high probability bounds.
result Proposes efficient learning algorithms with theoretical guarantees.
New PAC-Bayes meta-learning method improves few-shot learning accuracy and calibration.
problem Few-shot learning with limited data.
method PAC-Bayes framework extended to meta-learning, estimating task-specific posteriors.
result State-of-the-art calibration and classification results on benchmarks.
Paper extends PAC-Bayesian theory using shifted Rademacher processes.
problem Improving PAC-Bayesian bounds for fast rates.
method Using shifted Rademacher processes to match Catoni's bounds and derive new fast-rate bounds.
result New fast-rate PAC-Bayes bounds derived in terms of empirical risk surface flatness.
Bayesian framework reduces online optimization regret.
problem Sequential optimization in dynamic environments with bounded losses.
method PAC-Bayes theory and Bayesian updating principles.
result Achieves O ( T ) \mathcal{O}(\sqrt{T}) O ( T ) regret for bounded losses. Investigates tight PAC-Bayes bounds for small datasets.
problem Tightening PAC-Bayes bounds for small data.
method Generic PAC-Bayes theorem, meta-learning, synthetic tasks.
result PAC-Bayes bounds are competitive with Chernoff bounds but not as tight.
Paper tightens PAC-Bayes bounds using coin-betting for better estimates.
problem Estimating mean of random elements with possibly S-dependent parameters.
method Refined PAC-Bayes proof strategy based on coin-betting framework.
result Derives tighter concentration inequalities for all sample sizes.
New method optimises learning via surrogate PAC-Bayes bounds.
problem Computational intractability of optimising generalisation bounds.
method Iteratively optimising surrogate training objectives derived from PAC-Bayes bounds.
result Iteratively optimising surrogates implies optimising original generalisation bounds.
New PAC-Bayes bounds for unbounded loss functions.
problem Generalization bounds for learning problems with unbounded loss functions.
method Introducing HYPE, a new notion for loss range, and deriving a novel PAC-Bayesian generalization bound.
result PAC-Bayes framework extended to unbounded loss functions.
Unified framework for learning flexible probabilistic programs using DPP and PAC-Bayes bounds.
problem Learning and generalizing from complex probabilistic models.
method Unified DPP representation and PAC-Bayes bounds for stochastic programs.
result Improved performance and generalization prediction using flexible DPP model representations and learned complexity measures.
Unified approach to neural network learning with PAC-Bayes bounds.
problem Overfitting and extrapolation issues in neural networks.
method Combines variational inference and PAC-Bayes for scalable learning.
result Validates theory and shows better generalization on high-dimensional tasks.
PAC-Bayes bound requires prior to place mass on high-performing predictors.
problem Explaining generalization in machine learning.
method Analyzing necessary conditions for PAC-Bayes bounds to provide meaningful generalization guarantees.
result Achieving a target generalisation level requires the prior to place sufficient mass on high-performing predictors.
New method reduces computational cost for estimating PAC-Bayes bounds.
problem High computational cost in estimating PAC-Bayes bounds.
method General alternative method that makes computational savings.
result Reduces computational cost on the order of the dataset size.
Derandomizing PAC-Bayes bounds for smooth loss functions
problem Derandomizing PAC-Bayes bounds for smooth loss functions
method Exploiting smoothness properties of both the loss and the predictor class
result Bounds for deterministic predictors that involve flatness quantities
New PAC-Bayes training method improves model generalization for unbounded loss.
problem Improving generalization of complex models under unbounded loss.
method Established new PAC-Bayes bound for unbounded loss, jointly training prior and posterior.
result Outperforms existing PAC-Bayes training algorithms and matches ERM accuracy.
This work extends PAC-Bayesian learning guarantees to non-compact symmetries and non-invariant data.
problem Lack of theoretical guarantees explaining the benefits of symmetries in machine learning models.
method Adapting and tightening PAC-Bayes bounds for non-compact symmetries and non-invariant data distributions.
result Theoretical evidence that symmetric models are preferable for symmetric data, beyond compact groups and invariant distributions.
PAC-Bayes with Backprop trains neural nets with competitive error estimates and tighter risk bounds.
problem Training probabilistic neural networks with PAC-Bayes bounds.
method Two training objectives derived from PAC-Bayes bounds, evaluated on MNIST and UCI data.
result Competitive test set error estimates and tighter risk bounds than previous results.
New PAC-Bayes bounds for martingale mixtures simplify prior work.
problem Bounding mixtures of martingales uniformly over distributions and times.
method PAC-Bayes approach, extending Bernstein inequalities.
result Tight concentration bounds for martingale mixtures.
New PAC-Bayes method updates priors without losing confidence information.
problem Lack of sequential prior updates in PAC-Bayes without losing confidence information.
method Recursive PAC-Bayes decomposition of expected loss.
result Sequential prior updates with no information loss.
Researchers estimate optimal PAC-Bayes bounds using Hamiltonian Monte Carlo.
problem Estimating tight PAC-Bayes bounds with restricted posterior families.
method Sampling from optimal Gibbs posterior using Hamiltonian Monte Carlo, estimating KL divergence, and proposing high-probability bounds.
result Significant tightness gaps in PAC-Bayes bounds, up to 5-6% in some cases.
Introduces PAC-Bayes bounds for understanding learning procedures.
problem Understanding the generalization ability of learning procedures.
method PAC-Bayesian bounds and their applications to neural networks.
result Simplified version of localization technique described.
Paper combines PAC-Bayes bounds with algorithm stability for risk estimation.
problem Estimating the risk of machine learning algorithms with data-dependent priors.
method Combines PAC-Bayes approach with algorithm stability, using instance-dependent priors.
result Estimates the risk of randomized algorithms in terms of hypothesis stability.
The paper improves PAC-Bayes bounds for data-dependent predictors.
problem Guaranteeing the quality of predictions on unseen examples.
method Basic PAC-Bayes inequality for stochastic kernels, leading to various bounds.
result Validates PAC-Bayes bounds without fixed 'data-free' priors and bounded losses.
New PAC bound for meta-learning improves generalization guarantees.
problem Provide strong generalization guarantees in meta-learning.
method PAC-Bayes and uniform stability frameworks applied to gradient-based meta-learning.
result Derives a tighter PAC bound for gradient-based meta-learning.
Meta-learning framework improves learning from past tasks.
problem Learning from related tasks to facilitate new, unseen tasks.
method Extended PAC-Bayes theory for meta-learning, incorporating prior knowledge through gradient-based optimization.
result Improved performance in learning new tasks through meta-learning.
The paper improves PAC-Bayes bounds for losses with finite moments.
problem Bounding generalization for losses with heavy tails and finite moments.
method Truncation method and PAC-Bayes bounds for unbounded losses with heavy tails and bounded variance.
result Bounds interpolate between slow and fast rates depending on the moment.
Survey and compare PAC-Bayes bounds for bandit problems.
problem Designing and evaluating bandit algorithms with strong performance guarantees.
method PAC-Bayes bounds applied to bandit problems.
result PAC-Bayes bounds useful for offline bandit algorithms, but loose for online algorithms.
PAC-Bayesian theory applied to learning optimization algorithms with generalization guarantees.
problem Learning optimization algorithms with provable generalization guarantees and explicit trade-offs.
method PAC-Bayes theory applied to learning-to-optimize, reformulating the learning procedure into a one-dimensional minimization problem.
result Learned optimization algorithms outperform deterministic worst-case analysis algorithms, even in the limit case of guaranteed convergence.
PAC-Bayes analysis explains sentence vector learning from unlabeled data.
problem Understanding and improving sentence vector learning from unlabeled data.
method PAC-Bayes bound analysis for transfer learning.
result Simple heuristics and new algorithms derived from PAC-Bayes analysis.
New empirical PAC-Bayes bound for Markov chains with finite state space.
problem Lack of empirical bounds for Markov chains with temporal dependence.
method Proved a new PAC-Bayes bound for Markov chains, providing an empirical pseudo-spectral gap.
result First fully empirical PAC-Bayes bound for Markov chains with finite state space.
Unified derivation of PAC-Bayes and MI bounds for general VC classes with fast rates.
problem Generalization bounds for machine learning models with VC classes.
method Unified derivation of conditional PAC-Bayesian and mutual information bounds, including MAC-Bayesian bounds.
result Nontrivial bounds for general VC classes and faster rates for specific conditions.
Pac-Bayes bounds are among the most accurate generalization bounds for classifiers learned from independently and identically distributed (IID) data, and it is particularly so for margin classifiers: there have been recent contributions showing how practical these bounds can be either to perform model selection (Ambrol…
The cold posterior effect is explored through PAC-Bayes bounds for small sample sizes.
problem The cold posterior effect in approximate Bayesian inference for small datasets.
method Investigation through PAC-Bayes generalization bounds, focusing on temperature parameter λ.
result The temperature parameter λ in PAC-Bayes bounds captures the cold posterior effect.
New online GP algorithm offers performance guarantees for streaming data.
problem Training and inference of GPs require all historic data, limiting online decision-making.
method Developed a new theoretical framework based on PAC-Bayes theory, optimizing empirical risk and parameter divergence.
result Offers both a guarantee of generalized performance and good accuracy.
This paper tackles non-vacuous generalization bounds in ReLU networks by resolving rescaling invariances.
problem Non-vacuous generalization guarantees for ReLU networks with rescaling invariances.
method Proposes a lifted representation to resolve rescaling invariances and studies KL-based rescaling-invariant PAC-Bayes bounds.
result KL-based rescaling-invariant PAC-Bayes bounds provide tighter guarantees and resolve discrepancies in network complexity.
New PAC-Bayes bound controls multiple error types simultaneously.
problem Current PAC-Bayes bounds are limited to scalar metrics.
method Bounding KL divergence between empirical and true probabilities of multiple error types.
result First PAC-Bayes bound for rich information-rich certificates.
PAC-Bayesian bounds estimate adversarial robustness.
problem Estimating robustness to imperceptible input perturbations.
method PAC-Bayesian framework for averaging over hypotheses.
result General bounds valid for any type of adversarial attacks.
The paper explores the limits of tight PAC-Bayes bounds for cheap models in robust statistics.
problem The challenge of obtaining meaningful bounds on the error of learning algorithms without prior assumptions.
method Investigates tight PAC-Bayes bounds for robust models with minimal cost.
result Demonstrates the limits of obtaining tight PAC-Bayes bounds for cheap models.
New inequality for ternary variables improves on existing measures.
problem Analyzing excess losses and weighted majority votes with ternary random variables.
method Developed a split-kl inequality and its PAC-Bayes extension.
result Outperforms existing inequalities in certain regimes.