PAC-Bayesian bounds for MLPs with cross entropy loss validated.
problem Generalization bounds for MLPs with cross entropy loss.
method Introduced probabilistic explanations and proved PAC-Bayesian bounds using ELBO.
result MLPs with cross entropy loss inherently guarantee PAC-Bayesian generalization bounds.
This research improves PAC-Bayesian bounds for classification tasks using convexified loss.
problem Deriving generalization bounds for classification tasks with non-convex loss functions.
method Shift focus to misclassification excess risk bounds for PAC-Bayesian classification using convex surrogate loss and leveraging PAC-Bayesian relative bounds in expectation.
result Improved PAC-Bayesian bounds for classification tasks with convex surrogate loss.
Develops a new PAC-Bayesian bound for evaluating randomness in predictors.
problem Lack of a PAC-Bayesian bound for deterministic predictors and continuous loss functions.
method PAC-Bayesian transportation bound unifying PAC-Bayesian and chaining methods.
result First PAC-Bayesian bound relating risks of any two predictors by their distance.
New algorithms minimize PAC-Bayesian C-Bound for majority voting, leading to scalable and accurate predictors.
problem Improving majority vote classifiers using PAC-Bayesian bounds.
method Directly optimizing PAC-Bayesian guarantees on the C-Bound with gradient descent.
result Self-bounding majority vote learning algorithms with scalable and accurate predictors.
New PAC-Bayesian bounds for multi-view learning using Rényi divergence.
problem Applying PAC-Bayesian theory to multi-view learning.
method Introducing novel PAC-Bayesian bounds based on Rényi divergence for multi-view learning.
result Efficient optimization algorithms that align with theoretical bounds.
PAC-Bayesian bounds for stochastic LTI systems derived.
problem Error bounds for stochastic LTI systems.
method PAC-Bayesian theory applied to autonomous stochastic LTI models.
result Error bounds for stochastic LTI systems derived.
Adapts PAC-Bayesian analysis to convolutional neural networks.
problem Generalization error of convolutional neural networks.
method PAC-Bayesian framework applied to convolutional layers.
result Margin bounds for convolutional neural networks.
This paper relaxes PAC-Bayesian assumptions for dependent, heavy-tailed data.
problem Connecting the generalization ability of an aggregation distribution to empirical risk and KL divergence.
method Introduces PAC-Bayesian bounds using Csiszár's f-divergence for dependent, heavy-tailed data. result Provides simplified PAC-Bayesian bounds for hostile data.
New PAC-Bayesian bounds provide practical guarantees for neural networks.
problem Loose derandomization step in PAC-Bayesian bounds for deterministic models.
method Introduce disintegrated PAC-Bayesian bounds for deterministic models.
result Significant practical improvement over state-of-the-art bounds.
Paper improves PAC-Bayesian bounds for linear regression.
problem Improving PAC-Bayesian error bounds for linear regression.
method Proposes a tighter error bound for linear regression that converges to the generalization loss with a well-chosen temperature parameter. Also applies to non-i.i.d. training data.
result Error bound applies to certain time series generated by dynamical models.
New PAC-Bayesian bounds for online learning with data streams.
problem Challenges of traditional PAC-Bayesian bounds in dynamic data collection.
method Developed new PAC-Bayesian bounds in online learning framework, using updated regret definition and batch-to-online conversion.
result PAC-Bayesian bounds hold for online learning with dependent data and non-convex losses.
PAC-Bayesian theory applied to data-dependent hypothesis sets yields uniform generalization bounds.
problem Proving uniform generalization bounds for data-dependent hypothesis sets.
method Applying PAC-Bayesian framework on 'random sets' and considering data-dependent hypothesis sets.
result Data-dependent uniform generalization bounds are proven, providing tighter and unified results.
Paper derives PAC-Bayesian bounds for LTI systems learning from empirical data.
problem Characterizing predictive power of LTI systems learned from data.
method PAC-Bayesian bounds for LTI stochastic dynamical systems with inputs.
result Finite-sample error bounds for learning algorithms of LTI systems.
The study improves PAC-Bayesian bounds for adversarial generative models.
problem Improving generalization bounds for adversarial generative models.
method Extending PAC-Bayesian theory to generative models, developing bounds for Wasserstein and total variation distances.
result New training objectives for Wasserstein and Energy-Based GANs.
The study examines how equivariance in networks affects generalization error using PAC-Bayesian bounds.
problem Understanding how equivariance in networks impacts generalization error.
method Utilized PAC-Bayesian analysis for equivariant networks, deriving norm-based bounds for generalization error.
result The bound indicates that using larger group size in the model improves generalization error.
We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This …
Paper derives an error bound for stochastic LTI systems.
problem Stochastic LTI systems with inputs in control engineering and econometrics.
method PAC-Bayesian-Like error bound derivation.
result Derived an error bound for stochastic LTI systems.
New PAC-Bayesian bounds improve Sliced-Wasserstein distances.
problem Improving statistical properties of Sliced-Wasserstein distances.
method Leveraging PAC-Bayesian theory to provide bounds and learning procedures.
result PAC-Bayesian generalization bounds for adaptive SW distances.
Paper introduces new PAC-Bayesian bounds for multi-view domain adaptation.
problem Lack of attention to multi-view learning in domain adaptation.
method Adapted distance measure for multi-view domain adaptation using Pac-Bayesian theory.
result Introduced novel Pac-Bayesian bounds for multi-view domain adaptation.
Unified framework for deriving generalization bounds in supervised learning.
problem Generalization error bounds in supervised learning.
method Data Processing Inequality PAC-Bayesian framework.
result Unified bounds on binary Kullback-Leibler generalization gap for various divergences.
This work provides statistical guarantees for VAEs using PAC-Bayesian theory.
problem Theoretical properties of VAEs remain open questions.
method PAC-Bayesian theory to derive statistical guarantees.
result Upper bounds on Wasserstein distance between input and generative model.
We propose an extensive analysis of the behavior of majority votes in binary classification. In particular, we introduce a risk bound for majority votes, called the C-bound, that takes into account the average quality of the voters and their average disagreement. We also propose an extensive PAC-Bayesian analysis that …
We prove new concentration inequalities for random variables.
problem Concentration of random variables in nonlinear functions.
method Efron-Stein inequalities and PAC-Bayesian approach.
result User-friendly concentration bounds for various applications.
Innovative inequalities for divergences with applications in PAC-Bayesian bounds and Monte Carlo.
problem Developing new inequalities for divergences.
method Introducing novel change of measure inequalities for f-divergences and α-divergences. result Applications in PAC-Bayesian bounds and Monte Carlo estimates.
Paper extends PAC-Bayesian theory using shifted Rademacher processes.
problem Improving PAC-Bayesian bounds for fast rates.
method Using shifted Rademacher processes to match Catoni's bounds and derive new fast-rate bounds.
result New fast-rate PAC-Bayes bounds derived in terms of empirical risk surface flatness.
Study binary activated deep neural networks using PAC-Bayesian theory.
problem Generalization bounds for binary activated deep neural networks.
method Developed an end-to-end framework and provided PAC-Bayesian generalization bounds.
result Nonvacuous PAC-Bayesian generalization bounds for binary activated deep neural networks.
Bayesian priors offer a compact yet general means of incorporating domain knowledge into many learning tasks. The correctness of the Bayesian analysis and inference, however, largely depends on accuracy and correctness of these priors. PAC-Bayesian methods overcome this problem by providing bounds that hold regardless …
New risk measures control subgroup imbalances, improving PAC-Bayesian bounds.
problem Insufficient risk bounds for subgroup imbalances in data.
method Introduce constrained f-entropic risk measures and derive PAC-Bayesian bounds.
result First disintegrated PAC-Bayesian guarantees beyond standard risks.
PAC-Bayesian bounds improve understanding of K-NN classifier performance.
problem Improving the understanding of K-NN classifier's generalization error.
method PAC-Bayesian analysis applied to K-NN classifier in kernel space.
result PAC-Bayesian bounds provide a function of the number of redundant training examples.
Unified PAC-Bayesian framework for deep learning generalization.
problem Limitations of existing PAC-Bayesian norm-based bounds for deep neural networks.
method Unified framework using anisotropic Gaussian posteriors and sensitivity matrix.
result Comparable or tighter generalization bounds compared to state-of-the-art approaches.
New algorithm optimizes PAC-Bayes bound without surrogate loss.
problem Mismatch between optimisation objective and generalisation bound in stochastic neural networks.
method Proposes a novel training algorithm that optimizes the PAC-Bayesian bound directly.
result Empirical results show improved performance over existing PAC-Bayesian training methods.
We exhibit a strong link between frequentist PAC-Bayesian risk bounds and the Bayesian marginal likelihood. That is, for the negative log-likelihood loss function, we show that the minimization of PAC-Bayesian generalization risk bounds maximizes the Bayesian marginal likelihood. This provides an alternative explanatio…
New bounds improve generalization in machine learning with high probability.
problem Erratic behavior of KL divergence limits practical applications.
method Replaced KL divergence with Wasserstein distance for better bounds.
result Proved high probability generalization bounds for i.i.d. and non-i.i.d. data.
PAC-Bayesian bounds estimate adversarial robustness.
problem Estimating robustness to imperceptible input perturbations.
method PAC-Bayesian framework for averaging over hypotheses.
result General bounds valid for any type of adversarial attacks.
New PAC-Bayesian bound improves generalization performance.
problem Improving generalization bounds for learning algorithms.
method Developed a new PAC-Bayesian generalization bound using a Bernstein-like inequality.
result The new bound consistently outperforms existing bounds and converges faster under certain conditions.
This paper provides a PAC-Bayesian bound for CVaR in machine learning.
problem Learning algorithms minimizing CVaR of empirical loss.
method Generalization bound of PAC-Bayesian type, reducing CVaR estimation to expectation estimation.
result The bound is small when empirical CVaR is small, providing concentration inequalities for CVaR.
New PAC-Bayesian bound with convexity in posterior and trade-off parameter.
problem Constructing a hypothesis space with convex bounds for empirical performance and complexity.
method Proposed a new PAC-Bayesian bound and alternating minimization procedure.
result The bound can be minimized to find optimal trade-off between empirical performance and complexity.
New PAC-Bayesian bounds explain few-shot learning performance gaps.
problem Gap between PAC-Bayesian theory and practice in few-shot learning.
method Developed new PAC-Bayesian bounds for few-shot learning, derived MAML and Reptile from these bounds, and introduced a new PACMAML algorithm.
result PAC-Bayesian bounds explain performance of MAML and Reptile, and outperform existing algorithms.
New reinforcement learning bound improves generalization for sequential data.
problem Challenges in obtaining generalization guarantees for reinforcement learning due to sequential data.
method PAC-Bayesian reinforcement learning with consideration of Markov dependencies and mixing time.
result Demonstrated practical utility through PB-SAC, providing meaningful confidence certificates.
We study the issue of PAC-Bayesian domain adaptation: We want to learn, from a source domain, a majority vote model dedicated to a target one. Our theoretical contribution brings a new perspective by deriving an upper-bound on the target risk where the distributions' divergence---expressed as a ratio---controls the tra…
New bounds for model generalization under deterministic gradient descent.
problem Establishing generalization bounds for models trained with gradient descent methods.
method PAC-Bayesian bounds for deterministic optimisation algorithms.
result Fully computable bounds that depend on initial distribution and Hessian.
Paper presents first PAC-Bayesian bounds for KGRL methods.
problem Lack of theoretical analysis for KGRL methods.
method Proposes ReED framework for KGRL, covering 15 models.
result Provides theoretical grounding for KGRL methods.
PAC-Bayesian analysis improves lifelong learning in multi-armed bandits.
problem Improving lifelong learning in multi-armed bandits.
method PAC-Bayesian analysis for deriving lower bounds and proposing lifelong learning algorithms.
result Proposed algorithms outperform baseline methods in lifelong multi-armed bandit problems.
Wide stochastic networks show Gaussian behavior and improve training with PAC-Bayesian methods.
problem Analyzing and training over-parameterised neural networks with large width.
method Establishing Gaussian behavior for a stochastic architecture, applying PAC-Bayesian training.
result PAC-Bayesian training on large but finite-width networks outperforms standard methods.
Paper provides robustness bounds for GNNs against adversarial attacks.
problem Adversarial robustness of GNNs for graph-related tasks.
method PAC-Bayesian framework applied to GCN and MP-GNN.
result Spectral norms of diffusion matrix and weights govern robustness.
New bound relaxes uniform gradient norm assumptions for PAC-Bayesian bounds.
problem Generalization bounds with strict assumptions like uniformly bounded loss.
method Relax uniform bounds assumptions to on-average bounded loss and gradient norm.
result Proposes a new generalization bound with a surrogate of model complexity.
This paper provides a theoretical analysis of domain adaptation based on the PAC-Bayesian theory. We propose an improvement of the previous domain adaptation bound obtained by Germain et al. in two ways. We first give another generalization bound tighter and easier to interpret. Moreover, we provide a new analysis of t…
New framework improves model reliability under distribution shifts.
problem Lack of formal guarantees connecting shift magnitude to prediction reliability in TTA methods.
method Develops a PAC-Bayesian framework interpreting MMD-balls as credal sets.
result Establishes generalization bounds and provides epistemic uncertainty quantification.