arXiv research

Improved MALA method for neural networks uncertainty quantification.

Bayesian inference for Levy density with Gibbs posterior in discrete sampling.

A new method improves uncertainty quantification in Bayesian inference.

In the popular approach of "Bayesian variable selection" (BVS), one uses prior and posterior distributions to select a subset of candidate variables to enter the model. A completely new direction will be considered here to study BVS with a Gibbs posterior originating in statistical mechanics. The Gibbs posterior is con…

PAC-Bayes bounds for Gibbs posteriors derived via singular learning theory.

The PAC-Bayesian approach is a powerful set of techniques to derive non- asymptotic risk bounds for random estimators. The corresponding optimal distribution of estimators, usually called the Gibbs posterior, is unfortunately intractable. One may sample from it using Markov chain Monte Carlo, but this is often too slow…

New methods for tuning alpha in Gibbs posteriors improve speed and accuracy.

Develops a Bayesian framework for portfolio choice with a new posterior distribution.

The paper extends entropy maximization to multiscale settings and applies it to neural networks.

A new framework for clustering with uncertainty quantification.

Researchers estimate optimal PAC-Bayes bounds using Hamiltonian Monte Carlo.

Combines MALA and Adam for efficient uncertainty quantification in deep learning.

We show that Entropy-SGD (Chaudhari et al., 2017), when viewed as a learning algorithm, optimizes a PAC-Bayes bound on the risk of a Gibbs (posterior) classifier, i.e., a randomized classifier obtained by a risk-sensitive perturbation of the weights of a learned classifier. Entropy-SGD works by optimizing the bound's p…

We derive PAC-Bayesian learning guarantees for heavy-tailed losses, and obtain a novel optimal Gibbs posterior which enjoys finite-sample excess risk bounds at logarithmic confidence. Our core technique itself makes use of PAC-Bayesian inequalities in order to derive a robust risk estimator, which by design is easy to …

Bayesian models' singular fluctuation is shown to be akin to specific heat, influencing model complexity and generalization.

Oracle inequality for sparse neural nets adapts to unknown structure.

GADD accelerates uniform-rate discrete diffusion models by 2 orders of magnitude.

The Gibbs algorithm's generalization error is bounded, improving with prior volume in low temperatures.

Bayesian framework reduces online optimization regret.

Paper introduces a Gibbs sampler for Bayesian inversion of ill-posed problems.

DABS uses a policy network to select experiments in high-dimensional design spaces.

New PAC-Bayes bounds for unbounded losses using Cramér-Chernoff techniques.

REALITrees uses a Rashomon ensemble approach for active learning in sparse decision trees.

We derive generalization and excess risk bounds for neural nets using a family of complexity measures based on a multilevel relative entropy. The bounds are obtained by introducing the notion of generated hierarchical coverings of neural nets and by using the technique of chaining mutual information introduced in Asadi…

Study detects boundaries in unlabeled noisy images without labels.

Advocates for a new posterior that predicts better than classical and generalised Bayes.

Bayesian framework detects symmetries in chaotic dynamical systems.

New information-theoretic bounds improve machine learning generalization.

This monograph deals with adaptive supervised classification, using tools borrowed from statistical mechanics and information theory, stemming from the PACBayesian approach pioneered by David McAllester and applied to a conception of statistical learning theory forged by Vladimir Vapnik. Using convex analysis on the se…