arXiv research

In the popular approach of "Bayesian variable selection" (BVS), one uses prior and posterior distributions to select a subset of candidate variables to enter the model. A completely new direction will be considered here to study BVS with a Gibbs posterior originating in statistical mechanics. The Gibbs posterior is con…

Gibbs-ERM learning is a natural idealized model of learning with stochastic optimization algorithms (such as Stochastic Gradient Langevin Dynamics and ---to some extent--- Stochastic Gradient Descent), while it also arises in other contexts, including PAC-Bayesian theory, and sampling mechanisms. In this work we study …

New method reduces computational cost for estimating PAC-Bayes bounds.

PAC-Bayes bounds for Gibbs posteriors derived via singular learning theory.

Sharp large deviations and Gibbs conditioning for portfolio credit risk models.

The paper extends entropy maximization to multiscale settings and applies it to neural networks.

Develops a Bayesian framework for portfolio choice with a new posterior distribution.

The sensitivity to risk that most people (hence, financial operators) feel affects the dynamics of financial transactions. Here we present an approach to this problem based on a current generalization of Boltzmann-Gibbs statistical mechanics.

Study risk-sensitive market making with entropy regularization for better quote control.

In this work, we propose a PAC-Bayes bound for the generalization risk of the Gibbs classifier in the multi-class classification framework. The novelty of our work is the critical use of the confusion matrix of a classifier as an error measure; this puts our contribution in the line of work aiming at dealing with perfo…

We analyze Gibbs-based transfer learning algorithms using information theory.

We propose a categorical data synthesizer with a quantifiable disclosure risk. Our algorithm, named Perturbed Gibbs Sampler, can handle high-dimensional categorical data that are often intractable to represent as contingency tables. The algorithm extends a multiple imputation strategy for fully synthetic data by utiliz…

Decentralized learning achieves centralized performance via Gibbs measures.

The PAC-Bayesian approach is a powerful set of techniques to derive non- asymptotic risk bounds for random estimators. The corresponding optimal distribution of estimators, usually called the Gibbs posterior, is unfortunately intractable. One may sample from it using Markov chain Monte Carlo, but this is often too slow…

In this work, we propose a model for estimating volatility from financial time series, extending the non-Gaussian family of space-state models with exact marginal likelihood proposed by Gamerman, Santos and Franco (2013). On the literature there are models focused on estimating financial assets risk, however, most of t…

Paper shows how meta-learning can reduce prior learning cost.

We show that Entropy-SGD (Chaudhari et al., 2017), when viewed as a learning algorithm, optimizes a PAC-Bayes bound on the risk of a Gibbs (posterior) classifier, i.e., a randomized classifier obtained by a risk-sensitive perturbation of the weights of a learned classifier. Entropy-SGD works by optimizing the bound's p…

We derive PAC-Bayesian learning guarantees for heavy-tailed losses, and obtain a novel optimal Gibbs posterior which enjoys finite-sample excess risk bounds at logarithmic confidence. Our core technique itself makes use of PAC-Bayesian inequalities in order to derive a robust risk estimator, which by design is easy to …

We derive generalization and excess risk bounds for neural nets using a family of complexity measures based on a multilevel relative entropy. The bounds are obtained by introducing the notion of generated hierarchical coverings of neural nets and by using the technique of chaining mutual information introduced in Asadi…

The Gibbs sampler is one of the most popular algorithms for inference in statistical models. In this paper, we introduce a herding variant of this algorithm, called herded Gibbs, that is entirely deterministic. We prove that herded Gibbs has an $O(1/T)$ convergence rate for models with independent variables and for ful…

Gibbs sampling is a Markov chain Monte Carlo method that is often used for learning and inference on graphical models. Minibatching, in which a small random subset of the graph is used at each iteration, can help make Gibbs sampling scale to large graphical models by reducing its computational cost. In this paper, we p…

We propose Lomax delegate racing (LDR) to explicitly model the mechanism of survival under competing risks and to interpret how the covariates accelerate or decelerate the time to event. LDR explains non-monotonic covariate effects by racing a potentially infinite number of sub-risks, and consequently relaxes the ubiqu…

We prove a large deviation principle for a sequence of point processes defined by Gibbs probability measures on a Polish space. This is obtained as a consequence of a more general Laplace principle for the non-normalized Gibbs measures. We consider three main applications: Conditional Gibbs measures on compact spaces, …

We develop a framework for approximating collapsed Gibbs sampling in generative latent variable cluster models. Collapsed Gibbs is a popular MCMC method, which integrates out variables in the posterior to improve mixing. Unfortunately for many complex models, integrating out these variables is either analytically or co…

The pairwise influence matrix of Dobrushin has long been used as an analytical tool to bound the rate of convergence of Gibbs sampling. In this work, we use Dobrushin influence as the basis of a practical tool to certify and efficiently improve the quality of a discrete Gibbs sampler. Our Dobrushin-optimized Gibbs samp…

We review a simple model of closed economy, where the economic agents make money transactions and a saving criterion is present. We observe the Gibbs distribution for zero saving propensity, and non-Gibbs distributions otherwise. While the exact solution in the case of zero saving propensity is already known to be give…

The study uncovers universality laws for Gaussian mixtures in generalized linear models.

For large scale on-line inference problems the update strategy is critical for performance. We derive an adaptive scan Gibbs sampler that optimizes the update frequency by selecting an optimum mini-batch size. We demonstrate performance of our adaptive batch-size Gibbs sampler by comparing it against the collapsed Gibb…

New Gibbs sampling method improves MCMC efficiency.

We prove a generalization of the fundamental inequality of Guivarc'h relating entropy, drift and critical exponent to Gibbs measures on geometrically finite quotients of CAT(-1) metric spaces. For random walks with finite superexponential moment, we show that the equality is achieved if and only if the Gibbs density is…

Gibbs sampler contracts entropy under strong log-concavity, improving mixing time.

The notion of Berman-Gibbs stability was originally introduced by Robert Berman for $\mathbb{Q}$-Fano varieties $X$. We show that the pair $(X, -K_X)$ is K-stable (resp. K-semistable) provided that $X$ is Berman-Gibbs stable (resp. semistable).

Souriau studies Gibbs states for symplectic manifolds with group actions.

New model estimates Gibbs free energies using machine learning and isobaric-isothermal flows.

Modified Gibbs-Helmholtz equation geometric models for thermodynamics.

We enhance conformal prediction for risk-averse decisions with action-conditional guarantees.

Study on Metropolis-within-Gibbs schemes for high-dimensional Bayesian models.

Introduces HMC method for sampling Gibbs densities.

New method improves uncertainty quantification in latent variable models.

DiGS improves sampling from multi-modal distributions.

The Gibbs sampler is a particularly popular Markov chain used for learning and inference problems in Graphical Models (GMs). These tasks are computationally intractable in general, and the Gibbs sampler often suffers from slow mixing. In this paper, we study the Swendsen-Wang dynamics which is a more sophisticated Mark…

Improved MALA method for neural networks uncertainty quantification.

This work analyzes Gibbs samplers for Bayesian hierarchical models without dimensionality constraints.

Bayesian inference for Levy density with Gibbs posterior in discrete sampling.

Gibbs sampling is the de facto Markov chain Monte Carlo method used for inference and learning on large scale graphical models. For complicated factor graphs with lots of factors, the performance of Gibbs sampling can be limited by the computational cost of executing a single update step of the Markov chain. This cost …

A new algorithm for sampling from complex distributions.

Gibbs sampler mixes quickly for certain smooth distributions.

Study non-asymptotic Langevin Monte Carlo for Gibbs distributions.