arXiv research

New algorithms improve convergence rates for non-log-concave sampling and log-partition estimation.

Bounds on the log partition function are important in a variety of contexts, including approximate inference, model fitting, decision theory, and large deviations analysis. We introduce a new class of upper bounds on the log partition function, based on convex combinations of distributions in the exponential domain, th…

Paper tackles parameter learning for log-supermodular models, improving on existing bounds.

Paper proposes a new method to learn EBMs and their partition function.

Neural method improves learning and inference for undirected models.

Recent research has made significant progress on the problem of bounding log partition functions for exponential family graphical models. Such bounds have associated dual parameters that are often used as heuristic estimates of the marginal probabilities required in inference and learning. However these variational est…

We propose a relaxation-based approximate inference algorithm that samples near-MAP configurations of a binary pairwise Markov random field. We experiment on MAP inference tasks in several restricted Boltzmann machines. We also use our underlying sampler to estimate the log-partition function of restricted Boltzmann ma…

Constant-time approximation of partition functions for dense models.

DS-MLR tackles multinomial logistic regression scaling issues.

Paper optimizes change detection in unnormalized distributions.

For many large undirected models that arise in real-world applications, exact maximumlikelihood training is intractable, because it requires computing marginal distributions of the model. Conditional training is even more difficult, because the partition function depends not only on the parameters, but also on the obse…

We analyze a reweighted version of the Kikuchi approximation for estimating the log partition function of a product distribution defined over a region graph. We establish sufficient conditions for the concavity of our reweighted objective function in terms of weight assignments in the Kikuchi expansion, and show that a…

This paper introduces the Metric-Free Natural Gradient (MFNG) algorithm for training Boltzmann Machines. Similar in spirit to the Hessian-Free method of Martens [8], our algorithm belongs to the family of truncated Newton methods and exploits an efficient matrix-vector product to avoid explicitely storing the natural g…

We introduce a new class of lower bounds on the log partition function of a Markov random field which makes use of a reversed Jensen's inequality. In particular, our method approximates the intractable distribution using a linear combination of spanning trees with negative weights. This technique is a lower-bound count…

A robust multiclass logistic regression method using Tsallis divergence.

It is known that fixed points of loopy belief propagation (BP) correspond to stationary points of the Bethe variational problem, where we minimize the Bethe free energy subject to normalization and marginalization constraints. Unfortunately, this does not entirely explain BP because BP is a dual rather than primal algo…

Paper improves variational inference on Boolean hypercube using quantum methods.

AdVIL improves inference and learning for MRFs with minimal assumptions.

New methods improve on the Gumbel trick for sampling and estimating partition functions.

Geometric analysis improves convergence of variational inference.

Entropy analysis via kernel methods for probabilistic inference.

We develop efficient methods to approximate maximum entropy distributions for pairwise moments.

Optimizes submodular extensions for efficient marginal estimation.

We give polynomial-time algorithms for the exact computation of lowest-energy (ground) states, worst margin violators, log partition functions, and marginal edge probabilities in certain binary undirected graphical models. Our approach provides an interesting alternative to the well-known graph cut paradigm in that it …

We use SMC with twist functions to improve probabilistic inference in LLMs.

Looking for associations among multiple variables is a topical issue in statistics due to the increasing amount of data encountered in biology, medicine and many other domains involving statistical applications. Graphical models have recently gained popularity for this purpose in the statistical literature. Following t…

Extends likelihood ratio exponential families to analyze various optimization methods.

Paper analyzes latent space geometry in generative models using Fisher information.

New bounds on multi-armed bandit probabilities for exponential families.

The paper solves portfolio selection using Rényi divergence and optimization.

GRM models k-way dependencies in univariate exponential families.

Paper addresses inefficiency in converting EFGs to NFGs for learning.

In this paper, we consider an infinite dimensional exponential family, $\mathcal{P}$ of probability densities, which are parametrized by functions in a reproducing kernel Hilbert space, $H$ and show it to be quite rich in the sense that a broad class of densities on $\mathbb{R}^d$ can be approximated arbitrarily well i…

New MI bounds improve estimation in deep generative models.

New neural network models for complex functional data analysis.

Knot signature function defined and conditions for its existence are given.

Distance function to a finite set is a topological Morse function.

Introduces new weighted floating functions and affine surface areas.

Develops methods for selecting and estimating smooth functional coefficients in high-dimensional multivariate functional data.

The paper introduces geodesic φ-convex functions and their properties.

Neural networks can approximate functionals on RKHS with error bounds.

FFBO optimizes functions as inputs and outputs, improving on existing BO methods.

Analyzes properties of transnormal Finsler functions on compact manifolds.

The study explores the Dehn functions of Kähler groups and their properties.

Chirped sinosoids and interferometric phase plots are functions that are not periodic, but are the composition of a smooth function and a periodic function. These functions functions factor into a pair of maps: from their domain to a circle, and from a circle to their codomain. One can easily imagine replacing the circ…

The Fridman function is bounded by the injectivity radius for certain hyperbolic manifolds.

Optimally estimates a functional using nuisance function tuning and sample splitting.

The paper extends mixability theory to function-valued forecasts, proving various loss functions are mixable.