A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Paper introduces kernel deformed exponential families for sparse continuous attention.
We investigate penalized maximum log-likelihood estimation for exponential family distributions whose natural parameter resides in a reproducing kernel Hilbert space. Key to our approach is a novel technique, doubly dual embedding, that avoids computation of the partition function. This technique also allows the develo…
Paper explores duality in DPPs using embedding structure analysis.
Paper explores robust estimators for kernel exponential families using smoothed total variation distances.
We propose a novel approach for density estimation with exponential families for the case when the true density may not fall within the chosen family. Our approach augments the sufficient statistics with features designed to accumulate probability mass in the neighborhood of the observed points, resulting in a non-para…
A nonparametric family of conditional distributions is introduced, which generalizes conditional exponential families using functional parameters in a suitable RKHS. An algorithm is provided for learning the generalized natural parameter, and consistency of the estimator is established in the well specified case. In ex…
We propose Kernel Hamiltonian Monte Carlo (KMC), a gradient-free adaptive MCMC algorithm based on Hamiltonian Monte Carlo (HMC). On target densities where classical HMC is not an option due to intractable gradients, KMC adaptively learns the target's gradient structure by fitting an exponential family model in a Reprod…
The kernel exponential family is a rich class of distributions, which can be fit efficiently and with statistical guarantees by score matching. Being required to choose a priori a simple kernel such as the Gaussian, however, limits its practical applicability. We provide a scheme for learning a kernel parameterized by …
Develops kernels for matchings, overcoming computational challenges.
Unified analysis of kernel-based and locally adaptive bandit optimization methods.
We prove a conjecture about approximating Gaussian Processes on one dimension.
In this paper we de ne conditional random elds in reproducing kernel Hilbert spaces and show connections to Gaussian Process classi cation. More speci cally, we prove decomposition results for undirected graphical models and we give constructions for kernels. Finally we present e cient means of solving the optimization…
We study optimal solutions to an abstract optimization problem for measures, which is a generalization of classical variational problems in information theory and statistical physics. In the classical problems, information and relative entropy are defined using the Kullback-Leibler divergence, and for this reason optim…
We propose a fast method with statistical guarantees for learning an exponential family density model where the natural parameter is in a reproducing kernel Hilbert space, and may be infinite-dimensional. The model is learned by fitting the derivative of the log density, the score, thus avoiding the need to compute a n…
We study the mixtures of factorizing probability distributions represented as visible marginal distributions in stochastic layered networks. We take the perspective of kernel transitions of distributions, which gives a unified picture of distributed representations arising from Deep Belief Networks (DBN) and other netw…
Squared families are a new model class derived from linear transformations, offering convenient properties and universal approximation.
New algorithm speeds up polynomial kernel approximations.
Exponential family distributions are highly useful in machine learning since their calculation can be performed efficiently through natural parameters. The exponential family has recently been extended to the t-exponential family, which contains Student-t distributions as family members and thus allows us to handle noi…
Two adaptive kernel selection methods improve the accuracy of Kernelized Diffusion Maps.
New Hida-Matérn kernels enable flexible process priors and efficient GP inference.
In the context of kernel methods, the similarity between data points is encoded by the kernel function which is often defined thanks to the Euclidean distance, a common example being the squared exponential kernel. Recently, other distances relying on optimal transport theory - such as the Wasserstein distance between …
The study explores generalized divergences and exponential families with a focus on sufficient conditions and laws of large numbers.
Correspondence found between exponential families and affine Grassmannians.
Study compares exponential and power-law kernels in modeling high-frequency trading data.
Quantum kernel methods can lead to trivial models due to exponential concentration of kernel values.
New Fourier features improve high-precision approximation in large-scale problems.
A theoretical framework for non-negative matrix factorization based on generalized dual Kullback-Leibler divergence, which includes members of the exponential family of models, is proposed. A family of algorithms is developed using this framework and its convergence proven using the Expectation-Maximization algorithm. …
A faster method for density estimation using denoising score matching with random Fourier features.
We provide a classification of graphical models according to their representation as subfamilies of exponential families. Undirected graphical models with no hidden variables are linear exponential families (LEFs), directed acyclic graphical models and chain graphs with no hidden variables, including Bayesian networks …
Constructing exponential families from statistical manifolds.
We prove an exponential estimate for the asymptotics of Bergman kernels of a positive line bundle under hypotheses of bounded geometry. We give further Bergman kernel proofs of complex geometry results, such as separation of points, existence of local coordinates and holomorphic convexity by sections of positive line b…
Thompson Sampling has been demonstrated in many complex bandit models, however the theoretical guarantees available for the parametric multi-armed bandit are still limited to the Bernoulli case. Here we extend them by proving asymptotic optimality of the algorithm using the Jeffreys prior for 1-dimensional exponential …
Moment polytope of toric exponential families is a projection of a simplex.
New Thompson sampling algorithm reduces regret for exponential family bandits.
We study online learning under logarithmic loss with regular parametric models. Hedayati and Bartlett (2012b) showed that a Bayesian prediction strategy with Jeffreys prior and sequential normalized maximum likelihood (SNML) coincide and are optimal if and only if the latter is exchangeable, and if and only if the opti…
The versatility of exponential families, along with their attendant convexity properties, make them a popular and effective statistical model. A central issue is learning these models in high-dimensions, such as when there is some sparsity pattern of the optimal parameter. This work characterizes a certain strong conve…
New insights into natural exponential families improve regret bounds for bandit problems.
Kernel estimator optimally recovers function from noisy exponential Radon transform.
Kernel-based learning predicts ICU escalation from COVID-19 chest X-rays.
Extends likelihood ratio exponential families to analyze various optimization methods.
Efficient method for learning continuous exponential families beyond Gaussian.
Spectral risk measures (SRMs) belong to the family of coherent risk measures. A natural estimator for the class of SRMs has the form of L-statistics. Various authors have studied and derived the asymptotic properties of the empirical estimator of SRM. We propose a kernel based estimator of SRM. We investigate the large…
Maximum likelihood learning with exponential families leads to moment-matching of the sufficient statistics, a classic result. This can be generalized to conditional exponential families and/or when there are hidden data. This document gives a first-principles explanation of these generalized moment-matching conditions…
EFDA extends LDA to non-Gaussian models using exponential families.
We propose a data-driven approach to quantify the uncertainty of models constructed by kernel methods. Our approach minimizes the needed distributional assumptions, hence, instead of working with, for example, Gaussian processes or exponential families, it only requires knowledge about some mild regularity of the measu…
New bounds for score matching in polynomial exponential families.
Exponential family extensions of principal component analysis (EPCA) have received a considerable amount of attention in recent years, demonstrating the growing need for basic modeling tools that do not assume the squared loss or Gaussian distribution. We extend the EPCA model toolbox by presenting the first exponentia…
New hyperbolic manifolds show exponential homology torsion growth.