Exponential family distributions are highly useful in machine learning since their calculation can be performed efficiently through natural parameters. The exponential family has recently been extended to the t-exponential family, which contains Student-t distributions as family members and thus allows us to handle noi…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Exponential family distributions”
Finite-Time Regret of Thompson Sampling Algorithms for Exponential Family Multi-Armed Banditsstat.ML
New Thompson sampling algorithm reduces regret for exponential family bandits.
problem Minimizing regret in multi-armed bandit problems with exponential family rewards.
method Proposes ExpTS and ExpTS algorithms using novel sampling distributions.
result Minimizes both finite-time and asymptotic regret for exponential family rewards.
We provide a classification of graphical models according to their representation as subfamilies of exponential families. Undirected graphical models with no hidden variables are linear exponential families (LEFs), directed acyclic graphical models and chain graphs with no hidden variables, including Bayesian networks …
Extends likelihood ratio exponential families to analyze various optimization methods.
problem Analyzing optimization methods like rate-distortion and information bottleneck.
method Linking geometric mixture paths to exponential families and using hypothesis testing.
result Provides a common mathematical framework for understanding these methods.
Thompson Sampling has been demonstrated in many complex bandit models, however the theoretical guarantees available for the parametric multi-armed bandit are still limited to the Bernoulli case. Here we extend them by proving asymptotic optimality of the algorithm using the Jeffreys prior for 1-dimensional exponential …
Undirected graphical models, or Markov networks, are a popular class of statistical models, used in a wide variety of applications. Popular instances of this class include Gaussian graphical models and Ising models. In many settings, however, it might not be clear which subclass of graphical models to use, particularly…
NatPN provides fast, accurate uncertainty estimation for exponential family distributions.
problem Uncertainty in machine learning models.
method NatPN uses Normalizing Flows to fit a single density in a latent space, updating predictions based on likelihood.
result NatPN delivers competitive performance in classification, regression, and count prediction tasks.
Flexible mean field variational inference using mixtures of non-overlapping exponential familiesmath.ST
Improves variational inference for sparse models using mixtures of exponential families.
problem Intractability of posterior distributions in Bayesian sparse models.
method Flexible mean field variational inference using mixtures of non-overlapping exponential families.
result Mixtures of exponential families with non-overlapping support form an exponential family, enabling analytical updates.
In this paper, a Bayesian inference technique based on Taylor series approximation of the logarithm of the likelihood function is presented. The proposed approximation is devised for the case, where the prior distribution belongs to the exponential family of distributions. The logarithm of the likelihood function is li…
New method for robust PCA with exponential family distributions.
problem Recovering low-rank structure from data matrices with outliers.
method Alternating Direction Method of Multipliers for .
result Demonstrated effectiveness in steel sheet defect detection and crime activity monitoring.
Extends diffusion models to handle exponential family distributions for inverse problems.
problem Intractability of likelihood score for non-Gaussian observations.
method Evidence trick to approximate likelihood score for exponential family distributions.
result Effective Bayesian inference on complex Poisson processes and malaria prevalence prediction.
We consider the design of prediction market mechanisms known as automated market makers. We show that we can design these mechanisms via the mold of \emph{exponential family distributions}, a popular and well-studied probability distribution template used in statistics. We give a full development of this relationship a…
We study the problem of finding the smallest such that every element of an exponential family can be written as a mixture of elements of another exponential family. We propose an approach based on coverings and packings of the face lattice of the corresponding convex support polytopes and results from coding th…
Approximating exponential family models (not single distributions) with a two-network architecturestat.ML
Recently much attention has been paid to deep generative models, since they have been used to great success for variational inference, generation of complex data types, and more. In most all of these settings, the goal has been to find a particular member of that model family: optimized parameters index a distribution …
We study online learning under logarithmic loss with regular parametric models. Hedayati and Bartlett (2012b) showed that a Bayesian prediction strategy with Jeffreys prior and sequential normalized maximum likelihood (SNML) coincide and are optimal if and only if the latter is exchangeable, and if and only if the opti…
We propose a novel approach for density estimation with exponential families for the case when the true density may not fall within the chosen family. Our approach augments the sufficient statistics with features designed to accumulate probability mass in the neighborhood of the observed points, resulting in a non-para…
Efficiently learns exponential family distributions with i.i.d. samples.
problem Learning natural parameters of truncated exponential families efficiently.
method Proposes a novel loss function and computationally efficient estimator.
result Achieves optimal sample complexity and asymptotic normality.
Evidence for the exponential distribution of income in the USAcond-mat.stat-mech
Using tax and census data, we demonstrate that the distribution of individual income in the USA is exponential. Our calculated Lorenz curve without fitting parameters and Gini coefficient 1/2 agree well with the data. From the individual income distribution, we derive the distribution function of income for families wi…
Optimal learning for parametric prophet inequalities with exponential-type distributions
problem Learning in prophet inequalities with unknown parameters
method Confidence-based dynamic-programming policy
result Achieves optimal asymptotic competitive ratio using online observations
EFDA extends LDA to non-Gaussian models using exponential families.
problem Classifying non-Gaussian data with LDA's limitations.
method EFDA uses exponential families to derive closed-form estimators for natural parameters and a linear decision rule.
result EFDA matches LDA's accuracy while reducing ECE by 2-6x, proving asymptotic calibration and efficiency.
Inference for normal and Monte Carlo distributions using minimum relative entropy.
problem Inference from partial information on expectations and covariances.
method Minimum relative entropy sub-manifolds, analytical formulas, Monte Carlo simulations.
result Improved numerical implementation for inference from partial information.
Efficient method for learning continuous exponential families beyond Gaussian.
problem Learning continuous exponential families with unbounded support.
method Interaction Screening approach for scalable learning of continuous graphical models.
result Our estimator maintains similar accuracy and sample complexity scalings compared to alternative approaches, while improving run-time.
New insights into natural exponential families improve regret bounds for bandit problems.
problem Improving regret bounds for bandit problems with subexponential tails.
method Proving self-concordance for natural exponential families and applying to bandits.
result Optimistic algorithms for generalized linear bandits have second-order regret bounds that are free of an exponential dependence on problem parameters.
Representing networks in a low dimensional latent space is a crucial task with many interesting applications in graph learning problems, such as link prediction and node classification. A widely applied network representation learning paradigm is based on the combination of random walks for sampling context nodes and t…
Exponential family extensions of principal component analysis (EPCA) have received a considerable amount of attention in recent years, demonstrating the growing need for basic modeling tools that do not assume the squared loss or Gaussian distribution. We extend the EPCA model toolbox by presenting the first exponentia…
Unified view on learning unnormalized distributions using NCE.
problem Learning unnormalized distributions across different communities.
method Noise-Contrastive Estimation (NCE) for a unified perspective.
result Established finite-sample convergence rates for exponential families.
New method uses neural exponential families for likelihood-free inference.
problem Bayesian Likelihood-Free Inference with intractable likelihood.
method Score Matching neural conditional exponential families for approximate likelihood.
result State-of-the-art performance in posterior sampling for intractable likelihood models.
New bounds for score matching in polynomial exponential families.
problem Understanding the sample complexity of score matching for polynomial exponential families.
method Non-asymptotic sample complexity analysis for score matching.
result First finite sample bounds for score matching in polynomial exponential families.
SMRL uses score matching for efficient RL with exponential family models.
problem Efficient reinforcement learning with unknown transition models.
method Score matching for parameter estimation of exponential family models.
result Achieves online regret of .
Paper defines ε-Safe Decision Regions for exponential family distributions and approximates them for unbalanced data.
problem Need probabilistic guarantees for reliable predictions in machine learning.
method Formalizes ε-Safe Decision Regions, proves their form for exponential family distributions, and develops Multi Cost SVM for unbalanced data.
result Formal definition and analytical determination of ε-Safe Decision Regions for exponential family distributions.
We present Vector-Space Markov Random Fields (VS-MRFs), a novel class of undirected graphical models where each variable can belong to an arbitrary vector space. VS-MRFs generalize a recent line of work on scalar-valued, uni-parameter exponential family and mixed graphical models, thereby greatly broadening the class o…
Generic Unsupervised Optimization for a Latent Variable Model With Exponential Family Observablescs.LG
New LVMs optimize any exponential family distribution without specific assumptions.
problem Optimizing latent variable models with non-Gaussian observables.
method Generic optimization using EM approach for exponential family distributions.
result Concise parameter update equations applicable to various data types.
MLE and CVE are equivalent under exponential families, leading to faster and more stable EM algorithms.
problem Finding maximum likelihood estimators (MLE) efficiently and stably.
method Proved equivalence between MLE and CVE under exponential families, leading to an EM algorithm.
result EM algorithm achieves the same asymptotic variance as MLE and is faster and more stable.
This note investigates a conjugate class for the Dirichlet distribution class in the exponential family.
In a range of fields including the geosciences, molecular biology, robotics and computer vision, one encounters problems that involve random variables on manifolds. Currently, there is a lack of flexible probabilistic models on manifolds that are fast and easy to train. We define an extremely flexible class of exponent…
A nonparametric family of conditional distributions is introduced, which generalizes conditional exponential families using functional parameters in a suitable RKHS. An algorithm is provided for learning the generalized natural parameter, and consistency of the estimator is established in the well specified case. In ex…
Paper explores robust estimators for kernel exponential families using smoothed total variation distances.
problem Outliers can severely impact classical estimators in statistical inference.
method Proposes smoothed total variation (STV) distance as a class of IPMs for robust estimation of kernel exponential families.
result STV-based estimators are robust against distribution contamination for kernel exponential families.
A Sufficient Statistics Construction of Bayesian Nonparametric Exponential Family Conjugate Modelscs.LG
Conjugate pairs of distributions over infinite dimensional spaces are prominent in statistical learning theory, particularly due to the widespread adoption of Bayesian nonparametric methodologies for a host of models and applications. Much of the existing literature in the learning community focuses on processes posses…
BART is extended to handle various response variables.
problem Modeling nonlinear regression functions for diverse response types.
method Generalized Bayesian Additive Regression Trees (GBART) for exponential family distributions.
result The posterior concentrates at a minimax rate for certain response distributions.
EM algorithm converges in KL divergence for exponential families via mirror descent.
problem Lack of understanding of EM's non-asymptotic convergence properties.
method Viewing EM as a mirror descent algorithm, showing convergence rates in KL divergence.
result KL divergence rates for EM in exponential families, invariant to parametrization.
Paper extends sparse alternatives to softmax for continuous domains, enabling efficient attention mechanisms.
problem Efficiently assigning zero probability to irrelevant categories in continuous domains.
method Extend alpha-entmax to continuous domains, introducing continuous-domain attention mechanisms.
result Continuous attention allows attending to time intervals and compact regions, improving text classification, machine translation, and visual question answering.
Exponential families are a particular class of statistical manifolds which are particularly important in statistical inference, and which appear very frequently in statistics. For example, the set of normal distributions, with mean μ and deviation σ, form a 2-dimensional exponential family. In this paper, we show that …
Boosting improves data fitting while maintaining fairness guarantees.
problem Ensuring fairness in data preprocessing.
method Boosting algorithm to learn sufficient statistics of exponential families.
result The learned distribution maintains fairness guarantees while fitting the data better.
Distributed learning of probabilistic models from multiple data repositories with minimum communication is increasingly important. We study a simple communication-efficient learning framework that first calculates the local maximum likelihood estimates (MLE) based on the data subsets, and then combines the local MLEs t…
Geometric Gaussian approximations capture any distribution.
problem Approximating complex probability distributions.
method Geometric Gaussian approximations through diffeomorphisms or exponential maps.
result Geometric Gaussian approximations are universal, capturing any distribution.
Unified framework for understanding TVO and improving model learning.
problem Improving the tightness and efficiency of variational inference bounds.
method Exponential family interpretation and equal spacing in moment parameters.
result Unified framework and improved gradient estimator for TVO.
New DP framework using data truncation for efficient estimation.
problem Differential privacy in unbounded data support.
method Data truncation, exponential family distributions, maximum likelihood estimation, DP stochastic gradient descent.
result Near-optimal sample complexity for Gaussian mean and covariance estimation.
On the Geometry of Discrete Exponential Families with Application to Exponential Random Graph Modelsstat.ML
There has been an explosion of interest in statistical models for analyzing network data, and considerable interest in the class of exponential random graph (ERG) models, especially in connection with difficulties in computing maximum likelihood estimates. The issues associated with these difficulties relate to the bro…